spir64_gen support¶

Compilation behaviors involving AOT for GPU involve an additional call to the OpenCL Offline compiler (OCLOC). This call occurs after the post-link step performed by sycl-post-link and the SPIR-V translation step which is done by llvm-spirv. Additional options passed by the user via the -Xsycl-target-backend=spir64_gen <opts> command as well as the implied options set via target options such as -fsycl-targets=intel_gpu_skl will be processed by a new options to the wrapper, --gpu-tool-arg=<arg>

To support multiple target specifications, for instance: -fsycl-targets=intel_gpu_skl,intel_gpu_pvc, multiple --gpu-tool-arg options can be passed on the command line. Each instance will be considered a separate OCLOC call passing along the <args> as options to the OCLOC call. The compiler driver will be responsible for putting together the full option list to be passed along.

-fsycl -fsycl-targets=spir64_gen,intel_gpu_skl -Xsycl-target-backend=spir64_gen “-device pvc -options -extraopt_pvc” -Xsycl-target-backend=intel_gpu_skl “-options -extraopt_skl”

Example: spir64_gen enabling options

–gpu-tool-arg=”-device pvc -options extraopt_pvc” –gpu-tool-arg=”-options -extraopt_skl”

Example: clang-linker-wrapper options

Each OCLOC call will be represented as a separate device binary that is individually wrapped and linked into the final executable.

Additionally, the syntax can be expanded to enable the ability to pass specific options to a specific device GPU target for spir64_gen. The syntax will resemble --gpu-tool-arg=<arch> <arg>. This corresponds to the existing option syntax of -fsycl-targets=intel_gpu_arch where arch can be a fixed set of targets.

–offload-arch¶

For SYCL offloading to Intel GPUs, Intel CPUs, NVidia and AMD GPUs, specify the device architecture using --offload-arch option. For instance --offload-arch=sm_80 to target an NVidia Tesla A100, --offload-arch=gfx90a to target an AMD Instinct MI250X, or --offload-arch=sm_80,gfx90a to target both.

For Intel Graphics AOT target, valid values for --offload-arch are mapped to valid device names accepted by OCLOC (the Intel GPU AOT compiler) via the -device option.

SYCL offloading with --offload-arch for Intel CPUs and Intel GPUs is currently enabled only with --offload-new-driver option.

Example:

$ clang++ -fsycl -offload-arch=bdw --offload-new-driver -c foo.cpp // SYCL AOT for Intel GPU.
$ clang++ -fsycl -offload-arch=broadwell --offload-new-driver -c foo.cpp // SYCL AOT for Intel CPU.

The following table shows a mapping of the accepted values for --offload-arch to enable SYCL offloading to Intel GPUs and the corresponding -device value passed to OCLOC.

nvptx64-nvidia-cuda support¶

For SYCL offloading to NVidia GPUs using --offload-arch option, the following table lists the accepted values.

amdgcn-amd-amdhsa support¶

For SYCL offloading to AMD GPUs using --offload-arch option, the following table lists the accepted values.

spir64_fpga support¶

Compilation behaviors involving AOT for FPGA involve an additional call to either aoc (for Hardware/Simulation) or opencl-aot (for Emulation). This call occurs after the post-link step performed by sycl-post-link and the SPIR-V translation step performed by llvm-spirv. Additional options passed by the user via the -Xsycl-target-backend=spir64_fpga <opts> command will be processed by a new options to the wrapper, --fpga-tool-arg=<arg>

The FPGA target also has support for additional generated binaries that contain intermediate files specific for FPGA. These binaries (aoco, aocr and aocx) can reside in archives and are treated differently than traditional device binaries.

Generation of the AOCR and AOCX type binary is triggered by the command line option -fsycl-link, where -fsycl-link=image creates AOCX archives and -fsycl-link=early generates AOCR archives. The files generated by these options are handled in a specific manner when encountered.

Any archive with an AOCR type device binary will have the AOCR binary extracted and passed to aoc to produce an AOCX final image. This final image is wrapped and added to the final binary during the host link. The use of -fsycl-link=image with an AOCR binary will create an AOCX based archive instead of completing the host link. Any archive with an AOCX type device binary skips the aoc step and is wrapped and added to the final binary during the host link. Archives with any AOCO device binaries are extracted and passed through to aoc -library-list=<listfile>

As the clang-linker-wrapper is responsible for understanding the archives that are added on the command line, it will need to know when to look for these unique device binaries based on the expected compilation targets. The behavior of creating the AOCX/AOCR type archive will be triggered via an additional command line option specified by the driver when -fsycl-link options are used. The --fpga-link=<type> option will tell the wrapper when these handlings need to occur.

When using the -fintelfpga option to enable AOT for FPGA, there are additional expectations during the compilation. Use of the option will enable debug generation and also generate dependency information. The dependency generation should be packaged along with the device binary for use during the link phase. It is expected that the full fat object, containing host, device and dependency file is generated before being passed to the link phase. The dependency information is only used when compiling for hardware.

The clang-linker-wrapper tool will be responsible to determine which FPGA tool is being used during the AOT device compilation phase. The use of -simulation or -hardware as passed in by --fpga-tool-arg signifies which tool is used.

spir64_x86_64 support¶

Compilation behaviors involving AOT for CPU involve an additional call to opencl-aot. This call occurs after the post-link step performed by sycl-post-link and the SPIR-V translation step performed by llvm-spirv. Additional options passed by the user via the -Xsycl-target-backend=spir64_x86_64 <opts> command will be processed by a new option to the wrapper, --cpu-tool-arg=<arg>

Similar to SYCL offloading to Intel GPUs using --offload-arch, SYCL AOT for Intel CPUs will also leverage the --offload-arch option. The valid CPU device names accepted for --offload-arch are CPU names from clang -march. These names are more verbose, and do not overlap with the Intel GPU names. These user input CPU names are mapped to the corresponding opencl-aot -march option.

The following table shows a mapping of the accepted values for --offload-arch to enable SYCL offloading to Intel CPUs and the corresponding -march value passed to opencl-aot.

Beyond llvm-foreach and similar job hiding tools¶

Tools like llvm-foreach, file-table-tform, spirv-to-ir-wrapper were all introduced to provide a way to manipulate behaviors that could only be determined at runtime of the compiler toolchain. These were needed to work around the fact that the toolchain commands constructed by the driver is a fixed state of commands.

Moving the functionality into clang-linker-wrapper presents the opportunity step away from the static command construction and create the call chain on the fly based on real time output from corresponding tools being called.