Distribute LLVM bitcode linker as a preview component by kjetilkjeka · Pull Request #123423 · rust-lang/rust

@rustbot rustbot added S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

T-bootstrap

Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

labels

Apr 3, 2024

clubby789

Mark-Simulacrum

@rustbot rustbot added the O-wasi

Operating system: Wasi, Webassembly System Interface

label

Apr 15, 2024

@bors bors added S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

and removed S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

labels

Apr 15, 2024

jieyouxu added a commit to jieyouxu/rust that referenced this pull request

Apr 15, 2024
…mponent, r=Mark-Simulacrum

Distribute LLVM bitcode linker as a preview component

The self-contained LLVM bitcode linker was recently added in rust-lang#117458. It is currently only in use to link the Nvidia ptx assembly tests when running rustc tests (local or CI). In fact, the linker itself is currently only usable for the `nvptx64-nvidia-cuda` target, but more targets will be supported in the future.

The reason a new linker was needed for the ptx format is that the [old one](https://github.com/denzp/rust-ptx-linker) has not been updated the last few years. It worked fine for a while, but as LLVM changed it broke and the nvptx tests was [disabled in rustc back in 2019](rust-lang@f8f9a28). It was ad-hoc patched and have been used in a sub-optimal state by the community until now.

If this PR is merged, the LLVM bitcode linker will be distributed as a preview component that can be used as a replacement for the old ptx-linker for development in addition to rustc tests. In addition to installing the `llvm-bitcode-linker` component, also the `llvm-tools` component must be installed as the `llvm-bitcode-linker` works by calling llvm tools.

Even though the LLVM bitcode linker is in its early stages it already now provides a lot of value over the old ptx-linker just by working and using up-to-date llvm tooling. By shipping it as a component it will be easier to gather user experience and improving it.

`@petrochenkov` when installing as a component it will be installed in the self-contained folder and will not work with `-Clink-self-contained=no` (although for some reason I expect to be a bug `-Clink-self-contained=-linker` doesn't properly disable it). However, when building using `x.py build` it will be placed in the `lib/rustlib/<target>/bin` directory and will be available for internal tests even if `-Clink-self-contained=no` is passed.

CC: `@Mark-Simulacrum` as I very briefly discussed it with you some months ago https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/.E2.9C.94.20How.20to.20ship.20a.20new.20tool.20.28embedded.20linker.29.20to.20users.3F

This was referenced

Apr 15, 2024

bors added a commit to rust-lang-ci/rust that referenced this pull request

Apr 15, 2024

rust-timer added a commit to rust-lang-ci/rust that referenced this pull request

Apr 15, 2024
Rollup merge of rust-lang#123423 - kjetilkjeka:llvm_bitcode_linker_component, r=Mark-Simulacrum

Distribute LLVM bitcode linker as a preview component

The self-contained LLVM bitcode linker was recently added in rust-lang#117458. It is currently only in use to link the Nvidia ptx assembly tests when running rustc tests (local or CI). In fact, the linker itself is currently only usable for the `nvptx64-nvidia-cuda` target, but more targets will be supported in the future.

The reason a new linker was needed for the ptx format is that the [old one](https://github.com/denzp/rust-ptx-linker) has not been updated the last few years. It worked fine for a while, but as LLVM changed it broke and the nvptx tests was [disabled in rustc back in 2019](rust-lang@f8f9a28). It was ad-hoc patched and have been used in a sub-optimal state by the community until now.

If this PR is merged, the LLVM bitcode linker will be distributed as a preview component that can be used as a replacement for the old ptx-linker for development in addition to rustc tests. In addition to installing the `llvm-bitcode-linker` component, also the `llvm-tools` component must be installed as the `llvm-bitcode-linker` works by calling llvm tools.

Even though the LLVM bitcode linker is in its early stages it already now provides a lot of value over the old ptx-linker just by working and using up-to-date llvm tooling. By shipping it as a component it will be easier to gather user experience and improving it.

``@petrochenkov`` when installing as a component it will be installed in the self-contained folder and will not work with `-Clink-self-contained=no` (although for some reason I expect to be a bug `-Clink-self-contained=-linker` doesn't properly disable it). However, when building using `x.py build` it will be placed in the `lib/rustlib/<target>/bin` directory and will be available for internal tests even if `-Clink-self-contained=no` is passed.

CC: ``@Mark-Simulacrum`` as I very briefly discussed it with you some months ago https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/.E2.9C.94.20How.20to.20ship.20a.20new.20tool.20.28embedded.20linker.29.20to.20users.3F

jieyouxu added a commit to jieyouxu/rust that referenced this pull request

Apr 20, 2024
…ild_manifest, r=Mark-Simulacrum

Add llvm-bitcode-linker to build manifest

When creating rust-lang#123423 I didn't realize I also had to add the new component to the build-manifest. This PR finishes the work of adding it, by also adding it to the build manifest.

r? `@Mark-Simulacrum`

jieyouxu added a commit to jieyouxu/rust that referenced this pull request

Apr 20, 2024
…ild_manifest, r=Mark-Simulacrum

Add llvm-bitcode-linker to build manifest

When creating rust-lang#123423 I didn't realize I also had to add the new component to the build-manifest. This PR finishes the work of adding it, by also adding it to the build manifest.

r? ``@Mark-Simulacrum``

rust-timer added a commit to rust-lang-ci/rust that referenced this pull request

Apr 21, 2024
Rollup merge of rust-lang#124071 - kjetilkjeka:llvm_bitcode_linker_build_manifest, r=Mark-Simulacrum

Add llvm-bitcode-linker to build manifest

When creating rust-lang#123423 I didn't realize I also had to add the new component to the build-manifest. This PR finishes the work of adding it, by also adding it to the build manifest.

r? ``@Mark-Simulacrum``