Do not copy .rmeta files into the sysroot of the build compiler during check of rustc/std by Kobzol · Pull Request #144252 · rust-lang/rust

@rustbot rustbot added S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

T-bootstrap

Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

labels

Jul 21, 2025

@Kobzol Kobzol changed the title Do not copy .rmeta files into the sysroot of the build compiler during hcekc Do not copy .rmeta files into the sysroot of the build compiler during check

Jul 21, 2025

@Kobzol

This helps to avoid polluting the sysroot of the build compiler.

@Kobzol

@Kobzol Kobzol changed the title Do not copy .rmeta files into the sysroot of the build compiler during check Do not copy .rmeta files into the sysroot of the build compiler during check of rustc/std

Aug 16, 2025

@Kobzol Kobzol marked this pull request as ready for review

August 16, 2025 10:39

@rustbot rustbot added S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

and removed S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

labels

Aug 16, 2025

@rustbot rustbot added S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

and removed S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

labels

Aug 18, 2025

@Kobzol

@rustbot rustbot added S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

and removed S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

labels

Aug 18, 2025

jieyouxu

@rustbot rustbot added S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

and removed S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

labels

Aug 18, 2025

@Kobzol

@rustbot rustbot added S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

and removed S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

labels

Aug 18, 2025

@bors bors added S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

and removed S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

labels

Aug 18, 2025

Zalathar added a commit to Zalathar/rust that referenced this pull request

Aug 19, 2025
Do not copy .rmeta files into the sysroot of the build compiler during check of rustc/std

Before, when bootstrap did a check build of rustc stage N (with a build compiler that was stage N-1), it automatically copied the resulting `.rmeta` artifacts into the sysroot of the stage N-1 build compiler, so that stage N `rustc_private` tools such as `miri` could be compiled using the stage N-1 build compiler. This has a number of issues:

- It was done unconditionally, even if no `rustc_private` tools were actually built.
- If we did a check and a build of the same stage compiler in the same bootstrap invocation, the generated rmeta and rlib files could clash. This is also why you can see that `check::Std` actually doesn't copy the artifacts anymore (which forced us to build std instead of just checking it in a bunch of `Check` steps).
- It was polluting the sysroot of the build compiler. This is especially annoying for the stage 0 compiler, because we are forced to create an artificial sysroot for it, so that we can copy new stuff into it.
- It was very implicit in bootstrap.

Based on suggestions by `@cuviper` and `@bjorn3,` I tried to change how this behaves. Instead of copying the rmeta artifacts into the sysroot of the build compiler (from where they would be loaded implicitly), they are now stored in a separate transient bootstrap build directory, and they are then explicitly passed *only* when checking `rustc_private` tools using the `-L` flag. The flags are passed out-of-band through our rustc wrapper, to avoid invalidating the build cache. This is the first commit.

The second commit does the same for std. For a few months, we used to build std instead of just checking it when doing a cross-compile check of something that required std, this now fixes it. There is still the previous ordering requirement though, that `check::Std` has to be executed as the last check step, or rather nothing that requires checked std should be executed *after* it, because it will run into rmeta/rlib duplications (https://github.com/rust-lang/rust/blob/4fa90ef7996f891f7f1e126411e5d75afe64accf/src/bootstrap/src/core/builder/mod.rs#L1066). I tried to fix in this PR, but it quickly runs into the fact that building things currently copies *rlib* artifacts into the build compiler sysroot. I want to fix that as one of the next steps. After we get rid of all the copies (or rather, we only do the copies for dist/stage2+ and do not copy anything into the stage0 compiler's sysroot), we could hopefully finally get rid of `stage0-sysroot`.

Based on my local tests, this seems to be working fine. If it works on CI, and we don't run into other issues after merging it, I'd like to do the same also for rlib artifacts generated during `x build`.

r? `@jieyouxu`

Zalathar added a commit to Zalathar/rust that referenced this pull request

Aug 19, 2025
Do not copy .rmeta files into the sysroot of the build compiler during check of rustc/std

Before, when bootstrap did a check build of rustc stage N (with a build compiler that was stage N-1), it automatically copied the resulting `.rmeta` artifacts into the sysroot of the stage N-1 build compiler, so that stage N `rustc_private` tools such as `miri` could be compiled using the stage N-1 build compiler. This has a number of issues:

- It was done unconditionally, even if no `rustc_private` tools were actually built.
- If we did a check and a build of the same stage compiler in the same bootstrap invocation, the generated rmeta and rlib files could clash. This is also why you can see that `check::Std` actually doesn't copy the artifacts anymore (which forced us to build std instead of just checking it in a bunch of `Check` steps).
- It was polluting the sysroot of the build compiler. This is especially annoying for the stage 0 compiler, because we are forced to create an artificial sysroot for it, so that we can copy new stuff into it.
- It was very implicit in bootstrap.

Based on suggestions by ``@cuviper`` and ``@bjorn3,`` I tried to change how this behaves. Instead of copying the rmeta artifacts into the sysroot of the build compiler (from where they would be loaded implicitly), they are now stored in a separate transient bootstrap build directory, and they are then explicitly passed *only* when checking `rustc_private` tools using the `-L` flag. The flags are passed out-of-band through our rustc wrapper, to avoid invalidating the build cache. This is the first commit.

The second commit does the same for std. For a few months, we used to build std instead of just checking it when doing a cross-compile check of something that required std, this now fixes it. There is still the previous ordering requirement though, that `check::Std` has to be executed as the last check step, or rather nothing that requires checked std should be executed *after* it, because it will run into rmeta/rlib duplications (https://github.com/rust-lang/rust/blob/4fa90ef7996f891f7f1e126411e5d75afe64accf/src/bootstrap/src/core/builder/mod.rs#L1066). I tried to fix in this PR, but it quickly runs into the fact that building things currently copies *rlib* artifacts into the build compiler sysroot. I want to fix that as one of the next steps. After we get rid of all the copies (or rather, we only do the copies for dist/stage2+ and do not copy anything into the stage0 compiler's sysroot), we could hopefully finally get rid of `stage0-sysroot`.

Based on my local tests, this seems to be working fine. If it works on CI, and we don't run into other issues after merging it, I'd like to do the same also for rlib artifacts generated during `x build`.

r? ``@jieyouxu``

Zalathar added a commit to Zalathar/rust that referenced this pull request

Aug 19, 2025

Zalathar added a commit to Zalathar/rust that referenced this pull request

Aug 19, 2025

bors added a commit that referenced this pull request

Aug 19, 2025
Rollup of 19 pull requests

Successful merges:

 - #140956 (`impl PartialEq<{str,String}> for {Path,PathBuf}`)
 - #141744 (Stabilize `ip_from`)
 - #142681 (Remove the `#[no_sanitize]` attribute in favor of `#[sanitize(xyz = "on|off")]`)
 - #142871 (Trivial improve doc for transpose )
 - #144252 (Do not copy .rmeta files into the sysroot of the build compiler during check of rustc/std)
 - #144476 (rustdoc-search: search backend with partitioned suffix tree)
 - #144567 (Fix RISC-V Test Failures in ./x test for Multiple Codegen Cases)
 - #144804 (Don't warn on never to any `as` casts as unreachable)
 - #144960 ([RTE-513] Ignore sleep_until test on SGX)
 - #145013 (overhaul `&mut` suggestions in borrowck errors)
 - #145041 (rework GAT borrowck limitation error)
 - #145243 (take attr style into account in diagnostics)
 - #145405 (cleanup: use run_in_tmpdir in run-make/rustdoc-scrape-examples-paths)
 - #145432 (cg_llvm: Small cleanups to `owned_target_machine`)
 - #145484 (Remove `LlvmArchiveBuilder` and supporting code/bindings)
 - #145557 (Fix uplifting in `Assemble` step)
 - #145563 (Remove the `From` derive macro from prelude)
 - #145565 (Improve context of bootstrap errors in CI)
 - #145584 (interpret: avoid forcing all integer newtypes into memory during clear_provenance)

Failed merges:

 - #145359 (Fix bug where `rustdoc-js` tester would not pick the right `search.js` file if there is more than one)
 - #145573 (Add an experimental unsafe(force_target_feature) attribute.)

r? `@ghost`
`@rustbot` modify labels: rollup

rust-timer added a commit that referenced this pull request

Aug 19, 2025
Rollup merge of #145557 - Kobzol:rustc-link-fix, r=jieyouxu

Fix uplifting in `Assemble` step

In #145310, I removed [this line](https://github.com/rust-lang/rust/pull/145310/files#diff-5a1e05f2688d271039171a547d407d0c8a96715ee64d35562fc76b4c9a874303L2109), which adjusted the stage of the build compiler if an uplift has happened. This broke stage3+ uplifted rustc builds (#145534). I could swear I tested this in the PR, but somehow I missed it.

Instead of keeping the original returned stage, I made it more explicit by returning the actually used `build_compiler` from the `Rustc` step, and then use that in the `Assemble` step.

The changes to `RustcLink` were needed to fix `ui-fulldeps`, which apparently build a stage3 rustc, because I haven't fixed the test steps yet 😅

Hopefully we might be able to remove `RustcLink` if the approach from #144252 will work.

Should fix #145534.

r? ``@jieyouxu``

rust-timer added a commit that referenced this pull request

Aug 19, 2025
Rollup merge of #144252 - Kobzol:rmeta-sysroot, r=jieyouxu

Do not copy .rmeta files into the sysroot of the build compiler during check of rustc/std

Before, when bootstrap did a check build of rustc stage N (with a build compiler that was stage N-1), it automatically copied the resulting `.rmeta` artifacts into the sysroot of the stage N-1 build compiler, so that stage N `rustc_private` tools such as `miri` could be compiled using the stage N-1 build compiler. This has a number of issues:

- It was done unconditionally, even if no `rustc_private` tools were actually built.
- If we did a check and a build of the same stage compiler in the same bootstrap invocation, the generated rmeta and rlib files could clash. This is also why you can see that `check::Std` actually doesn't copy the artifacts anymore (which forced us to build std instead of just checking it in a bunch of `Check` steps).
- It was polluting the sysroot of the build compiler. This is especially annoying for the stage 0 compiler, because we are forced to create an artificial sysroot for it, so that we can copy new stuff into it.
- It was very implicit in bootstrap.

Based on suggestions by ```@cuviper``` and ```@bjorn3,``` I tried to change how this behaves. Instead of copying the rmeta artifacts into the sysroot of the build compiler (from where they would be loaded implicitly), they are now stored in a separate transient bootstrap build directory, and they are then explicitly passed *only* when checking `rustc_private` tools using the `-L` flag. The flags are passed out-of-band through our rustc wrapper, to avoid invalidating the build cache. This is the first commit.

The second commit does the same for std. For a few months, we used to build std instead of just checking it when doing a cross-compile check of something that required std, this now fixes it. There is still the previous ordering requirement though, that `check::Std` has to be executed as the last check step, or rather nothing that requires checked std should be executed *after* it, because it will run into rmeta/rlib duplications (https://github.com/rust-lang/rust/blob/4fa90ef7996f891f7f1e126411e5d75afe64accf/src/bootstrap/src/core/builder/mod.rs#L1066). I tried to fix in this PR, but it quickly runs into the fact that building things currently copies *rlib* artifacts into the build compiler sysroot. I want to fix that as one of the next steps. After we get rid of all the copies (or rather, we only do the copies for dist/stage2+ and do not copy anything into the stage0 compiler's sysroot), we could hopefully finally get rid of `stage0-sysroot`.

Based on my local tests, this seems to be working fine. If it works on CI, and we don't run into other issues after merging it, I'd like to do the same also for rlib artifacts generated during `x build`.

r? ```@jieyouxu```

@bors bors added S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

and removed S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

labels

Aug 19, 2025

github-actions bot pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request

Aug 25, 2025

github-actions bot pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request

Aug 25, 2025