wasm: Store rlib metadata in wasm object files by alexcrichton · Pull Request #120588 · rust-lang/rust

@rustbot rustbot added S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

T-compiler

Relevant to the compiler team, which will review and decide on the PR/issue.

labels

Feb 2, 2024

This was referenced

Feb 2, 2024

bjorn3

wesleywiser

@rustbot rustbot added A-testsuite

Area: The testsuite used to check the correctness of rustc

T-bootstrap

Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

labels

Feb 9, 2024

wesleywiser

@rustbot rustbot added S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

and removed S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

labels

Feb 9, 2024

bjorn3

wesleywiser

@bors bors removed the S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

label

Feb 9, 2024

alexcrichton added a commit to alexcrichton/wasm-tools that referenced this pull request

Feb 10, 2024
Instead substitute this with `AtomicUsize` which panics on overflow to
catch any possible issues on 32-bit platforms. This is motivated by
rust-lang/rust#120588 which is pulling
`wasmparser` into the Rust compiler and `AtomicU64` is not available on
all the platforms the compiler itself is built for.

I plan on backporting this to the `0.118.x` release track as well which
is what's used by the `object` crate currently to avoid the need for a
whole bunch of cascading updates.

alexcrichton added a commit to bytecodealliance/wasm-tools that referenced this pull request

Feb 12, 2024
Instead substitute this with `AtomicUsize` which panics on overflow to
catch any possible issues on 32-bit platforms. This is motivated by
rust-lang/rust#120588 which is pulling
`wasmparser` into the Rust compiler and `AtomicU64` is not available on
all the platforms the compiler itself is built for.

I plan on backporting this to the `0.118.x` release track as well which
is what's used by the `object` crate currently to avoid the need for a
whole bunch of cascading updates.

alexcrichton added a commit to alexcrichton/wasm-tools that referenced this pull request

Feb 12, 2024
Instead substitute this with `AtomicUsize` which panics on overflow to
catch any possible issues on 32-bit platforms. This is motivated by
rust-lang/rust#120588 which is pulling
`wasmparser` into the Rust compiler and `AtomicU64` is not available on
all the platforms the compiler itself is built for.

I plan on backporting this to the `0.118.x` release track as well which
is what's used by the `object` crate currently to avoid the need for a
whole bunch of cascading updates.

alexcrichton added a commit to bytecodealliance/wasm-tools that referenced this pull request

Feb 12, 2024
Instead substitute this with `AtomicUsize` which panics on overflow to
catch any possible issues on 32-bit platforms. This is motivated by
rust-lang/rust#120588 which is pulling
`wasmparser` into the Rust compiler and `AtomicU64` is not available on
all the platforms the compiler itself is built for.

I plan on backporting this to the `0.118.x` release track as well which
is what's used by the `object` crate currently to avoid the need for a
whole bunch of cascading updates.

@alexcrichton

The goal of this commit is to remove warnings using LLVM tip-of-tree
`wasm-ld`. In llvm/llvm-project#78658 the `wasm-ld` LLD driver no longer
looks at archive indices and instead looks at all the objects in
archives. Previously `lib.rmeta` files were simply raw rustc metadata
bytes, not wasm objects, meaning that `wasm-ld` would emit a warning
indicating so.

WebAssembly targets previously passed `--fatal-warnings` to `wasm-ld` by
default which meant that if Rust were to update to LLVM 18 then all wasm
targets would not work. This immediate blocker was resolved in
rust-lang#120278 which removed `--fatal-warnings` which enabled a
theoretical update to LLVM 18 for wasm targets. This current state is
ok-enough for now because rustc squashes all linker output by default if
it doesn't fail. This means, for example, that rustc squashes all the
linker warnings coming out of `wasm-ld` about `lib.rmeta` files with
LLVM 18. This again isn't a pressing issue because the information is
all hidden, but it runs the risk of being annoying if another linker
error were to happen and then the output would have all these unrelated
warnings that couldn't be fixed.

Thus, this PR comes into the picture. The goal of this PR is to resolve
these warnings by using the WebAssembly object file format on wasm
targets instead of using raw rustc metadata. When I first implemented
the rlib-in-objects scheme in rust-lang#84449 I remember either concluding that
`wasm-ld` would either include the metadata in the output or I thought
we didn't have to do anything there at all. I think I was wrong on both
counts as `wasm-ld` does not include the metadata in the final output
unless the object is referenced and we do actually need to do something
to resolve these warnings.

This PR updates the object file format containing rustc metadata on
WebAssembly targets to be an actual WebAssembly file. This enables the
`wasm` feature of the `object` crate to be able to read the custom
section in the same manner as other platforms, but currently `object`
doesn't support writing wasm object files so a handwritten encoder is
used instead.

The only caveat I know of with this is that if `wasm-ld` does indeed
look at the object file then the metadata will be included in the final
output. I believe the only thing that could cause that at this time is
`--whole-archive` which I don't think is passed for rlibs. I would
clarify that I'm not 100% certain about this, however.

@bors bors added S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

and removed S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

labels

Feb 20, 2024