safe transmute: support `Single` enums by jswrenn · Pull Request #126358 · rust-lang/rust

@rustbot rustbot added S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

T-compiler

Relevant to the compiler team, which will review and decide on the PR/issue.

labels

Jun 12, 2024

jswrenn

@rustbot rustbot added S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

and removed S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

labels

Jun 12, 2024

compiler-errors

@jswrenn

Previously, the implementation of `Tree::from_enum` incorrectly
treated enums with `Variants::Single` and `Variants::Multiple`
identically. This is incorrect for `Variants::Single` enums,
which delegate their layout to that of a variant with a particular
index (or no variant at all if the enum is empty).

This flaw manifested first as an ICE. `Tree::from_enum` attempted
to compute the tag of variants other than the one at
`Variants::Single`'s `index`, and fell afoul of a sanity-checking
assertion in `compiler/rustc_const_eval/src/interpret/discriminant.rs`.
This assertion is non-load-bearing, and can be removed; the routine
its in is well-behaved even without it.

With the assertion removed, the proximate issue becomes apparent:
calling `Tree::from_variant` on a variant that does not exist is
ill-defined. A sanity check the given variant has
`FieldShapes::Arbitrary` fails, and the analysis is (correctly)
aborted with `Err::NotYetSupported`.

This commit corrects this chain of failures by ensuring that
`Tree::from_variant` is not called on variants that are, as far as
layout is concerned, nonexistent. Specifically, the implementation
of `Tree::from_enum` is now partitioned into three cases:

  1. enums that are uninhabited
  2. enums for which all but one variant is uninhabited
  3. enums with multiple inhabited variants

`Tree::from_variant` is now only invoked in the third case. In the
first case, `Tree::uninhabited()` is produced. In the second case,
the layout is delegated to `Variants::Single`'s index.

Fixes rust-lang#125811

compiler-errors

@bors bors added S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

and removed S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

labels

Jun 13, 2024

bors added a commit to rust-lang-ci/rust that referenced this pull request

Jun 13, 2024
…kingjubilee

Rollup of 10 pull requests

Successful merges:

 - rust-lang#125674 (Rewrite `symlinked-extern`, `symlinked-rlib` and `symlinked-libraries` `run-make` tests in `rmake.rs` format)
 - rust-lang#125688 (Walk into alias-eq nested goals even if normalization fails)
 - rust-lang#126142 (Harmonize using root or leaf obligation in trait error reporting)
 - rust-lang#126303 (Urls to docs in rust_hir)
 - rust-lang#126328 (Add Option::is_none_or)
 - rust-lang#126337 (Add test for walking order dependent opaque type behaviour)
 - rust-lang#126353 (Move `MatchAgainstFreshVars` to old solver)
 - rust-lang#126356 (docs(rustc): Improve discoverable of Cargo docs)
 - rust-lang#126358 (safe transmute: support `Single` enums)
 - rust-lang#126362 (Make `try_from_target_usize` method public)

r? `@ghost`
`@rustbot` modify labels: rollup

RalfJung

rust-timer added a commit to rust-lang-ci/rust that referenced this pull request

Jun 13, 2024
Rollup merge of rust-lang#126358 - jswrenn:fix-125811, r=compiler-errors

safe transmute: support `Single` enums

Previously, the implementation of `Tree::from_enum` incorrectly treated enums with `Variants::Single` and `Variants::Multiple` identically. This is incorrect for `Variants::Single` enums, which delegate their layout to that of a variant with a particular index (or no variant at all if the enum is empty).

This flaw manifested first as an ICE. `Tree::from_enum` attempted to compute the tag of variants other than the one at `Variants::Single`'s `index`, and fell afoul of a sanity-checking assertion in `compiler/rustc_const_eval/src/interpret/discriminant.rs`. This assertion is non-load-bearing, and can be removed; the routine its in is well-behaved even without it.

With the assertion removed, the proximate issue becomes apparent: calling `Tree::from_variant` on a variant that does not exist is ill-defined. A sanity check the given variant has `FieldShapes::Arbitrary` fails, and the analysis is (correctly) aborted with `Err::NotYetSupported`.

This commit corrects this chain of failures by ensuring that `Tree::from_variant` is not called on variants that are, as far as layout is concerned, nonexistent. Specifically, the implementation of `Tree::from_enum` is now partitioned into three cases:

  1. enums that are uninhabited
  2. enums for which all but one variant is uninhabited
  3. enums with multiple inhabited variants

`Tree::from_variant` is now only invoked in the third case. In the first case, `Tree::uninhabited()` is produced. In the second case, the layout is delegated to `Variants::Single`'s index.

Fixes rust-lang#125811

jswrenn added a commit to jswrenn/rust that referenced this pull request

Jun 14, 2024
Previously, `Tree::from_enum`'s implementation branched into three disjoint
cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple inhabited variants

This branching (incorrectly) did not differentiate between variantful and
variantless uninhabited enums. In both cases, we assumed (and asserted) that
uninhabited enums are zero-sized types. This assumption is false for enums like:

   enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption
manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the
narrower category of "enums that are variantless". These enums, whose layouts
are described with `Variants::Single { index }`, are special in that they do not
have a variant at `index`.

The second case is revised to consider the broader category of "enums that defer
their layout to one of their variants"; i.e., enums whose layouts are described
with `Variants::Single { index }` but that do have a variant at `index`.

This PR also adds a comment requested by @RalfJung in his review of rust-lang#126358 to
`compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

jswrenn added a commit to jswrenn/rust that referenced this pull request

Jun 14, 2024
Previously, `Tree::from_enum`'s implementation branched into three disjoint
cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple inhabited variants

This branching (incorrectly) did not differentiate between variantful and
variantless uninhabited enums. In both cases, we assumed (and asserted) that
uninhabited enums are zero-sized types. This assumption is false for enums like:

    enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption
manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the
narrower category of "enums that are variantless". These enums, whose layouts
are described with `Variants::Single { index }`, are special in that they do not
have a variant at `index`.

The second case is revised to consider the broader category of "enums that defer
their layout to one of their variants"; i.e., enums whose layouts are described
with `Variants::Single { index }` but that do have a variant at `index`.

This PR also adds a comment requested by @RalfJung in his review of rust-lang#126358 to
`compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

jswrenn added a commit to jswrenn/rust that referenced this pull request

Jun 14, 2024
Previously, `Tree::from_enum`'s implementation branched into three disjoint
cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple inhabited variants

This branching (incorrectly) did not differentiate between variantful and
variantless uninhabited enums. In both cases, we assumed (and asserted) that
uninhabited enums are zero-sized types. This assumption is false for enums like:

    enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption
manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the
narrow category of "enums that are uninhabited ZSTs". These enums, whose layouts
are described with `Variants::Single { index }`, are special in their layouts
otherwise resemble the `!` type and cannot be descended into like typical enums.
This first case captures uninhabited enums like:

    enum Uninhabited { A(!, !), B(!) }

The second case is revised to consider the broader category of "enums that defer
their layout to one of their variants"; i.e., enums whose layouts are described
with `Variants::Single { index }` and that do have a variant at `index`. This
second case captures uninhabited enums that are not ZSTs, like:

    enum Uninhabited { A(!, u128) }

This PR also adds a comment requested by @RalfJung in his review of rust-lang#126358 to
`compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

jswrenn added a commit to jswrenn/rust that referenced this pull request

Jun 14, 2024
Previously, `Tree::from_enum`'s implementation branched into three disjoint
cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple inhabited variants

This branching (incorrectly) did not differentiate between variantful and
variantless uninhabited enums. In both cases, we assumed (and asserted) that
uninhabited enums are zero-sized types. This assumption is false for enums like:

    enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption
manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the
narrow category of "enums that are uninhabited ZSTs". These enums, whose layouts
are described with `Variants::Single { index }`, are special in their layouts
otherwise resemble the `!` type and cannot be descended into like typical enums.
This first case captures uninhabited enums like:

    enum Uninhabited { A(!, !), B(!) }

The second case is revised to consider the broader category of "enums that defer
their layout to one of their variants"; i.e., enums whose layouts are described
with `Variants::Single { index }` and that do have a variant at `index`. This
second case captures uninhabited enums that are not ZSTs, like:

    enum Uninhabited { A(!, u128) }

This PR also adds a comment requested by @RalfJung in his review of rust-lang#126358 to
`compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

jswrenn added a commit to jswrenn/rust that referenced this pull request

Jun 14, 2024
Previously, `Tree::from_enum`'s implementation branched into three disjoint
cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple inhabited variants

This branching (incorrectly) did not differentiate between variantful and
variantless uninhabited enums. In both cases, we assumed (and asserted) that
uninhabited enums are zero-sized types. This assumption is false for enums like:

    enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption
manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the
narrow category of "enums that are uninhabited ZSTs". These enums, whose layouts
are described with `Variants::Single { index }`, are special in their layouts
otherwise resemble the `!` type and cannot be descended into like typical enums.
This first case captures uninhabited enums like:

    enum Uninhabited { A(!, !), B(!) }

The second case is revised to consider the broader category of "enums that defer
their layout to one of their variants"; i.e., enums whose layouts are described
with `Variants::Single { index }` and that do have a variant at `index`. This
second case captures uninhabited enums that are not ZSTs, like:

    enum Uninhabited { A(!, u128) }

This PR also adds a comment requested by @RalfJung in his review of rust-lang#126358 to
`compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

jswrenn added a commit to jswrenn/rust that referenced this pull request

Jun 14, 2024
Previously, `Tree::from_enum`'s implementation branched into three disjoint
cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple inhabited variants

This branching (incorrectly) did not differentiate between variantful and
variantless uninhabited enums. In both cases, we assumed (and asserted) that
uninhabited enums are zero-sized types. This assumption is false for enums like:

    enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption
manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the
narrow category of "enums that are uninhabited ZSTs". These enums, whose layouts
are described with `Variants::Single { index }`, are special in their layouts
otherwise resemble the `!` type and cannot be descended into like typical enums.
This first case captures uninhabited enums like:

    enum Uninhabited { A(!, !), B(!) }

The second case is revised to consider the broader category of "enums that defer
their layout to one of their variants"; i.e., enums whose layouts are described
with `Variants::Single { index }` and that do have a variant at `index`. This
second case captures uninhabited enums that are not ZSTs, like:

    enum Uninhabited { A(!, u128) }

Finally, the third case is revised to cover the broader category of "enums with
multiple variants", which captures uninhabited enums like:

    enum Uninhabited { A(u8, !), B(!, u32) }

This PR also adds a comment requested by @RalfJung in his review of rust-lang#126358 to
`compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

jswrenn added a commit to jswrenn/rust that referenced this pull request

Jun 14, 2024
Previously, `Tree::from_enum`'s implementation branched into three disjoint
cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple inhabited variants

This branching (incorrectly) did not differentiate between variantful and
variantless uninhabited enums. In both cases, we assumed (and asserted) that
uninhabited enums are zero-sized types. This assumption is false for enums like:

    enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption
manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the
narrow category of "enums that are uninhabited ZSTs". These enums, whose layouts
are described with `Variants::Single { index }`, are special in their layouts
otherwise resemble the `!` type and cannot be descended into like typical enums.
This first case captures uninhabited enums like:

    enum Uninhabited { A(!, !), B(!) }

The second case is revised to consider the broader category of "enums that defer
their layout to one of their variants"; i.e., enums whose layouts are described
with `Variants::Single { index }` and that do have a variant at `index`. This
second case captures uninhabited enums that are not ZSTs, like:

    enum Uninhabited { A(!, u128) }

Finally, the third case is revised to cover the broader category of "enums with
multiple variants", which captures uninhabited enums like:

    enum Uninhabited { A(u8, !), B(!, u32) }

This PR also adds a comment requested by RalfJung in his review of rust-lang#126358 to
`compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

jswrenn added a commit to jswrenn/rust that referenced this pull request

Jun 14, 2024
Previously, `Tree::from_enum`'s implementation branched into three disjoint
cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple inhabited variants

This branching (incorrectly) did not differentiate between variantful and
variantless uninhabited enums. In both cases, we assumed (and asserted) that
uninhabited enums are zero-sized types. This assumption is false for enums like:

    enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption
manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the
narrow category of "enums that are uninhabited ZSTs". These enums, whose layouts
are described with `Variants::Single { index }`, are special in their layouts
otherwise resemble the `!` type and cannot be descended into like typical enums.
This first case captures uninhabited enums like:

    enum Uninhabited { A(!, !), B(!) }

The second case is revised to consider the broader category of "enums that defer
their layout to one of their variants"; i.e., enums whose layouts are described
with `Variants::Single { index }` and that do have a variant at `index`. This
second case captures uninhabited enums that are not ZSTs, like:

    enum Uninhabited { A(!, u128) }

...which represent their variants with `Variants::Single`.

Finally, the third case is revised to cover the broader category of "enums with
multiple variants", which captures uninhabited, non-ZST enums like:

    enum Uninhabited { A(u8, !), B(!, u32) }

...which represent their variants with `Variants::Multiple`.

This PR also adds a comment requested by RalfJung in his review of rust-lang#126358 to
`compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

jswrenn added a commit to jswrenn/rust that referenced this pull request

Jun 14, 2024
Previously, `Tree::from_enum`'s implementation branched into three disjoint
cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple inhabited variants

This branching (incorrectly) did not differentiate between variantful and
variantless uninhabited enums. In both cases, we assumed (and asserted) that
uninhabited enums are zero-sized types. This assumption is false for enums like:

    enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption
manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the
narrow category of "enums that are uninhabited ZSTs". These enums, whose layouts
are described with `Variants::Single { index }`, are special in their layouts
otherwise resemble the `!` type and cannot be descended into like typical enums.
This first case captures uninhabited enums like:

    enum Uninhabited { A(!, !), B(!) }

The second case is revised to consider the broader category of "enums that defer
their layout to one of their variants"; i.e., enums whose layouts are described
with `Variants::Single { index }` and that do have a variant at `index`. This
second case captures uninhabited enums that are not ZSTs, like:

    enum Uninhabited { A(!, u128) }

...which represent their variants with `Variants::Single`.

Finally, the third case is revised to cover the broader category of "enums with
multiple variants", which captures uninhabited, non-ZST enums like:

    enum Uninhabited { A(u8, !), B(!, u32) }

...which represent their variants with `Variants::Multiple`.

This PR also adds a comment requested by RalfJung in his review of rust-lang#126358 to
`compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this pull request

Jun 18, 2024
safe transmute: support non-ZST, variantful, uninhabited enums

Previously, `Tree::from_enum`'s implementation branched into three disjoint cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple variants

This branching (incorrectly) did not differentiate between variantful and variantless uninhabited enums. In both cases, we assumed (and asserted) that uninhabited enums are zero-sized types. This assumption is false for enums like:

    enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the narrow category of "enums that are uninhabited ZSTs". These enums, whose layouts are described with `Variants::Single { index }`, are special in their layouts otherwise resemble the `!` type and cannot be descended into like typical enums. This first case captures uninhabited enums like:

    enum Uninhabited { A(!, !), B(!) }

The second case is revised to consider the broader category of "enums that defer their layout to one of their variants"; i.e., enums whose layouts are described with `Variants::Single { index }` and that do have a variant at `index`. This second case captures uninhabited enums that are not ZSTs, like:

    enum Uninhabited { A(!, u128) }

...which represent their variants with `Variants::Single`.

Finally, the third case is revised to cover the broader category of "enums with multiple variants", which captures uninhabited enums like:

    enum Uninhabited { A(u8, !), B(!, u32) }

...which represent their variants with `Variants::Multiple`.

This PR also adds a comment requested by `@RalfJung` in his review of rust-lang#126358 to `compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

r? `@compiler-errors`

GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this pull request

Jun 18, 2024
safe transmute: support non-ZST, variantful, uninhabited enums

Previously, `Tree::from_enum`'s implementation branched into three disjoint cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple variants

This branching (incorrectly) did not differentiate between variantful and variantless uninhabited enums. In both cases, we assumed (and asserted) that uninhabited enums are zero-sized types. This assumption is false for enums like:

    enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the narrow category of "enums that are uninhabited ZSTs". These enums, whose layouts are described with `Variants::Single { index }`, are special in their layouts otherwise resemble the `!` type and cannot be descended into like typical enums. This first case captures uninhabited enums like:

    enum Uninhabited { A(!, !), B(!) }

The second case is revised to consider the broader category of "enums that defer their layout to one of their variants"; i.e., enums whose layouts are described with `Variants::Single { index }` and that do have a variant at `index`. This second case captures uninhabited enums that are not ZSTs, like:

    enum Uninhabited { A(!, u128) }

...which represent their variants with `Variants::Single`.

Finally, the third case is revised to cover the broader category of "enums with multiple variants", which captures uninhabited enums like:

    enum Uninhabited { A(u8, !), B(!, u32) }

...which represent their variants with `Variants::Multiple`.

This PR also adds a comment requested by ``@RalfJung`` in his review of rust-lang#126358 to `compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

r? ``@compiler-errors``

jieyouxu added a commit to jieyouxu/rust that referenced this pull request

Jun 19, 2024
safe transmute: support non-ZST, variantful, uninhabited enums

Previously, `Tree::from_enum`'s implementation branched into three disjoint cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple variants

This branching (incorrectly) did not differentiate between variantful and variantless uninhabited enums. In both cases, we assumed (and asserted) that uninhabited enums are zero-sized types. This assumption is false for enums like:

    enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the narrow category of "enums that are uninhabited ZSTs". These enums, whose layouts are described with `Variants::Single { index }`, are special in their layouts otherwise resemble the `!` type and cannot be descended into like typical enums. This first case captures uninhabited enums like:

    enum Uninhabited { A(!, !), B(!) }

The second case is revised to consider the broader category of "enums that defer their layout to one of their variants"; i.e., enums whose layouts are described with `Variants::Single { index }` and that do have a variant at `index`. This second case captures uninhabited enums that are not ZSTs, like:

    enum Uninhabited { A(!, u128) }

...which represent their variants with `Variants::Single`.

Finally, the third case is revised to cover the broader category of "enums with multiple variants", which captures uninhabited enums like:

    enum Uninhabited { A(u8, !), B(!, u32) }

...which represent their variants with `Variants::Multiple`.

This PR also adds a comment requested by ```@RalfJung``` in his review of rust-lang#126358 to `compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

r? ```@compiler-errors```

jieyouxu added a commit to jieyouxu/rust that referenced this pull request

Jun 19, 2024
safe transmute: support non-ZST, variantful, uninhabited enums

Previously, `Tree::from_enum`'s implementation branched into three disjoint cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple variants

This branching (incorrectly) did not differentiate between variantful and variantless uninhabited enums. In both cases, we assumed (and asserted) that uninhabited enums are zero-sized types. This assumption is false for enums like:

    enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the narrow category of "enums that are uninhabited ZSTs". These enums, whose layouts are described with `Variants::Single { index }`, are special in their layouts otherwise resemble the `!` type and cannot be descended into like typical enums. This first case captures uninhabited enums like:

    enum Uninhabited { A(!, !), B(!) }

The second case is revised to consider the broader category of "enums that defer their layout to one of their variants"; i.e., enums whose layouts are described with `Variants::Single { index }` and that do have a variant at `index`. This second case captures uninhabited enums that are not ZSTs, like:

    enum Uninhabited { A(!, u128) }

...which represent their variants with `Variants::Single`.

Finally, the third case is revised to cover the broader category of "enums with multiple variants", which captures uninhabited enums like:

    enum Uninhabited { A(u8, !), B(!, u32) }

...which represent their variants with `Variants::Multiple`.

This PR also adds a comment requested by ````@RalfJung```` in his review of rust-lang#126358 to `compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

r? ````@compiler-errors````

rust-timer added a commit to rust-lang-ci/rust that referenced this pull request

Jun 19, 2024
Rollup merge of rust-lang#126493 - jswrenn:fix-126460, r=compiler-errors

safe transmute: support non-ZST, variantful, uninhabited enums

Previously, `Tree::from_enum`'s implementation branched into three disjoint cases:

 1. enums that uninhabited
 2. enums for which all but one variant is uninhabited
 3. enums with multiple variants

This branching (incorrectly) did not differentiate between variantful and variantless uninhabited enums. In both cases, we assumed (and asserted) that uninhabited enums are zero-sized types. This assumption is false for enums like:

    enum Uninhabited { A(!, u128) }

...which, currently, has the same size as `u128`. This faulty assumption manifested as the ICE reported in rust-lang#126460.

In this PR, we revise the first case of `Tree::from_enum` to consider only the narrow category of "enums that are uninhabited ZSTs". These enums, whose layouts are described with `Variants::Single { index }`, are special in their layouts otherwise resemble the `!` type and cannot be descended into like typical enums. This first case captures uninhabited enums like:

    enum Uninhabited { A(!, !), B(!) }

The second case is revised to consider the broader category of "enums that defer their layout to one of their variants"; i.e., enums whose layouts are described with `Variants::Single { index }` and that do have a variant at `index`. This second case captures uninhabited enums that are not ZSTs, like:

    enum Uninhabited { A(!, u128) }

...which represent their variants with `Variants::Single`.

Finally, the third case is revised to cover the broader category of "enums with multiple variants", which captures uninhabited enums like:

    enum Uninhabited { A(u8, !), B(!, u32) }

...which represent their variants with `Variants::Multiple`.

This PR also adds a comment requested by ````@RalfJung```` in his review of rust-lang#126358 to `compiler/rustc_const_eval/src/interpret/discriminant.rs`.

Fixes rust-lang#126460

r? ````@compiler-errors````