ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests` by Enselic · Pull Request #142886 · rust-lang/rust

@Enselic

The skipped test passes since nightly-2024-11-29. Let's stop skipping
it to increase the chance of detecing a regression.

@rustbot rustbot added A-testsuite

Area: The testsuite used to check the correctness of rustc

S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

T-infra

Relevant to the infrastructure team, which will review and decide on the PR/issue.

labels

Jun 22, 2025

This was referenced

Jun 23, 2025

rust-bors bot added a commit that referenced this pull request

Jun 25, 2025

rust-bors bot added a commit that referenced this pull request

Jun 25, 2025

rust-bors bot added a commit that referenced this pull request

Jun 25, 2025
…try>

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

Normally a `run-fail` ui test shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal however. Introduce and use `run-fail-without-exit-code` for those tests.

This adds further (cc #142304, #142886) protection against the regression in #123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [ ] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See #143002 (comment).

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

try-job: x86_64-msvc-1
try-job: x86_64-msvc-2

rust-bors bot added a commit that referenced this pull request

Jun 25, 2025
…try>

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

Normally a `run-fail` ui test shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal however. Introduce and use ~`run-fail-without-exit-code`~ `run-crash` for those tests.

This adds further (cc #142304, #142886) protection against the regression in #123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [ ] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See #143002 (comment).

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

try-job: x86_64-msvc-1
try-job: x86_64-msvc-2

compiler-errors added a commit to compiler-errors/rust that referenced this pull request

Jun 27, 2025

bors added a commit that referenced this pull request

Jun 27, 2025
…try>

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

Normally a `run-fail` ui test shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal however. Introduce and use ~`run-fail-without-exit-code`~ `run-crash` for those tests.

This adds further (cc #142304, #142886) protection against the regression in #123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [x] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See #143002 (comment).

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

//  try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-gnu
try-job: dist-i586-gnu-i586-i686-musl
try-job: test-various

rust-bors bot added a commit that referenced this pull request

Jun 30, 2025
…try>

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

Normally a `run-fail` ui test shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal however. Introduce and use ~`run-fail-without-exit-code`~ `run-crash` for those tests.

This adds further (cc #142304, #142886) protection against the regression in #123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [x] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See #143002 (comment).
- [ ] Handle targets without unwind support

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

//  try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-gnu
try-job: dist-i586-gnu-i586-i686-musl
try-job: test-various
try-job: test-various

rust-bors bot added a commit that referenced this pull request

Jun 30, 2025
…try>

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

Normally a `run-fail` ui test shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal however. Introduce and use ~`run-fail-without-exit-code`~ `run-crash` for those tests.

This adds further (cc #142304, #142886) protection against the regression in #123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [x] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See #143002 (comment).
- [ ] Handle targets without unwind support

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

//  try-job: aarch64-apple
// try-job: x86_64-msvc-1
// try-job: x86_64-gnu
// try-job: dist-i586-gnu-i586-i686-musl
try-job: test-various
try-job: test-various

rust-bors bot added a commit that referenced this pull request

Jun 30, 2025
…try>

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

Normally a `run-fail` ui test shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal however. Introduce and use ~`run-fail-without-exit-code`~ `run-crash` for those tests.

This adds further (cc #142304, #142886) protection against the regression in #123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [x] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See #143002 (comment).
- [ ] Handle targets without unwind support

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

try-job: aarch64-apple
// try-job: x86_64-msvc-1
// try-job: x86_64-gnu
// try-job: dist-i586-gnu-i586-i686-musl
// try-job: test-various
try-job: aarch64-apple

rust-bors bot added a commit that referenced this pull request

Jul 1, 2025
ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests`

The skipped test passes since `nightly-2024-11-29`. See #123733 (comment) and #123733 (comment) for more info.

Let's stop skipping it to increase the chance of detecting a regression.

r? `@cuviper` who added the skip in #123828

Also see #142304 for an alternative regression test that I am hoping to also land in the near future to complement the test we now stop skipping, but I need to investigate that setup more.

<!-- homu-ignore:start -->
<!--
If this PR is related to an unstable feature or an otherwise tracked effort,
please link to the relevant tracking issue here. If you don't know of a related
tracking issue or there are none, feel free to ignore this.

This PR will get automatically assigned to a reviewer. In case you would like
a specific user to review your work, you can assign it to them by using

    r? <reviewer name>
-->
<!-- homu-ignore:end -->

try-job: aarch64-gnu

@bors bors added S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

and removed S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

labels

Jul 1, 2025

jhpratt added a commit to jhpratt/rust that referenced this pull request

Jul 1, 2025
ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests`

The skipped test passes since `nightly-2024-11-29`. See rust-lang#123733 (comment) and rust-lang#123733 (comment) for more info.

Let's stop skipping it to increase the chance of detecting a regression.

r? `@cuviper` who added the skip in rust-lang#123828

Also see rust-lang#142304 for an alternative regression test that I am hoping to also land in the near future to complement the test we now stop skipping, but I need to investigate that setup more.

bors added a commit that referenced this pull request

Jul 2, 2025
Rollup of 12 pull requests

Successful merges:

 - #141829 (Specialize sleep_until implementation for unix (except mac))
 - #141847 (Explain `TOCTOU` on the top of `std::fs`, and reference it in functions)
 - #142138 (Add `Vec::into_chunks`)
 - #142321 (Expose elf abi on ppc64 targets)
 - #142568 (Use the .drectve section for exporting symbols from dlls on Windows)
 - #142886 (ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests`)
 - #143038 (avoid suggesting traits from private dependencies)
 - #143194 (fix bitcast of single-element SIMD vectors)
 - #143206 (Align attr fixes)
 - #143258 (Don't recompute `DisambiguatorState` for every RPITIT in trait definition)
 - #143260 (Use the correct export kind for __rust_alloc_error_handler_should_panic)
 - #143274 (ci: support optional jobs)

r? `@ghost`
`@rustbot` modify labels: rollup

jhpratt added a commit to jhpratt/rust that referenced this pull request

Jul 2, 2025
ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests`

The skipped test passes since `nightly-2024-11-29`. See rust-lang#123733 (comment) and rust-lang#123733 (comment) for more info.

Let's stop skipping it to increase the chance of detecting a regression.

r? ``@cuviper`` who added the skip in rust-lang#123828

Also see rust-lang#142304 for an alternative regression test that I am hoping to also land in the near future to complement the test we now stop skipping, but I need to investigate that setup more.

rust-bors bot added a commit that referenced this pull request

Jul 2, 2025
…try>

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

Normally a `run-fail` ui test shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal however. Introduce and use ~`run-fail-without-exit-code`~ `run-crash` for those tests.

This adds further (cc #142304, #142886) protection against the regression in #123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [x] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See #143002 (comment).
- [x] Handle targets without unwind support
- [ ] Add `run-fail-or-crash` for some sanitizer tests

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

try-job: aarch64-apple
// try-job: x86_64-msvc-1
// try-job: x86_64-gnu
// try-job: dist-i586-gnu-i586-i686-musl
// try-job: test-various
try-job: aarch64-apple

bors added a commit that referenced this pull request

Jul 2, 2025
Rollup of 11 pull requests

Successful merges:

 - #141829 (Specialize sleep_until implementation for unix (except mac))
 - #141847 (Explain `TOCTOU` on the top of `std::fs`, and reference it in functions)
 - #142138 (Add `Vec::into_chunks`)
 - #142321 (Expose elf abi on ppc64 targets)
 - #142886 (ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests`)
 - #143038 (avoid suggesting traits from private dependencies)
 - #143194 (fix bitcast of single-element SIMD vectors)
 - #143206 (Align attr fixes)
 - #143258 (Don't recompute `DisambiguatorState` for every RPITIT in trait definition)
 - #143260 (Use the correct export kind for __rust_alloc_error_handler_should_panic)
 - #143274 (ci: support optional jobs)

r? `@ghost`
`@rustbot` modify labels: rollup

jhpratt added a commit to jhpratt/rust that referenced this pull request

Jul 2, 2025
ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests`

The skipped test passes since `nightly-2024-11-29`. See rust-lang#123733 (comment) and rust-lang#123733 (comment) for more info.

Let's stop skipping it to increase the chance of detecting a regression.

r? ```@cuviper``` who added the skip in rust-lang#123828

Also see rust-lang#142304 for an alternative regression test that I am hoping to also land in the near future to complement the test we now stop skipping, but I need to investigate that setup more.

bors added a commit that referenced this pull request

Jul 2, 2025
Rollup of 12 pull requests

Successful merges:

 - #141847 (Explain `TOCTOU` on the top of `std::fs`, and reference it in functions)
 - #142138 (Add `Vec::into_chunks`)
 - #142321 (Expose elf abi on ppc64 targets)
 - #142886 (ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests`)
 - #143038 (avoid suggesting traits from private dependencies)
 - #143194 (fix bitcast of single-element SIMD vectors)
 - #143206 (Align attr fixes)
 - #143231 (Suggest use another lifetime specifier instead of underscore lifetime)
 - #143232 ([COMPILETEST-UNTANGLE 3/N] Use "directives" consistently within compiletest)
 - #143258 (Don't recompute `DisambiguatorState` for every RPITIT in trait definition)
 - #143260 (Use the correct export kind for __rust_alloc_error_handler_should_panic)
 - #143274 (ci: support optional jobs)

r? `@ghost`
`@rustbot` modify labels: rollup

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request

Jul 2, 2025
ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests`

The skipped test passes since `nightly-2024-11-29`. See rust-lang#123733 (comment) and rust-lang#123733 (comment) for more info.

Let's stop skipping it to increase the chance of detecting a regression.

r? ````@cuviper```` who added the skip in rust-lang#123828

Also see rust-lang#142304 for an alternative regression test that I am hoping to also land in the near future to complement the test we now stop skipping, but I need to investigate that setup more.

rust-bors bot added a commit that referenced this pull request

Jul 2, 2025
…try>

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

Normally a `run-fail` ui test shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal however. Introduce and use ~`run-fail-without-exit-code`~ `run-crash` for those tests.

This adds further (cc #142304, #142886) protection against the regression in #123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [x] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See #143002 (comment).
- [x] Handle targets without unwind support
- [ ] Add `run-fail-or-crash` for some sanitizer tests

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-gnu
try-job: dist-i586-gnu-i586-i686-musl
try-job: test-various

bors added a commit that referenced this pull request

Jul 2, 2025
Rollup of 11 pull requests

Successful merges:

 - #141847 (Explain `TOCTOU` on the top of `std::fs`, and reference it in functions)
 - #142138 (Add `Vec::into_chunks`)
 - #142321 (Expose elf abi on ppc64 targets)
 - #142886 (ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests`)
 - #143038 (avoid suggesting traits from private dependencies)
 - #143194 (fix bitcast of single-element SIMD vectors)
 - #143231 (Suggest use another lifetime specifier instead of underscore lifetime)
 - #143232 ([COMPILETEST-UNTANGLE 3/N] Use "directives" consistently within compiletest)
 - #143258 (Don't recompute `DisambiguatorState` for every RPITIT in trait definition)
 - #143260 (Use the correct export kind for __rust_alloc_error_handler_should_panic)
 - #143274 (ci: support optional jobs)

r? `@ghost`
`@rustbot` modify labels: rollup

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request

Jul 2, 2025
ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests`

The skipped test passes since `nightly-2024-11-29`. See rust-lang#123733 (comment) and rust-lang#123733 (comment) for more info.

Let's stop skipping it to increase the chance of detecting a regression.

r? `````@cuviper````` who added the skip in rust-lang#123828

Also see rust-lang#142304 for an alternative regression test that I am hoping to also land in the near future to complement the test we now stop skipping, but I need to investigate that setup more.

bors added a commit that referenced this pull request

Jul 2, 2025
Rollup of 10 pull requests

Successful merges:

 - #141847 (Explain `TOCTOU` on the top of `std::fs`, and reference it in functions)
 - #142138 (Add `Vec::into_chunks`)
 - #142321 (Expose elf abi on ppc64 targets)
 - #142886 (ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests`)
 - #143194 (fix bitcast of single-element SIMD vectors)
 - #143231 (Suggest use another lifetime specifier instead of underscore lifetime)
 - #143232 ([COMPILETEST-UNTANGLE 3/N] Use "directives" consistently within compiletest)
 - #143258 (Don't recompute `DisambiguatorState` for every RPITIT in trait definition)
 - #143260 (Use the correct export kind for __rust_alloc_error_handler_should_panic)
 - #143274 (ci: support optional jobs)

r? `@ghost`
`@rustbot` modify labels: rollup

bors added a commit that referenced this pull request

Jul 2, 2025
Rollup of 9 pull requests

Successful merges:

 - #141847 (Explain `TOCTOU` on the top of `std::fs`, and reference it in functions)
 - #142138 (Add `Vec::into_chunks`)
 - #142321 (Expose elf abi on ppc64 targets)
 - #142886 (ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests`)
 - #143194 (fix bitcast of single-element SIMD vectors)
 - #143231 (Suggest use another lifetime specifier instead of underscore lifetime)
 - #143232 ([COMPILETEST-UNTANGLE 3/N] Use "directives" consistently within compiletest)
 - #143258 (Don't recompute `DisambiguatorState` for every RPITIT in trait definition)
 - #143274 (ci: support optional jobs)

r? `@ghost`
`@rustbot` modify labels: rollup

rust-timer added a commit that referenced this pull request

Jul 2, 2025
Rollup merge of #142886 - Enselic:aarch64-panic, r=cuviper

ci: aarch64-gnu: Stop skipping `panic_abort_doc_tests`

The skipped test passes since `nightly-2024-11-29`. See #123733 (comment) and #123733 (comment) for more info.

Let's stop skipping it to increase the chance of detecting a regression.

r? ``````@cuviper`````` who added the skip in #123828

Also see #142304 for an alternative regression test that I am hoping to also land in the near future to complement the test we now stop skipping, but I need to investigate that setup more.

bors added a commit that referenced this pull request

Jul 4, 2025
…etrochenkov

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

And introduce two new directives for ui tests:
* `run-crash`
* `run-fail-or-crash`

Normally a `run-fail` ui test like tests that panic shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal/crash however.  Introduce and use `run-crash` for those tests. Note that Windows crashes are not handled by signals but by certain high bits set on the process  exit code. Example exit code for crash on Windows: `0xc000001d` (`STATUS_ILLEGAL_INSTRUCTION`).  Because of this, we define "crash" on all platforms as "not exit with success and not exit with a regular failure code in the range 1..=127".

Some tests behave differently on different targets:
* Targets without unwind support will abort (crash) instead of exit with   failure code 101 after panicking. As a special case, allow crashes for   `run-fail` tests for such targets.
* Different sanitizer implementations handle detected memory problems   differently. Some abort (crash) the process while others exit with   failure code 1. Introduce and use `run-fail-or-crash` for such tests.

This adds further (cc #142304, #142886) protection against the regression in #123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [x] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See #143002 (comment).
- [x] Handle targets without unwind support
- [x] Add `run-fail-or-crash` for some sanitizer tests

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-gnu
try-job: dist-i586-gnu-i586-i686-musl
try-job: test-various

rust-bors bot added a commit that referenced this pull request

Jul 4, 2025
…try>

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

And introduce two new directives for ui tests:
* `run-crash`
* `run-fail-or-crash`

Normally a `run-fail` ui test like tests that panic shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal/crash however.  Introduce and use `run-crash` for those tests. Note that Windows crashes are not handled by signals but by certain high bits set on the process  exit code. Example exit code for crash on Windows: `0xc000001d` (`STATUS_ILLEGAL_INSTRUCTION`).  Because of this, we define "crash" on all platforms as "not exit with success and not exit with a regular failure code in the range 1..=127".

Some tests behave differently on different targets:
* Targets without unwind support will abort (crash) instead of exit with   failure code 101 after panicking. As a special case, allow crashes for   `run-fail` tests for such targets.
* Different sanitizer implementations handle detected memory problems   differently. Some abort (crash) the process while others exit with   failure code 1. Introduce and use `run-fail-or-crash` for such tests.

This adds further (cc #142304, #142886) protection against the regression in #123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [x] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See #143002 (comment).
- [x] Handle targets without unwind support
- [x] Add `run-fail-or-crash` for some sanitizer tests

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

// try-job: aarch64-apple
// try-job: x86_64-msvc-1
// try-job: x86_64-gnu
// try-job: dist-i586-gnu-i586-i686-musl
// try-job: test-various
try-job: armhf-gnu
try-job: armhf-gnu

fmease added a commit to fmease/rust that referenced this pull request

Jul 17, 2025
…signal, r=petrochenkov

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

And introduce two new directives for ui tests:
* `run-crash`
* `run-fail-or-crash`

Normally a `run-fail` ui test like tests that panic shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal/crash however.  Introduce and use `run-crash` for those tests. Note that Windows crashes are not handled by signals but by certain high bits set on the process  exit code. Example exit code for crash on Windows: `0xc000001d` (`STATUS_ILLEGAL_INSTRUCTION`).  Because of this, we define "crash" on all platforms as "not exit with success and not exit with a regular failure code in the range 1..=127".

Some tests behave differently on different targets:
* Targets without unwind support will abort (crash) instead of exit with   failure code 101 after panicking. As a special case, allow crashes for   `run-fail` tests for such targets.
* Different sanitizer implementations handle detected memory problems   differently. Some abort (crash) the process while others exit with   failure code 1. Introduce and use `run-fail-or-crash` for such tests.

This adds further (cc rust-lang#142304, rust-lang#142886) protection against the regression in rust-lang#123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [x] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See rust-lang#143002 (comment).
- [x] Handle targets without unwind support
- [x] Add `run-fail-or-crash` for some sanitizer tests
- [x] remote-test-client. See rust-lang#143448

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-gnu
try-job: dist-i586-gnu-i586-i686-musl
try-job: test-various
try-job: armhf-gnu

bors added a commit that referenced this pull request

Jul 20, 2025
…etrochenkov

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

And introduce two new directives for ui tests:
* `run-crash`
* `run-fail-or-crash`

Normally a `run-fail` ui test like tests that panic shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal/crash however.  Introduce and use `run-crash` for those tests. Note that Windows crashes are not handled by signals but by certain high bits set on the process  exit code. Example exit code for crash on Windows: `0xc000001d` (`STATUS_ILLEGAL_INSTRUCTION`).  Because of this, we define "crash" on all platforms as "not exit with success and not exit with a regular failure code in the range 1..=127".

Some tests behave differently on different targets:
* Targets without unwind support will abort (crash) instead of exit with   failure code 101 after panicking. As a special case, allow crashes for   `run-fail` tests for such targets.
* Different sanitizer implementations handle detected memory problems   differently. Some abort (crash) the process while others exit with   failure code 1. Introduce and use `run-fail-or-crash` for such tests.

This adds further (cc #142304, #142886) protection against the regression in #123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [x] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See #143002 (comment).
- [x] Handle targets without unwind support
- [x] Add `run-fail-or-crash` for some sanitizer tests
- [x] remote-test-client. See #143448

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-gnu
try-job: dist-i586-gnu-i586-i686-musl
try-job: test-various
try-job: armhf-gnu

github-actions bot pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request

Jul 21, 2025
…etrochenkov

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

And introduce two new directives for ui tests:
* `run-crash`
* `run-fail-or-crash`

Normally a `run-fail` ui test like tests that panic shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal/crash however.  Introduce and use `run-crash` for those tests. Note that Windows crashes are not handled by signals but by certain high bits set on the process  exit code. Example exit code for crash on Windows: `0xc000001d` (`STATUS_ILLEGAL_INSTRUCTION`).  Because of this, we define "crash" on all platforms as "not exit with success and not exit with a regular failure code in the range 1..=127".

Some tests behave differently on different targets:
* Targets without unwind support will abort (crash) instead of exit with   failure code 101 after panicking. As a special case, allow crashes for   `run-fail` tests for such targets.
* Different sanitizer implementations handle detected memory problems   differently. Some abort (crash) the process while others exit with   failure code 1. Introduce and use `run-fail-or-crash` for such tests.

This adds further (cc rust-lang/rust#142304, rust-lang/rust#142886) protection against the regression in rust-lang/rust#123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [x] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See rust-lang/rust#143002 (comment).
- [x] Handle targets without unwind support
- [x] Add `run-fail-or-crash` for some sanitizer tests
- [x] remote-test-client. See rust-lang/rust#143448

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-gnu
try-job: dist-i586-gnu-i586-i686-musl
try-job: test-various
try-job: armhf-gnu

Muscraft pushed a commit to Muscraft/rust that referenced this pull request

Jul 21, 2025
…gnal, r=petrochenkov

tests: Require `run-fail` ui tests to have an exit code (`SIGABRT` not ok)

And introduce two new directives for ui tests:
* `run-crash`
* `run-fail-or-crash`

Normally a `run-fail` ui test like tests that panic shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement.

Some of our current tests do terminate by a signal/crash however.  Introduce and use `run-crash` for those tests. Note that Windows crashes are not handled by signals but by certain high bits set on the process  exit code. Example exit code for crash on Windows: `0xc000001d` (`STATUS_ILLEGAL_INSTRUCTION`).  Because of this, we define "crash" on all platforms as "not exit with success and not exit with a regular failure code in the range 1..=127".

Some tests behave differently on different targets:
* Targets without unwind support will abort (crash) instead of exit with   failure code 101 after panicking. As a special case, allow crashes for   `run-fail` tests for such targets.
* Different sanitizer implementations handle detected memory problems   differently. Some abort (crash) the process while others exit with   failure code 1. Introduce and use `run-fail-or-crash` for such tests.

This adds further (cc rust-lang#142304, rust-lang#142886) protection against the regression in rust-lang#123733 since that bug also manifested as `SIGABRT` in `tests/ui/panics/panic-main.rs` (shown as `Aborted (core dumped)` in the logs attached to that issue, and I have also been able to reproduce this locally).

### TODO
- [x] **Q:** what about on Windows?
    **A:** we'll treat any exit code outside of 1 - 127 as "crashed", and we'll do the same on unix.
- [x] test all permutations of actual vs expected
    **Done:** See rust-lang#143002 (comment).
- [x] Handle targets without unwind support
- [x] Add `run-fail-or-crash` for some sanitizer tests
- [x] remote-test-client. See rust-lang#143448

### Zulip discussion

See https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/compiletest.3A.20terminate.20by.20signal.20vs.20exit.20with.20error/with/525611235

try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-gnu
try-job: dist-i586-gnu-i586-i686-musl
try-job: test-various
try-job: armhf-gnu