Allow cross-compiling doctests by Goirad · Pull Request #60387 · rust-lang/rust

@Goirad Goirad changed the title Test expansion Improvements for cross-compiling and cross-running tests

Apr 29, 2019

jethrogb

@kennytm kennytm added the S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

label

Apr 30, 2019

@Goirad Goirad changed the title Improvements for cross-compiling and cross-running tests Allow cross-compiling doctests

Jun 5, 2019

kennytm

@kennytm kennytm removed the S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

label

Jun 6, 2019

@Goirad

updated and augmented tests in html/markdown.rs
…e at argument parsing time if no --target arguments

@Goirad

@Goirad

@bors bors added S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

and removed S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

labels

Sep 10, 2019

bors added a commit that referenced this pull request

Sep 10, 2019
Allow cross-compiling doctests

This PR allows doctest to receive a --runtool argument, as well as possibly many --runtool-arg arguments, which are then used to run cross compiled doctests.
Also, functionality has been added to rustdoc to allow it to skip testing doctests on a per-target basis, in the same way that compiletest does it. For example, tagging the doctest with "ignore-sgx" disables testing on any targets that contain "sgx". A plain "ignore" still skips testing on all targets.

See [here](rust-lang/cargo#6892) for the companion PR in the cargo project that extends functionality in Cargo so that it passes the appropriate parameters to rustdoc when cross compiling and testing doctests.

Part of [#6460](rust-lang/cargo#6460)

@bors bors mentioned this pull request

Sep 10, 2019

bors added a commit to rust-lang/cargo that referenced this pull request

Sep 18, 2019
Added ability to crosscompile doctests

This commit adds the ability to cross-compile and run doctests.
Like before cargo checks if target == host, the difference is that if there is a runtool defined in config.toml, it passes the information forward to rustdoc so that it can run the doctests with that tool. If no tool is defined and the target != host, cargo instead displays a message that doctests will not be compiled because of the missing runtool.

See [here](rust-lang/rust#60387) for the companion PR in the rust project that modifies rustdoc to accept the relevant options as well as allow ignoring doctests on a per target level.
Partially resolves [#6460](#6460)

See [here](#7040) for the tracking issue.

@ojeda ojeda mentioned this pull request

Dec 13, 2024

48 tasks

@ehuss ehuss mentioned this pull request

Apr 28, 2025

github-merge-queue bot pushed a commit to rust-lang/cargo that referenced this pull request

May 14, 2025
This stabilizes the doctest-xcompile feature by unconditionally enabling
it.

Closes #7040
Closes #12118

## What is being stabilized?

This changes it so that cargo will run doctests when using the
`--target` flag for a target that is not the host. Previously, cargo
would ignore doctests (and show a note if passing `--verbose`).

A wrapper for running the doctest can be specified with the
[`target.*.runner`](https://doc.rust-lang.org/cargo/reference/config.html#targettriplerunner)
configuration option (which is powered by the `--test-runtool` rustdoc
flag). This would typically be something like qemu to run under
emulation. It is my understanding that this should work just like
running other kinds of tests.

Additionally, the
[`target.*.linker`](https://doc.rust-lang.org/cargo/reference/config.html#targettriplelinker)
config option is honored for using a custom linker.

Already stabilized in rustdoc is the ability to [ignore tests
per-target](https://doc.rust-lang.org/nightly/rustdoc/write-documentation/documentation-tests.html#ignoring-targets).

## Motivation

The lack of doctest cross-compile support has always been simply due to
the lack of functionality in rustdoc to support this. Rustdoc gained the
ability to cross-compile doctests some time ago, but there were
additional flags like the test runner that were not stabilized until
just recently.

This is intended to ensure that projects have full test coverage even
when doing cross-compilation. It can be
[surprising](#12118) to some
that this was not happening, particularly since cargo is silent about
it.

## Risks

The cargo team had several conversations about how to roll out this
feature. Ultimately we decided to enable it unconditionally with the
understanding that most projects will probably want to have their
doctests covered, and that any breakage will be a local concern that can
be resolved by either fixing the test or ignoring the target.

Tests in rust-lang/rust run into this issue, [particularly on
android](rust-lang/rust#119147 (comment)),
and those will need to be fixed before this reaches beta. This is
something I am looking into.

Some cross-compiling scenarios may need codegen flags that are not
supported. It's not clear how common this will be, or if ignoring will
be a solution, or how difficult it would be to update rustdoc and cargo
to support these. Additionally, the split between RUSTFLAGS and
RUSTDOCFLAGS can be cumbersome.

## Implementation history

- rust-lang/rust#60387 -- Support added to
rustdoc to support the `--target` flag and runtool and
per-target-ignores.
- #6892 -- Initial support in
cargo.
- #7391 -- Added unstable
documentation.
- #8094 -- Fix target for doc
test cross compilation
- #8358 -- Fixed regression with
`--target=HOST` not working on stable.
- #10132 -- Added note about
doctests not running (under `--verbose`).
- rust-lang/rust#112751 -- Fixed
`--test-run-directory` interaction with `--test-runtool`.
- rust-lang/rust#137096 -- Stabilization (and
rename) of the rustdoc `--test-runtool` and `--test-runtool-arg` CLI
args, and drops `--enable-per-target-ignores` unconditionally enabling
it.

## Test coverage

Cargo tests:
-
[artifact_dep::cross_doctests_works_with_artifacts](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/artifact_dep.rs#L1248-L1326)
-- Checks that doctest has access to the artifact dependencies.
-
[build_script::duplicate_script_with_extra_env](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/build_script.rs#L5514-L5614)
-- Checks that build-script env and cfg values are correctly handled on
host versus target when cross running doctests.
-
[cross_compile::cross_tests](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/cross_compile.rs#L416-L502)
-- Basic test that cross-compiled tests work.
-
[cross_compile::doctest_xcompile_linker](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/cross_compile.rs#L1139-L1182)
-- Checks that the linker config argument works.
-
[custom_target::custom_target_minimal](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/custom_target.rs#L39-L71)
-- Checks that `.json` targets work with rustdoc cross tests.
-
[test::cargo_test_doctest_xcompile_ignores](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/test.rs#L4743-L4777)
-- Checks the `ignore-*` syntax works.
-
[test::cargo_test_doctest_xcompile_runner](https://github.com/ehuss/cargo/blob/2603268cda3e32565ac27ee642f2b755fa590bac/tests/testsuite/test.rs#L4783-L4863)
-- Checks runner with cross doctests.
-
[test::cargo_test_doctest_xcompile_no_runner](https://github.com/ehuss/cargo/blob/2603268cda3e32565ac27ee642f2b755fa590bac/tests/testsuite/test.rs#L4869-L4907)
-- Checks cross doctests without a runner.

Rustdoc tests:
-
[run-make/doctest-runtool](https://github.com/rust-lang/rust/tree/25cdf1f67463c9365d8d83778c933ec7480e940b/tests/run-make/doctests-runtool)
-- Tests behavior of `--test-run-directory` with relative paths of the
runner.
-
[rustdoc/doctest/doctest-runtool](https://github.com/rust-lang/rust/blob/25cdf1f67463c9365d8d83778c933ec7480e940b/tests/rustdoc/doctest/doctest-runtool.rs)
-- Tests for `--test-runtool` and `--test-runtool-arg`.

## Future concerns

There have been some discussions
(rust-lang/testing-devex-team#5) about
changing how doctests are driven. My understanding is that stabilizing
this should not affect those plans, since if cargo becomes the driver,
it will simply need to build things with `--target` and use the
appropriate runner.

## Change notes

This PR changed tests a little:
- artifact_dep::no_cross_doctests_works_with_artifacts was changed now
that doctests actually work.
- cross_compile::cross_tests was changed to properly check doctests.
- cross_compile::no_cross_doctests dropped since it is no longer
relevant.
- standard_lib::doctest didn't need `-Zdoctest-xcompile` since
`-Zbuild-std` no longer uses a target.
- test::cargo_test_doctest_xcompile was removed since it is a duplicate
of cross_compile::cross_tests

I think this should probably wait until the next release cutoff, moving
this to 1.89 (will update the PR accordingly if that happens).