Implement Modified Condition/Decision Coverage by ZhuUx · Pull Request #123409 · rust-lang/rust
rustbot
added
S-waiting-on-review
labels
Apr 3, 2024
ZhuUx
mentioned this pull request
rustbot
added
A-testsuite
labels
Apr 4, 2024rust-timer added a commit to rust-lang-ci/rust that referenced this pull request
Apr 20, 2024Rollup merge of rust-lang#123409 - ZhuUx:master, r=oli-obk Implement Modified Condition/Decision Coverage This is an implementation based on llvm backend support (>= 18) by `@evodius96` and branch coverage support by `@Zalathar.` ### Major changes: * Add -Zcoverage-options=mcdc as switch. Now coverage options accept either `no-branch`, `branch`, or `mcdc`. `mcdc` also enables `branch` because it is essential to work. * Add coverage mapping for MCDCBranch and MCDCDecision. Note that MCDCParameter evolves from llvm 18 to llvm 19. The mapping in rust side mainly references to 19 and is casted to 18 types in llvm wrapper. * Add wrapper for mcdc instrinc functions from llvm. And inject associated statements to mir. * Add BcbMappingKind::Decision, I'm not sure is it proper but can't find a better way temporarily. * Let coverage-dump support parsing MCDCBranch and MCDCDecision from llvm ir. * Add simple tests to check whether mcdc works. * Same as clang, currently rustc does not generate instrument for decision with more than 6 condtions or only 1 condition due to considerations of resource. ### Implementation Details 1. To get information about conditions and decisions, `MCDCState` in `BranchInfoBuilder` is used during hir lowering to mir. For expressions with logical op we call `Builder::visit_coverage_branch_operation` to record its sub conditions, generate condition ids for them and save their spans (to construct the span of whole decision). This process mainly references to the implementation in clang and is described in comments over `MCDCState::record_conditions`. Also true marks and false marks introduced by branch coverage are used to detect where the decision evaluation ends: the next id of the condition == 0. 2. Once the `MCDCState::decision_stack` popped all recorded conditions, we can ensure that the decision is checked over and push it into `decision_spans`. We do not manually insert decision span to avoid complexity from then_else_break in nested if scopes. 3. When constructing CoverageSpans, add condition info to BcbMappingKind::Branch and decision info to BcbMappingKind::Decision. If the branch mapping has non-zero condition id it will be transformed to MCDCBranch mapping and insert `CondBitmapUpdate` statements to its evaluated blocks. While decision bcb mapping will insert `TestVectorBitmapUpdate` in all its end blocks. ### Usage ```bash echo "[build]\nprofiler=true" >> config.toml ./x build --stage 1 ./x test tests/coverage/mcdc_if.rs ``` to build the compiler and run tests. ```shell export PATH=path/to/llvm-build:$PATH rustup toolchain link mcdc build/host/stage1 cargo +mcdc rustc --bin foo -- -Cinstrument-coverage -Zcoverage-options=mcdc cd target/debug LLVM_PROFILE_FILE="foo.profraw" ./foo llvm-profdata merge -sparse foo.profraw -o foo.profdata llvm-cov show ./foo -instr-profile=foo.profdata --show-mcdc ``` to check "foo" code. ### Problems to solve For now decision mapping will insert statements to its all end blocks, which may be optimized by inserting a final block of the decision. To do this we must also trace the evaluated value at each end of the decision and join them separately. This implementation is not heavily tested so there should be some unrevealed issues. We are going to check our rust products in the next. Please let me know if you had any suggestions or comments.
This was referenced
Apr 21, 2024
ZhuUx
mentioned this pull request
GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this pull request
Apr 22, 2024coverage: Prepare for improved branch coverage When trying to rebase my new branch coverage work (including rust-lang#124154) on top of the introduction of MC/DC coverage (rust-lang#123409), I found it a lot harder than anticipated. With the benefit of hindsight, the branch coverage code and MC/DC code have become more interdependent than I'm happy with. This PR therefore disentangles them a bit, so that it will be easier for both areas of code to evolve independently without interference. --- This PR also includes a few extra branch coverage tests that I had sitting around from my current branch coverage work. They mostly just demonstrate that certain language constructs listed in rust-lang#124118 currently don't have branch coverage support. `@rustbot` label +A-code-coverage
GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this pull request
Apr 22, 2024coverage: Prepare for improved branch coverage When trying to rebase my new branch coverage work (including rust-lang#124154) on top of the introduction of MC/DC coverage (rust-lang#123409), I found it a lot harder than anticipated. With the benefit of hindsight, the branch coverage code and MC/DC code have become more interdependent than I'm happy with. This PR therefore disentangles them a bit, so that it will be easier for both areas of code to evolve independently without interference. --- This PR also includes a few extra branch coverage tests that I had sitting around from my current branch coverage work. They mostly just demonstrate that certain language constructs listed in rust-lang#124118 currently don't have branch coverage support. ``@rustbot`` label +A-code-coverage
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request
Apr 23, 2024Rollup merge of rust-lang#124217 - Zalathar:pre-branch, r=oli-obk coverage: Prepare for improved branch coverage When trying to rebase my new branch coverage work (including rust-lang#124154) on top of the introduction of MC/DC coverage (rust-lang#123409), I found it a lot harder than anticipated. With the benefit of hindsight, the branch coverage code and MC/DC code have become more interdependent than I'm happy with. This PR therefore disentangles them a bit, so that it will be easier for both areas of code to evolve independently without interference. --- This PR also includes a few extra branch coverage tests that I had sitting around from my current branch coverage work. They mostly just demonstrate that certain language constructs listed in rust-lang#124118 currently don't have branch coverage support. ``@rustbot`` label +A-code-coverage
aytey
mentioned this pull request
This was referenced
Apr 29, 2024bors added a commit to rust-lang-ci/rust that referenced this pull request
Apr 29, 2024…ons, r=oli-obk MCDC coverage: support nested decision coverage rust-lang#123409 provided the initial MCDC coverage implementation. As referenced in rust-lang#124144, it does not currently support "nested" decisions, like the following example : ```rust fn nested_if_in_condition(a: bool, b: bool, c: bool) { if a && if b || c { true } else { false } { say("yes"); } else { say("no"); } } ``` Note that there is an if-expression (`if b || c ...`) embedded inside a boolean expression in the decision of an outer if-expression. This PR proposes a workaround for this cases, by introducing a Decision context stack, and by handing several `temporary condition bitmaps` instead of just one. When instrumenting boolean expressions, if the current node is a leaf condition (i.e. not a `||`/`&&` logical operator nor a `!` not operator), we insert a new decision context, such that if there are more boolean expressions inside the condition, they are handled as separate expressions. On the codegen LLVM side, we allocate as many `temp_cond_bitmap`s as necessary to handle the maximum encountered decision depth.
bors added a commit to rust-lang-ci/rust that referenced this pull request
Apr 29, 2024…ons, r=Zalathar MCDC coverage: support nested decision coverage rust-lang#123409 provided the initial MCDC coverage implementation. As referenced in rust-lang#124144, it does not currently support "nested" decisions, like the following example : ```rust fn nested_if_in_condition(a: bool, b: bool, c: bool) { if a && if b || c { true } else { false } { say("yes"); } else { say("no"); } } ``` Note that there is an if-expression (`if b || c ...`) embedded inside a boolean expression in the decision of an outer if-expression. This PR proposes a workaround for this cases, by introducing a Decision context stack, and by handing several `temporary condition bitmaps` instead of just one. When instrumenting boolean expressions, if the current node is a leaf condition (i.e. not a `||`/`&&` logical operator nor a `!` not operator), we insert a new decision context, such that if there are more boolean expressions inside the condition, they are handled as separate expressions. On the codegen LLVM side, we allocate as many `temp_cond_bitmap`s as necessary to handle the maximum encountered decision depth.
jieyouxu added a commit to jieyouxu/rust that referenced this pull request
Apr 29, 2024coverage: Avoid hard-coded values when visiting logical ops This is a tiny little thing that I noticed during the final review of rust-lang#123409, and I didn't want to hold up the whole PR just for this. Instead of separately hard-coding the operation being visited, we can get it from the match arm pattern by using an as-pattern. `@rustbot` label +A-code-coverage
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request
Apr 29, 2024Rollup merge of rust-lang#124508 - Zalathar:op, r=jieyouxu coverage: Avoid hard-coded values when visiting logical ops This is a tiny little thing that I noticed during the final review of rust-lang#123409, and I didn't want to hold up the whole PR just for this. Instead of separately hard-coding the operation being visited, we can get it from the match arm pattern by using an as-pattern. `@rustbot` label +A-code-coverage
bors added a commit to rust-lang-ci/rust that referenced this pull request
Apr 30, 2024…errors coverage: Replace boolean options with a `CoverageLevel` enum After rust-lang#123409, and some discussion at rust-lang#79649 (comment) and rust-lang#124120, it became clear to me that we should have a unified concept of “coverage level”, instead of having several separate boolean flags that aren't actually independent. This PR therefore introduces a `CoverageLevel` enum, to replace the existing boolean flags for `branch` and `mcdc`. The `no-branch` value (for `-Zcoverage-options`) has been renamed to `block`, instructing the compiler to only instrument for block coverage, with no branch coverage or MD/DC instrumentation. `@rustbot` label +A-code-coverage cc `@ZhuUx` `@Lambdaris` `@RenjiSann`
This was referenced
Apr 30, 2024matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request
May 3, 2024coverage: Clean up creation of MC/DC condition bitmaps This PR improves the code for creating and initializing [MC/DC](https://en.wikipedia.org/wiki/Modified_condition/decision_coverage) condition bitmap variables, as introduced by rust-lang#123409 and modified by rust-lang#124255. - The condition bitmap variables are now created eagerly at the start of per-function codegen, via a new `init_coverage` method in `CoverageInfoBuilderMethods`. This avoids having to retroactively create the bitmaps while doing codegen for an individual coverage statement. - As a result, we can now create and initialize those bitmaps using existing safe APIs, instead of having to perform our own unsafe call to `llvm::LLVMBuildAlloca`. - This PR also tweaks the way we count the number of condition bitmaps needed, by tracking the total number of bitmaps needed (max depth + 1), instead of only tracking the maximum depth. This reduces the potential for subtle off-by-one confusion.
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request
May 3, 2024Rollup merge of rust-lang#124555 - Zalathar:init-coverage, r=nnethercote coverage: Clean up creation of MC/DC condition bitmaps This PR improves the code for creating and initializing [MC/DC](https://en.wikipedia.org/wiki/Modified_condition/decision_coverage) condition bitmap variables, as introduced by rust-lang#123409 and modified by rust-lang#124255. - The condition bitmap variables are now created eagerly at the start of per-function codegen, via a new `init_coverage` method in `CoverageInfoBuilderMethods`. This avoids having to retroactively create the bitmaps while doing codegen for an individual coverage statement. - As a result, we can now create and initialize those bitmaps using existing safe APIs, instead of having to perform our own unsafe call to `llvm::LLVMBuildAlloca`. - This PR also tweaks the way we count the number of condition bitmaps needed, by tracking the total number of bitmaps needed (max depth + 1), instead of only tracking the maximum depth. This reduces the potential for subtle off-by-one confusion.
bors added a commit to rust-lang-ci/rust that referenced this pull request
May 5, 2024coverage: Split out MC/DC mappings from `BcbMappingKind` These variants were added to `BcbMappingKind` as part of the [MC/DC coverage](https://en.wikipedia.org/wiki/Modified_Condition/Decision_Coverage) implementation in rust-lang#123409, because that was the path-of-least-resistance for integrating them into the existing code. However, they ultimately represent complex concepts that the enum was not intended to handle, leading to more complexity in the code that processes them. This PR therefore follows in the footsteps of rust-lang#124545, and splits the MC/DC mappings out into their own dedicated vectors of structs. After that, `BcbMappingKind` itself ends up having only one variant (`Code`), so this PR also flattens that enum into its enclosing struct, renamed to `mapping::CodeMapping`. --- No functional changes. This will conflict slightly with rust-lang#124571, but hopefully that should be easy to resolve either way. `@rustbot` label +A-code-coverage
Zalathar added a commit to Zalathar/rust that referenced this pull request
Aug 8, 2025coverage: Remove all unstable support for MC/DC instrumentation Preliminary support for a partial implementation of “Modified Condition/Decision Coverage” instrumentation was added behind the unstable flag `-Zcoverage-options=mcdc` in 2024. These are the most substantial PRs involved: - rust-lang#123409 - rust-lang#126733 At the time, I accepted these PRs with relatively modest scrutiny, because I did not want to stand in the way of independent work on MC/DC instrumentation. My hope was that ongoing work by interested contributors would lead to the code becoming clearer and more maintainable over time. --- However, that MC/DC code has proven itself to be a major burden on overall maintenance of coverage instrumentation, and a major obstacle to other planned improvements, such as internal changes needed for proper support of macro expansion regions. I have also become reluctant to accept any further MC/DC-related changes that would increase this burden. That tension has resulted in an unhappy impasse. On one hand, the present MC/DC implementation is not yet complete, and shows little sign of being complete at an acceptable level of code quality in the foreseeable future. On the other hand, the continued existence of this partial MC/DC implementation is imposing serious maintenance burdens on every other aspect of coverage instrumentation, and is preventing some of the very improvements that would make it easier to accept expanded MC/DC support in the future. While I know this will be disappointing to some, I think the healthy way forward is accept that I made the wrong call in accepting the current implementation, and to remove it entirely from the compiler.
rust-timer added a commit that referenced this pull request
Aug 8, 2025Rollup merge of #144999 - Zalathar:remove-mcdc, r=oli-obk coverage: Remove all unstable support for MC/DC instrumentation Preliminary support for a partial implementation of “Modified Condition/Decision Coverage” instrumentation was added behind the unstable flag `-Zcoverage-options=mcdc` in 2024. These are the most substantial PRs involved: - #123409 - #126733 At the time, I accepted these PRs with relatively modest scrutiny, because I did not want to stand in the way of independent work on MC/DC instrumentation. My hope was that ongoing work by interested contributors would lead to the code becoming clearer and more maintainable over time. --- However, that MC/DC code has proven itself to be a major burden on overall maintenance of coverage instrumentation, and a major obstacle to other planned improvements, such as internal changes needed for proper support of macro expansion regions. I have also become reluctant to accept any further MC/DC-related changes that would increase this burden. That tension has resulted in an unhappy impasse. On one hand, the present MC/DC implementation is not yet complete, and shows little sign of being complete at an acceptable level of code quality in the foreseeable future. On the other hand, the continued existence of this partial MC/DC implementation is imposing serious maintenance burdens on every other aspect of coverage instrumentation, and is preventing some of the very improvements that would make it easier to accept expanded MC/DC support in the future. While I know this will be disappointing to some, I think the healthy way forward is accept that I made the wrong call in accepting the current implementation, and to remove it entirely from the compiler.
github-actions bot pushed a commit to rust-lang/miri that referenced this pull request
Aug 9, 2025coverage: Remove all unstable support for MC/DC instrumentation Preliminary support for a partial implementation of “Modified Condition/Decision Coverage” instrumentation was added behind the unstable flag `-Zcoverage-options=mcdc` in 2024. These are the most substantial PRs involved: - rust-lang/rust#123409 - rust-lang/rust#126733 At the time, I accepted these PRs with relatively modest scrutiny, because I did not want to stand in the way of independent work on MC/DC instrumentation. My hope was that ongoing work by interested contributors would lead to the code becoming clearer and more maintainable over time. --- However, that MC/DC code has proven itself to be a major burden on overall maintenance of coverage instrumentation, and a major obstacle to other planned improvements, such as internal changes needed for proper support of macro expansion regions. I have also become reluctant to accept any further MC/DC-related changes that would increase this burden. That tension has resulted in an unhappy impasse. On one hand, the present MC/DC implementation is not yet complete, and shows little sign of being complete at an acceptable level of code quality in the foreseeable future. On the other hand, the continued existence of this partial MC/DC implementation is imposing serious maintenance burdens on every other aspect of coverage instrumentation, and is preventing some of the very improvements that would make it easier to accept expanded MC/DC support in the future. While I know this will be disappointing to some, I think the healthy way forward is accept that I made the wrong call in accepting the current implementation, and to remove it entirely from the compiler.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters