rustfmt: Check that a token can begin a nonterminal kind before parsing it as a macro arg by compiler-errors · Pull Request #120218 · rust-lang/rust

compiler-errors

compiler-errors

@compiler-errors

@compiler-errors

@compiler-errors

@bors bors added S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

and removed S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

labels

Jan 22, 2024

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request

Jan 22, 2024
…=calebcartwright,ytmimi

rustfmt: Check that a token can begin a nonterminal kind before parsing it as a macro arg

r? `@ytmimi` and/or `@calebcartwright`
cc `@fmease`

I'm putting this on r-l/rust since it should fix the nightly rustfmt version. If you don't care about having this regression until the next rustfmt->rust sync, then I can move that PR over to r-l/rustfmt.

---

> Any idea why the formatting would have changed [from rust-lang#119099]?

**Copied over explanation:**

This has to do with the weirdness of the way that `parse_macro_arg` works. Unlike parsing nonterminal args in a macro-by-example, it eagerly tries, for example, to parse a type without checking that the beginning token may begin a type:

https://github.com/rust-lang/rustfmt/blob/bf967319e258acb9b1648a952bba52665eceaf52/src/parse/macros/mod.rs#L54

Contrast this to the nonterminal parsing code, which first checks that the nonterminal may begin with a given token:

https://github.com/rust-lang/rust/blob/ef71f1047e04438181d7cb925a833e2ada6ab390/compiler/rustc_parse/src/parser/nonterminal.rs#L47

In rust-lang#119099, `@fmease` implemented a change so that `const Tr` would be parsed as `dyn const Tr` (a trait object to a const trait) in edition 2015.

This is okay for the purposes of macros, because he explicitly made sure that `const` did not get added to the list of tokens that may begin a `:ty` nonterminal kind: rust-lang#119099 (comment)

However, since rustfmt is not so careful about eagerly parsing macro args before checking that they're legal in macro position, this changed the way that the string of tokens is being parsed into macro args.

bors added a commit to rust-lang-ci/rust that referenced this pull request

Jan 22, 2024
…iaskrgr

Rollup of 11 pull requests

Successful merges:

 - rust-lang#117910 (Refactor uses of `objc_msgSend` to no longer have clashing definitions)
 - rust-lang#118639 (Undeprecate lint `unstable_features` and make use of it in the compiler)
 - rust-lang#119801 (Fix deallocation with wrong allocator in (A)Rc::from_box_in)
 - rust-lang#120058 (bootstrap: improvements for compiler builds)
 - rust-lang#120059 (Make generic const type mismatches not hide trait impls from the trait solver)
 - rust-lang#120097 (Report unreachable subpatterns consistently)
 - rust-lang#120137 (Validate AggregateKind types in MIR)
 - rust-lang#120164 (`maybe_lint_impl_trait`: separate `is_downgradable` from `is_object_safe`)
 - rust-lang#120181 (Allow any `const` expression blocks in `thread_local!`)
 - rust-lang#120204 (Builtin macros effectively have implicit #[collapse_debuginfo(yes)])
 - rust-lang#120218 (rustfmt: Check that a token can begin a nonterminal kind before parsing it as a macro arg)

r? `@ghost`
`@rustbot` modify labels: rollup

bors added a commit to rust-lang-ci/rust that referenced this pull request

Jan 22, 2024
…iaskrgr

Rollup of 10 pull requests

Successful merges:

 - rust-lang#117910 (Refactor uses of `objc_msgSend` to no longer have clashing definitions)
 - rust-lang#118639 (Undeprecate lint `unstable_features` and make use of it in the compiler)
 - rust-lang#119801 (Fix deallocation with wrong allocator in (A)Rc::from_box_in)
 - rust-lang#120058 (bootstrap: improvements for compiler builds)
 - rust-lang#120059 (Make generic const type mismatches not hide trait impls from the trait solver)
 - rust-lang#120097 (Report unreachable subpatterns consistently)
 - rust-lang#120137 (Validate AggregateKind types in MIR)
 - rust-lang#120164 (`maybe_lint_impl_trait`: separate `is_downgradable` from `is_object_safe`)
 - rust-lang#120181 (Allow any `const` expression blocks in `thread_local!`)
 - rust-lang#120218 (rustfmt: Check that a token can begin a nonterminal kind before parsing it as a macro arg)

r? `@ghost`
`@rustbot` modify labels: rollup

rust-timer added a commit to rust-lang-ci/rust that referenced this pull request

Jan 22, 2024
Rollup merge of rust-lang#120218 - compiler-errors:parse_macro_arg, r=calebcartwright,ytmimi

rustfmt: Check that a token can begin a nonterminal kind before parsing it as a macro arg

r? ``@ytmimi`` and/or ``@calebcartwright``
cc ``@fmease``

I'm putting this on r-l/rust since it should fix the nightly rustfmt version. If you don't care about having this regression until the next rustfmt->rust sync, then I can move that PR over to r-l/rustfmt.

---

> Any idea why the formatting would have changed [from rust-lang#119099]?

**Copied over explanation:**

This has to do with the weirdness of the way that `parse_macro_arg` works. Unlike parsing nonterminal args in a macro-by-example, it eagerly tries, for example, to parse a type without checking that the beginning token may begin a type:

https://github.com/rust-lang/rustfmt/blob/bf967319e258acb9b1648a952bba52665eceaf52/src/parse/macros/mod.rs#L54

Contrast this to the nonterminal parsing code, which first checks that the nonterminal may begin with a given token:

https://github.com/rust-lang/rust/blob/ef71f1047e04438181d7cb925a833e2ada6ab390/compiler/rustc_parse/src/parser/nonterminal.rs#L47

In rust-lang#119099, ``@fmease`` implemented a change so that `const Tr` would be parsed as `dyn const Tr` (a trait object to a const trait) in edition 2015.

This is okay for the purposes of macros, because he explicitly made sure that `const` did not get added to the list of tokens that may begin a `:ty` nonterminal kind: rust-lang#119099 (comment)

However, since rustfmt is not so careful about eagerly parsing macro args before checking that they're legal in macro position, this changed the way that the string of tokens is being parsed into macro args.

This was referenced

Jun 12, 2024