fix(lexer): Only allow horizontal whitespace in frontmatter by epage · Pull Request #146106 · rust-lang/rust

added 5 commits

September 1, 2025 20:51
In writing up the reference for frontmatter, I realized that we probably
shouldn't be accepting Unicode Line Ending characters between the code
fence and infostring or trailing after the infostring or a code fence.

In digging into the unicode specification we use for Whitespace, it
divides it up into categories, so I'm deferring to what it says for
horizontal whitespace for what should be used within a line.

Note, I am leaving out support for Unicde Default Ignorable characters.
I figure that can be discussed outside of this change within the
reference and tracking issue.

@rustbot rustbot added S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

T-compiler

Relevant to the compiler team, which will review and decide on the PR/issue.

labels

Sep 2, 2025

bjorn3

@bors bors added S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

and removed S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

labels

Sep 3, 2025

tgross35 added a commit to tgross35/rust that referenced this pull request

Sep 3, 2025
fix(lexer): Only allow horizontal whitespace in frontmatter

In writing up the reference for frontmatter, I realized that we probably
shouldn't be accepting Unicode Line Ending characters between the code
fence and infostring or trailing after the infostring or a code fence.

In digging into the unicode specification we use for Whitespace, it
divides it up into categories, so I'm deferring to what it says for
horizontal whitespace for what should be used within a line.

Note, I am leaving out support for Unicode Default Ignorable characters.
I figure that can be discussed outside of this change within the
reference and tracking issue.

Fixes rust-lang#145971

Frontmatter tracking issue: rust-lang#136889

bors added a commit that referenced this pull request

Sep 3, 2025
Rollup of 8 pull requests

Successful merges:

 - #139113 (unstable book: in a sanitizer example, check the code)
 - #145279 (Constify conversion traits (part 1))
 - #145414 (unicode-table-generator refactors)
 - #145823 (editorconfig: don't use nonexistent syntax)
 - #145944 (std: Start supporting WASIp2 natively )
 - #145961 (resolve: Avoid a regression from splitting prelude into two scopes)
 - #146032 (Explicity disable LSX feature for `loongarch64-unknown-none` target)
 - #146106 (fix(lexer): Only allow horizontal whitespace in frontmatter )

r? `@ghost`
`@rustbot` modify labels: rollup

bors added a commit that referenced this pull request

Sep 3, 2025
Rollup of 8 pull requests

Successful merges:

 - #139113 (unstable book: in a sanitizer example, check the code)
 - #145279 (Constify conversion traits (part 1))
 - #145414 (unicode-table-generator refactors)
 - #145823 (editorconfig: don't use nonexistent syntax)
 - #145944 (std: Start supporting WASIp2 natively )
 - #145961 (resolve: Avoid a regression from splitting prelude into two scopes)
 - #146032 (Explicity disable LSX feature for `loongarch64-unknown-none` target)
 - #146106 (fix(lexer): Only allow horizontal whitespace in frontmatter )

r? `@ghost`
`@rustbot` modify labels: rollup

Zalathar added a commit to Zalathar/rust that referenced this pull request

Sep 3, 2025
fix(lexer): Only allow horizontal whitespace in frontmatter

In writing up the reference for frontmatter, I realized that we probably
shouldn't be accepting Unicode Line Ending characters between the code
fence and infostring or trailing after the infostring or a code fence.

In digging into the unicode specification we use for Whitespace, it
divides it up into categories, so I'm deferring to what it says for
horizontal whitespace for what should be used within a line.

Note, I am leaving out support for Unicode Default Ignorable characters.
I figure that can be discussed outside of this change within the
reference and tracking issue.

Fixes rust-lang#145971

Frontmatter tracking issue: rust-lang#136889

bors added a commit that referenced this pull request

Sep 3, 2025
Rollup of 9 pull requests

Successful merges:

 - #145279 (Constify conversion traits (part 1))
 - #145414 (unicode-table-generator refactors)
 - #145823 (editorconfig: don't use nonexistent syntax)
 - #145944 (std: Start supporting WASIp2 natively )
 - #145961 (resolve: Avoid a regression from splitting prelude into two scopes)
 - #146032 (Explicity disable LSX feature for `loongarch64-unknown-none` target)
 - #146106 (fix(lexer): Only allow horizontal whitespace in frontmatter )
 - #146112 (don't uppercase error messages)
 - #146154 (CI: rfl: move job forward to Linux v6.17-rc3 plus 2 commits)

r? `@ghost`
`@rustbot` modify labels: rollup

bors added a commit that referenced this pull request

Sep 3, 2025
Rollup of 8 pull requests

Successful merges:

 - #145279 (Constify conversion traits (part 1))
 - #145414 (unicode-table-generator refactors)
 - #145823 (editorconfig: don't use nonexistent syntax)
 - #145944 (std: Start supporting WASIp2 natively )
 - #145961 (resolve: Avoid a regression from splitting prelude into two scopes)
 - #146032 (Explicity disable LSX feature for `loongarch64-unknown-none` target)
 - #146106 (fix(lexer): Only allow horizontal whitespace in frontmatter )
 - #146154 (CI: rfl: move job forward to Linux v6.17-rc3 plus 2 commits)

r? `@ghost`
`@rustbot` modify labels: rollup

rust-timer added a commit that referenced this pull request

Sep 3, 2025
Rollup merge of #146106 - epage:whitespace, r=fee1-dead

fix(lexer): Only allow horizontal whitespace in frontmatter

In writing up the reference for frontmatter, I realized that we probably
shouldn't be accepting Unicode Line Ending characters between the code
fence and infostring or trailing after the infostring or a code fence.

In digging into the unicode specification we use for Whitespace, it
divides it up into categories, so I'm deferring to what it says for
horizontal whitespace for what should be used within a line.

Note, I am leaving out support for Unicode Default Ignorable characters.
I figure that can be discussed outside of this change within the
reference and tracking issue.

Fixes #145971

Frontmatter tracking issue: #136889

epage added a commit to epage/cargo that referenced this pull request

Sep 16, 2025
This is to align with rust-lang/rust#146106.

While no tests were affected by this,
I definitely broke a lot of tests when making this change.

This was referenced

Sep 16, 2025

epage added a commit to epage/cargo that referenced this pull request

Sep 19, 2025
This is to align with rust-lang/rust#146106.

While no tests were affected by this,
I definitely broke a lot of tests when making this change.

github-merge-queue bot pushed a commit to rust-lang/cargo that referenced this pull request

Sep 21, 2025
### What does this PR try to resolve?

This is to align with rust-lang/rust#146106 and is part of #12207.

### How to test and review this PR?

While no tests were affected by this,
I definitely broke a lot of tests when making this change.

@epage epage mentioned this pull request

Oct 23, 2025

10 tasks