fix(pypi): normalize extras in requirement strings per PEP 685 by kevinpark1217 · Pull Request #3588 · bazel-contrib/rules_python
Extras parsed from requirement strings (e.g., from requirements.txt) were not being normalized, causing mismatches when evaluating PEP 508 marker expressions. For example, `sqlalchemy[postgresql-psycopg2binary]` would fail to resolve `psycopg2-binary` because the marker expression `extra == "postgresql_psycopg2binary"` uses the underscore-normalized form per PEP 685, but the extras set contained the hyphenated form. This applies `normalize_name()` to extras during requirement parsing, consistent with how the package name itself is already normalized. Fixes bazel-contrib#3587
Normalize the literal string side of extra comparisons in marker_expr() so that hyphens/dots/case in wheel METADATA marker expressions (e.g., extra == "db-backend") match the already-normalized extras from requirement parsing (e.g., "db_backend"). Follows the same pattern as _ENV_ALIASES normalization for platform variables. Updated deps integration test to use un-normalized marker expressions as suggested in review.
…ctions Refactor the _ENV_ALIASES mechanism from dict-based lookups to function-based normalization, as suggested by @aignas. This unifies platform alias handling and PEP 685 extra name normalization into a single pattern: - pep508_env.bzl: Register normalize_name as the "extra" normalizer and wrap platform_machine_aliases in a lambda, both in create_env() - pep508_evaluate.bzl: Replace dict.get() with function calls using lambda x: x as default; remove separate if var_name == "extra" blocks - pep508_deps.bzl: Use create_env() inline (matching codebase pattern) to ensure _aliases normalization is available during marker evaluation
rickeylev pushed a commit to rickeylev/rules_python that referenced this pull request
Feb 23, 2026…-contrib#3588) ## Summary Extras parsed from requirement strings (e.g., from `requirements.txt`) were not being normalized, causing mismatches when evaluating PEP 508 marker expressions. For example, `sqlalchemy[postgresql-psycopg2binary]` would fail to resolve `psycopg2-binary` as a transitive dependency because the wheel METADATA marker expression `extra == "postgresql_psycopg2binary"` uses the underscore-normalized form (per PEP 685), while the extras set retained the original hyphenated form from the requirement string. ## Before ``` # requirements.txt sqlalchemy[postgresql-psycopg2binary]==2.0.36 # Parsed extras: ["postgresql-psycopg2binary"] # Marker evaluation: "postgresql-psycopg2binary" != "postgresql_psycopg2binary" -> MISS # Result: psycopg2-binary NOT included as a dependency ``` ## After ``` # requirements.txt sqlalchemy[postgresql-psycopg2binary]==2.0.36 # Parsed extras: ["postgresql_psycopg2binary"] (normalized) # Marker evaluation: "postgresql_psycopg2binary" == "postgresql_psycopg2binary" -> MATCH # Result: psycopg2-binary correctly included as a dependency ``` ## Changes - **`python/private/pypi/pep508_requirement.bzl`**: Apply `normalize_name()` to each extra during requirement parsing, consistent with how the package name is already normalized. - **`tests/pypi/pep508/requirement_tests.bzl`**: Updated existing test expectation for case normalization and added test case for hyphenated extras (`sqlalchemy[asyncio,postgresql-psycopg2binary,postgresql-asyncpg]`). - **`tests/pypi/pep508/deps_tests.bzl`**: Added `test_extras_with_hyphens_are_normalized` integration test confirming that dependencies gated behind hyphenated extras are correctly resolved. - **`CHANGELOG.md`**: Added entry under Unreleased > Fixed. Fixes bazel-contrib#3587 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> (cherry picked from commit 9fe42b1)
aignas added a commit to aignas/rules_python that referenced this pull request
Feb 23, 2026…-contrib#3588) ## Summary Extras parsed from requirement strings (e.g., from `requirements.txt`) were not being normalized, causing mismatches when evaluating PEP 508 marker expressions. For example, `sqlalchemy[postgresql-psycopg2binary]` would fail to resolve `psycopg2-binary` as a transitive dependency because the wheel METADATA marker expression `extra == "postgresql_psycopg2binary"` uses the underscore-normalized form (per PEP 685), while the extras set retained the original hyphenated form from the requirement string. ## Before ``` # requirements.txt sqlalchemy[postgresql-psycopg2binary]==2.0.36 # Parsed extras: ["postgresql-psycopg2binary"] # Marker evaluation: "postgresql-psycopg2binary" != "postgresql_psycopg2binary" -> MISS # Result: psycopg2-binary NOT included as a dependency ``` ## After ``` # requirements.txt sqlalchemy[postgresql-psycopg2binary]==2.0.36 # Parsed extras: ["postgresql_psycopg2binary"] (normalized) # Marker evaluation: "postgresql_psycopg2binary" == "postgresql_psycopg2binary" -> MATCH # Result: psycopg2-binary correctly included as a dependency ``` ## Changes - **`python/private/pypi/pep508_requirement.bzl`**: Apply `normalize_name()` to each extra during requirement parsing, consistent with how the package name is already normalized. - **`tests/pypi/pep508/requirement_tests.bzl`**: Updated existing test expectation for case normalization and added test case for hyphenated extras (`sqlalchemy[asyncio,postgresql-psycopg2binary,postgresql-asyncpg]`). - **`tests/pypi/pep508/deps_tests.bzl`**: Added `test_extras_with_hyphens_are_normalized` integration test confirming that dependencies gated behind hyphenated extras are correctly resolved. - **`CHANGELOG.md`**: Added entry under Unreleased > Fixed. Fixes bazel-contrib#3587 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters