"attempting to create a duplicate library" when multiple python versions with same major+minor are used (different patch version)

🐞 bug report

Affected Rule

Maybe pip.parse?

Is this a regression?

It appears to be, yes. I bisected from 1.6.1 to 1.7.0 and found that commit 7b88c87 (PR #3243) is the first bad commit.

Description

We're getting a "duplicate library" error.

Error in fail: attempting to create a duplicate library pypi_313_absl_py_py3_none_any_9824a48b for absl_py

🔬 Minimal Reproduction

TL;DR: running multiple python versions with the same minor version (eg 3.13.4 and 3.13.6) appears to break things.

I'll see what I can do about making a minimal reproduction.

Background

We have things set up so that each python target is actually a macro wrapping py_* for the current python version and py_* for the "pynext" python version. Example:

load("@rules_python//python:defs.bzl", _py_binary = "py_binary", _py_test = "py_test")
load("@python_versions//3.13:defs.bzl", _py_binary_next = "py_binary", _py_test_next = "py_test")  # py version kept in sync with MODULE.bazel
def pyle_py_binary(**kwargs):
    _py_binary(**kwargs)
    _py_binary_next(**kwargs)

And thus our MODULE.bazel file has:

PYTHON_VERSION = "3.13.4"
PYNEXT_VERSION = "3.13.6"  # must be different from PYTHON_VERSION. Before 1.7.0, a different **patch** version was sufficient

python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
    ...,
    is_default = True,
    python_version = PYTHON_VERSION,
)
python.toolchain(
    ...,
    is_default = False,
    python_version = PYNEXT_VERSION,
)
use_repo(python, PYTHON_VERSION_NAME, PYNEXT_VERSION_NAME, "python_versions")

pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
    experimental_index_url = ...,
    experimental_index_url_overrides = ...,
    hub_name = "pypi",
    python_version = PYTHON_VERSION,
    requirements_lock = "//:requirements.txt",
)
pip.parse(
    experimental_index_url = ...,
    experimental_index_url_overrides = ...,
    hub_name = "pypi",
    python_version = PYNEXT_VERSION,
    requirements_lock = "//:requirements.txt",
)

You'll notice that we're using the same requirements.txt lock file and hub_name for both current and "pynext" versions.

We then use build/test tags to only build/test the current python version or the "pynext" python version:

# (default: --test_tag_filters=-pynext, only run against the current python version)
bazel test //...

# optionally test against both versions
bazel test --test_tag_filters= //...

# optionally test against only the "pynext" version
bazel test --test_tag_filters=pynext //...

🔥 Exception or Error

 bazel build --nobuild //...
INFO: Invocation ID: 8783fed4-2abe-4061-a080-4b3640de62a1
ERROR: /usr/local/google/home/dthor/.cache/bazel/_bazel_dthor/0f8c52850e7230283fc2f8033149fba2/external/rules_python+/python/private/pypi/hub_builder.bzl:185:13: Traceback (most recent call last):
        File "/usr/local/google/home/dthor/.cache/bazel/_bazel_dthor/0f8c52850e7230283fc2f8033149fba2/external/rules_python+/python/private/pypi/extension.bzl", line 390, column 25, in _pip_impl
                mods = parse_modules(module_ctx, enable_pipstar = rp_config.enable_pipstar)
        File "/usr/local/google/home/dthor/.cache/bazel/_bazel_dthor/0f8c52850e7230283fc2f8033149fba2/external/rules_python+/python/private/pypi/extension.bzl", line 280, column 30, in parse_modules
                builder.pip_parse(
        File "/usr/local/google/home/dthor/.cache/bazel/_bazel_dthor/0f8c52850e7230283fc2f8033149fba2/external/rules_python+/python/private/pypi/hub_builder.bzl", line 58, column 47, in lambda
                pip_parse = lambda *a, **k: _pip_parse(self, *a, **k),
        File "/usr/local/google/home/dthor/.cache/bazel/_bazel_dthor/0f8c52850e7230283fc2f8033149fba2/external/rules_python+/python/private/pypi/hub_builder.bzl", line 125, column 22, in _pip_parse
                _create_whl_repos(
        File "/usr/local/google/home/dthor/.cache/bazel/_bazel_dthor/0f8c52850e7230283fc2f8033149fba2/external/rules_python+/python/private/pypi/hub_builder.bzl", line 440, column 29, in _create_whl_repos
                _add_whl_library(
        File "/usr/local/google/home/dthor/.cache/bazel/_bazel_dthor/0f8c52850e7230283fc2f8033149fba2/external/rules_python+/python/private/pypi/hub_builder.bzl", line 185, column 13, in _add_whl_library
                fail("attempting to create a duplicate library {} for {}".format(
Error in fail: attempting to create a duplicate library pypi_313_absl_py_py3_none_any_9824a48b for absl_py
ERROR: error evaluating module extension @@rules_python+//python/extensions:pip.bzl%pip
INFO: Elapsed time: 26.869s
INFO: 0 processes.
ERROR: Build did NOT complete successfully
Loading: 363 packages loaded
    currently loading: 
    Fetching module extension @@rules_python+//python/extensions:pip.bzl%pip; Fetch package lists from PyPI index 26s

🌍 Your Environment

Operating System:

Output of bazel version:

$ bazel version
Bazelisk version: v1.26.0
Build label: 8.3.1
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Mon Jun 30 16:23:40 2025 (1751300620)
Build timestamp: 1751300620
Build timestamp as int: 1751300620

Rules_python version:

1.7.0

Anything else relevant?

Removing any support for "pynext" allows us to bump to 1.7.0. Additionally, bumping PYNEXT_VERSION to a different minor version than PYTHON_VERSION appears to work (eg current=3.13, next=3.14), but I can't fully confirm this yet because our requirements are not compatible with python 3.14.

This might be PEBKAC - if there's a more "correct" way of running multiple python versions please let me know. Our requirements for such are:

Do not build/test/run any "pynext" version by default
Only need to support versions N and N+1, where N might be a major, minor, or patch version
Must use the same requirements.txt lock file.
Must use the same pypi hub name
- Or more accurately: the third-party deps of a target must not change between python versions.