pypi downloads blocked by experimental_downloader

pypi downloads blocked by experimental_downloader_config fail silently

🐞 bug report

Affected Rule

pip.parse with the bazel downloader enabled

Is this a regression?

not sure

Description

with some config like this:

pip.parse(
    experimental_extra_index_urls = [
        "https://download.pytorch.org/whl/cu121",
    ],
    experimental_index_url = "https://pypi.org/simple",
    experimental_index_url_overrides = {
        "torch": "https://download.pytorch.org/whl/cu121",
        "torchaudio": "https://download.pytorch.org/whl/cu121",
        "torchvision": "https://download.pytorch.org/whl/cu121",
    },
...
)

If your remote downloader config doesn't allow pytorch.org, rules_python can fail with this error:

Error in fail: Failed to download metadata for ["torch", "torchaudio", "torchvision"] for from urls: ["https://pypi.org/simple", "https://download.pytorch.org/whl/rocm6.2"]

Instead of the normal:

Error in download: java.io.IOException: Configured URL rewriter blocked all URLs: [https://download.pytorch.org/whl/rocm6.2/torch/]

I believe this is because of allow_fail = True https://github.com/bazelbuild/rules_python/blob/fcf7221c1e079307ff13d32239b7782d2f1dc48c/python/private/pypi/simpleapi_download.bzl#L208 and it makes it hard to spot that it's a misconfiguration on your downloader config.

🌍 Your Environment

Operating System:

linux aarch64

Output of bazel version:

95bbed03758a7651916d0708a892f5342a0421c5

Rules_python version:

fcf7221