Skip to content

pip.parse for rules_python_publish_deps doesn't set target_platforms, so twine only resolves for the platform that evaluated the module extension #3911

Description

@psalaberria002

tools/publish/BUILD.bazel's twine target explicitly transitions to python_version = "3.11" so it works independently of the root module's configured Python version — clearly intended to be usable from any bzlmod setup, on any platform, regardless of exec platform:

py_console_script_binary(
    name = "twine",
    pkg = "@rules_python_publish_deps//twine",
    python_version = "3.11",
    script = "twine",
)

The backing hub is declared in rules_python's own MODULE.bazel:

pip.parse(
    hub_name = "rules_python_publish_deps",
    python_version = "3.11",
    requirements_by_platform = {
        "//tools/publish:requirements_darwin.txt": "osx_*",
        "//tools/publish:requirements_linux.txt": "linux_*",
        "//tools/publish:requirements_windows.txt": "windows_*",
    },
)

target_platforms is never set here. Per the config-resolution error text you get when a wheel is missing ("Note that requirements_by_platform only affects the Bazel host platform unless target_platforms is also set."), this means requirements_by_platform's per-OS requirement files only ever produce a wheel for whichever platform happened to evaluate the module extension — not the linux/darwin/windows split the three separate requirement files clearly intend to support.

Repro

On any setup where the exec platform used for the twine binary differs from the machine that resolved the module extension — e.g. a CI/RBE config that prefers a Linux exec platform even from macOS clients via --extra_execution_platforms ordering — analyzing any py_wheel's generated .publish target fails with:

ERROR: .../rules_python_publish_deps/twine/BUILD.bazel:6:12: configurable attribute "actual" ... doesn't match this configuration: No matching wheel for current configuration's Python version and platform.

The current build configuration's Python version doesn't match any of the Python
wheels available for this distribution. This distribution supports the following Python
configuration settings:
    //_config:is_cp311_osx_aarch64

This reproduces on a completely clean checkout with zero downstream changes — it's purely a property of how the rules_python_publish_deps hub is generated, since requirements_linux.txt exists and is perfectly capable of producing a cp311_linux_x86_64 wheel; it's just never generated because target_platforms was never passed to pip.parse.

Possible fix

Add target_platforms to that pip.parse call, e.g.:

pip.parse(
    hub_name = "rules_python_publish_deps",
    python_version = "3.11",
    target_platforms = [
        "linux_x86_64",
        "linux_aarch64",
        "osx_aarch64",
        # ...
    ],
    requirements_by_platform = {
        "//tools/publish:requirements_darwin.txt": "osx_*",
        "//tools/publish:requirements_linux.txt": "linux_*",
        "//tools/publish:requirements_windows.txt": "windows_*",
    },
)

I patched this locally (adding target_platforms with linux_x86_64, linux_aarch64, osx_aarch64) and confirmed it resolves the issue: every py_wheel's .publish target across our monorepo (14k+ targets) now analyzes successfully under our macOS remote-execution config, where it previously failed on master with zero other changes.

Separately, #3909 looks related (native_binary resolving src via the target platform instead of exec) but is not sufficient to fix this on its own, since the underlying wheel hub never has an entry for anything but the platform that ran the module extension in the first place.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions