build(python): publish manylinux_2_28 wheels for broader glibc compat…

[phamgialinhlx]

Summary

Build released Linux wheels inside a PyPA manylinux_2_28 container so the binary runs on any Linux distribution shipping glibc >= 2.28 (RHEL 8, Debian 10+, Ubuntu 18.04+). The previous native build on the noble (glibc 2.39) CI image produced manylinux_2_39_x86_64 wheels that uv and pip refuse to install on common LTS hosts such as Ubuntu 22.04 / Debian 11.

Related Issue

Reported via the install path. On Ubuntu 22.04 (glibc 2.35):

Distribution openshell==0.0.36 @ registry+https://pypi.org/simple can't be installed because it doesn't have a source distribution or wheel for the current platform … only has wheels for: manylinux_2_39_aarch64, manylinux_2_39_x86_64, macosx_13_0_arm64

Changes

• New deploy/docker/Dockerfile.python-wheels-linux — builds the wheel inside quay.io/pypa/manylinux_2_28_{x86_64,aarch64}. Mirrors the layer layout (manifest copy → dummy-source dep build → real source rebuild) used by Dockerfile.python-wheels-macos so cargo cache mounts behave the same way. Maturin is invoked with --compatibility manylinux_2_28 so the resulting wheel is tagged accordingly.
• New mise tasks in tasks/python.toml:
  • build:python:wheel:linux:docker — driver task; takes TARGETARCH and WHEEL_OUTPUT_DIR.
  • build:python:wheel:linux:{amd64,arm64}:docker — per-arch wrappers.
  • The release-pipeline aliases python:build:linux:{amd64,arm64} are re-pointed at the Docker path.
  • The legacy native build:python:wheel:linux:{amd64,arm64} tasks remain for fast local iteration (still produce a wheel tagged for the host's glibc, which is fine for dev installs).
• Workflow .github/workflows/release-tag.yml and release-dev.yml, in the build-python-wheels-linux job:
  • Mount /var/run/docker.sock so the in-container mise run step can reach the host docker daemon to launch the manylinux build.
  • Add a setup-buildx step so the Dockerfile's BuildKit cache mounts (--mount=type=cache,target=/root/.cargo/registry etc.) work.
  • Both lines mirror the pre-existing build-python-wheel-macos job, which already uses this exact pattern for its osxcross Docker build.
• Docs architecture/build-containers.md updated to describe the new portable wheel build path and call out that the native task is now a local-iteration convenience, not the release path.

The pre-existing dead deploy/docker/Dockerfile.python-wheels (no references in any task or workflow) is left untouched to keep this PR additive — happy to remove it in a follow-up if maintainers want.

Alternatives considered

• maturin build --zig --compatibility manylinux_2_28 (cargo-zigbuild) was the smaller-diff option but carries real risk: bundled-z3 builds Z3 from C++ source via cmake, and Zig as the C++ toolchain on a heavy C++ codebase is unverified for this repo. Switching to the manylinux container keeps Z3 building under gcc-toolset-14, which is what the Z3 project itself tests against.
• Changing the base of Dockerfile.ci to an older-glibc distro would let the native build produce portable wheels, but it would simultaneously affect every other job that uses the CI image. Out of scope.

Testing

End-to-end validation on Ubuntu 22.04 / glibc 2.35 / Python 3.14:

Step	Result
docker buildx build -f deploy/docker/Dockerfile.python-wheels-linux --target wheels --build-arg TARGETARCH=amd64 …	exit 0, ~9 min on a warm cache
Wheel produced	openshell-0.0.0-py3-none-manylinux_2_28_x86_64.whl (18.8 MB)
uv add <wheel> into a Python 3.14 project that previously failed with the platform-tag error above	accepted
import openshell; openshell.SandboxClient	works

The build was iterated once during testing: the first attempt failed at dnf install perl-FindBin because that's not a standalone package on AlmaLinux 8 (manylinux_2_28's base). Replaced with perl-core, which provides FindBin and is the canonical "full perl" meta-package on the EL family. The Z3 bundled-z3 build (the highest-risk piece pre-test) worked cleanly under gcc-toolset-14 inside the manylinux container.

A workflow note: the docker socket mount means the linux wheel job, like the macOS wheel job, requires /var/run/docker.sock to be present on the self-hosted build-amd64 / build-arm64 runners. That matches the
contract the macOS job already relies on.

• mise run pre-commit passes
• Unit tests added/updated — N/A (build-system change, no library code)
• E2E tests added/updated — N/A; the existing release pipelines exercise
  the new task end-to-end on the next dev/tag run

Checklist

• Follows Conventional Commits — build(python): publish manylinux_2_28 wheels for broader glibc compatibility
• Commits are signed off (DCO)
• Architecture docs updated (architecture/build-containers.md)

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

All contributors have signed the DCO ✍️ ✅
_{Posted by the DCO Assistant Lite bot.}

[phamgialinhlx]

I have read the DCO document and I hereby sign the DCO.

[phamgialinhlx]

recheck

[pimlock]

Hi @phamgialinhlx, thank you for your contribution.

The problem this PR points at is real: the Linux wheel should not inherit an accidental glibc floor from whatever CI image happens to build it.

I think the wheel-specific direction should be:

1. choose one explicit lower manylinux floor, for example manylinux_2_28 or manylinux_2_31,
2. build the Linux wheel in a controlled manylinux environment, ideally via PyO3/maturin-action,
3. publish a single Linux wheel per architecture rather than multiple glibc-tiered wheels.

A lower-floor manylinux wheel remains installable on newer glibc systems, so I do not think we need both a manylinux_2_39 wheel and a manylinux_2_28 / manylinux_2_31 wheel.

This is also how projects like uv and ruff approach this. Their release workflows use PyO3/maturin-action with explicit manylinux settings for Linux wheels, and they publish separate musllinux wheels when they want to support musl/Alpine environments:

• ruff: https://github.com/astral-sh/ruff/blob/main/.github/workflows/build-binaries.yml
• uv: https://github.com/astral-sh/uv/blob/main/.github/workflows/build-release-binaries.yml

I do not think the standalone musl CLI artifact should be used for the manylinux wheel. A musllinux wheel is a different distribution target. It might technically be possible to package a static musl binary inside a manylinux wheel, but that would make the wheel tag misleading and would be confusing for users and maintainers.

This also connects to #1456, where we are discussing the broader glibc floor for host-executed Linux binaries. For wheels, though, I think the concrete path is simpler: pick the target manylinux floor, build with maturin in that controlled environment, and publish one broadly compatible wheel per architecture.

With that framing, this PR is directionally useful, but I think it should be reworked around the current CI/build structure and an explicit manylinux target before merging.

[pimlock]

Hi @phamgialinhlx. Thank you again for your contribution.

I have some updates on the Python SDK (and SDKs in general) and the outcome of these is that we will no longer bundle the CLI in our SDK, instead the SDK will only provide a library to interact with OpenShell and the CLI will remain a separate binary. This will simplify the build process of the SDK.

Here's the work in progress on that:

• RFC: rfc-0008: shared SDK rust core and language specific bindings #1764
• Initial implementation: feat(sdk): add openshell-sdk crate #1862

For extra context - we used the Python wheel as a distribution mechanism for OpenShell early on to simplify the installation for early users. We now have a dedicated install script and the CLI and gateway are separate binary.

The recommended way to install the OpenShell is through the install script. The Python SDK in this current form will be deprecated when we release the new version (based on the RFC linked above).

With all this, I will close this PR.