Repository navigation
Conversation
1 task
GitPython's Python implementations of repository storage couple its behavior to on-disk formats and object-ID widths. Delegate repository discovery, configuration, references, reflogs, object storage, tree construction, index operations, and revision parsing to `git.cmd.Git` so the default backend works with SHA-1/SHA-256 objects and files/reftable references. Require Git 2.52 or newer, retain the deprecated `GitDB` as an explicit choice, and preserve raw `repo.git` access. Use existing unsafe option/protocol primitives together with operand validation, NUL-framed records, and protected option ordering. Suppress implicit hooks, filesystem monitors, maintenance, lazy fetching, external diffs, and default text conversion in managed plumbing. Explicit commit hooks remain supported; signing and custom archive commands require opt-in. Preserve the established clean/smudge-filter behavior of existing worktree operations. Keep common workflows and map recognizable native failures to established exceptions. Remove or restrict low-level binary index/tree, raw reflog, precompressed object, and direct storage mutation APIs that cannot be exposed faithfully through Git. Preserve semantic index edits through private native indexes, discover worktree storage through Git, and reconnect retained submodule metadata without assuming its object or reference backend. Document the API and CLI changes in `doc/source/changes.rst`, add format and injection coverage, update minimum-Git CI and fuzz tooling, and bound the existing throughput benchmarks for subprocess-based operations. This work is planned for GitPython 3.4 some weeks before Git 3's official release, allowing users to test this branch and report compatibility feedback beforehand. Validation: the full pytest run produced 1,328 passes, 79 skips, one expected failure, and three submodule failures that were resolved and passed reruns. A fresh submodule/offline run had 224 passes; its two metadata-alias failures were fixed and retested, followed by eight passing retained-metadata checks. The Git 2.52 matrix passed 127 tests, plus four later quoted-branch checks. Ruff, mypy, basedpyright, Sphinx with warnings as errors, Python 3.8 package smoke tests, and deterministic fuzz-harness/version-guard checks pass. The updated fuzzing Docker image was not built and no long fuzz campaign was run. After rebasing onto the 3.2.1 security fixes, Alpine and Ubuntu CI failed `TestSubmodule.test_update_rejects_parent_component_in_name`: a NUL-bearing name reached `Git._check_operand()` first and raised `UnsafeOptionError` instead of the existing `ValueError` contract. Run the established submodule name and Windows filename validation before the CLI operand guard, retaining both exception compatibility and protection against command injection. The 22 focused name/Windows-path regressions, Ruff lint and formatting, and `git diff --check` pass locally.
The Cygwin performance job failed all six setup phases because the Cygwin package currently supplies Git 2.51.0, below the new Git 2.52 minimum. The same mismatch prevents the regular Cygwin suite from opening repositories. Build upstream Git `v2.52.0` with Cygwin tools and install it in `/usr/local` before preparing the fixtures. Include the HTTPS development dependencies so the existing clone and remote tests retain network transport support. Verify that the selected `git` is the built version before continuing. Validation: the workflow parses as YAML, the added shell block passes `bash -n` and ShellCheck, and `git diff --check` passes. The native Cygwin build and test suites require the Windows CI runner.
The macOS Python 3.8 and 3.14 CI jobs each passed 1,348 tests but failed `test_refresh_with_good_relative_git_path_arg`. Installing the supported Homebrew Git exposes `/opt/homebrew/opt/git/bin` on `PATH`; changing into that directory resolves it to the versioned Cellar directory. Compute the expected relative executable path from the current directory after changing into it. This preserves the documented `Git.refresh()` behavior and executable symlinks while removing the test's assumption that `shutil.which()` and `os.getcwd()` retain the same directory spelling. Validation: reproduced the failure using a symlinked directory on `PATH`. All 36 refresh tests pass with both ordinary and symlinked `PATH` values. Ruff lint, Ruff format, and `git diff --check` pass.
The Windows Python 3.8 CI job reported 42 failures and 179 setup errors. Most submodule failures shared a discovery bug: Git resolves relative gitfile targets using forward slashes even when the caller supplies a native Windows path. Normalize Git-facing discovery operands with the existing platform helper, and account for native separators in Git's worktree registry output. Use matching `surrogateescape` codecs for Git protocol paths so undecodable tree names round-trip without Windows filesystem encoding changing their bytes. Verify path/stage, mode, and object ID after materializing a private index: `git update-index --index-info` can exit successfully while dropping Windows-incompatible names. Raise `ValueError` before publishing such an index and document the platform restriction in `changes.rst`. Keep unusual names in object-only tests when the host cannot represent them in a checkout. Use native-valid paths for worktree tests, assert rejection of unsupported index names and quoted file references, and keep full quoted reference coverage with reftable. Fix separator and LF assumptions in config, URL, and packed-reference fixtures. Close test-owned repositories before submodule removal and make the fake Windows Git executable discoverable without shell execution. Also restore root paths for `Repo.tree()` results resolved directly from tree IDs while preserving explicit subtree paths. Validation: 116 repository tests and 10 focused submodule tests passed; index/helper tests passed 93 with 2 platform skips; the focused format/safety run passed 72; config/reference/remote tests passed 57 with 24 subtests; 36 refresh tests and 2 revision regressions passed. The Git 2.52 targeted index matrix passed 19 tests. Ruff, mypy, basedpyright, Sphinx with warnings as errors, and `git diff --check` pass. Native Windows validation awaits CI. The rebased Windows Python 3.14 job then failed only `test_quoted_remote_and_submodule_names`: registering an existing checkout still validates the submodule name as a possible metadata path, so `quoted"module` correctly raises `ValueError`. Create that fixture using a filesystem-safe name, then rename its `.gitmodules` section through the config writer. This exercises quoted config parsing without requesting a Windows-invalid metadata name or weakening filename validation. The focused remote and Windows destination-name tests passed 24 cases on macOS; native Windows confirmation awaits CI.
The partial Cygwin fast-suite log exposed a failure in `test_valid_unusual_index_names_round_trip`: native Git omitted a literal backslash filename. Cygwin Git recognizes Windows separators and applies NTFS path protection, even though Python reports a POSIX platform. Move that case into the existing unsupported-name assertions for Cygwin. Check that GitPython raises `ValueError`, preserves the published index, and removes its lock. Other POSIX systems retain the round-trip case; Windows retains the control-character and colon rejection cases. Document the Cygwin restriction in `changes.rst`. Validation: three focused index tests pass locally. The Cygwin and Windows test branches pass with only Git's ignored-record behavior simulated; native Cygwin verification awaits CI. Ruff lint, formatting, and `git diff --check` pass.
The Windows Python 3.8 partial CI log showed two failures in `test_submodule_allows_existing_metadata_symlinks`: preparing update and move aliases raised `PermissionError` before invoking GitPython. Git marks the submodule's `.git` file hidden; Python's `write_text()` attempts to recreate it, which Windows rejects for an existing hidden file. Open that fixture file with `r+`, write the replacement target, and truncate it. This preserves the hidden attribute while replacing the full contents. The separate fixture that creates a previously absent gitfile is unchanged. Validation: all six native/windows37 update, move, and remove alias modes pass locally. Ruff lint, formatting, and `git diff --check` pass. The Windows-specific file-attribute behavior will be verified by CI.
The Windows Python 3.15 CI job failed to set up twelve missing-submodule cases because `shutil.rmtree()` cannot remove read-only loose Git objects. The fixture deliberately removes retained metadata to model an absent submodule, so use the existing `git.util.rmtree()` helper, which clears read-only attributes when retrying Windows deletions. All twelve affected cases pass locally, along with Ruff lint and format checks. Production behavior and test coverage are unchanged.
The Windows Python 3.15 CI job failed to remove submodule checkouts because persistent `cat-file` processes still used them as working directories. `Submodule.update()` relied on collection of its temporary `Repo`, but captured log records retained a `Head` argument and therefore the repository and its process. Recursive updates also opened an extra unbounded repository. Close the owned repository after updates and on errors, and reuse it for recursion with final cleanup. This preserves `keep_going` behavior while releasing processes even when logs or callbacks retain repository objects. The compatibility test now scopes its own repository and closes it before removal; allocation tracing identified those separate caller-owned handles. Four regressions retain real logging arguments and verify process cleanup for normal, failing, recursive, and recursive `keep_going` updates. Both previously failing tests pass with tracing asserting no live checkout processes at each removal. Ruff, mypy, basedpyright, and `git diff --check` pass locally. Native Windows validation will run in CI.
The Cygwin full suite reached 1,372 passing tests but failed its native-Git detection check. The new source build installed Git into `/usr/local`, while GitPython's existing detector expects `uname` beside the selected Git executable. Cygwin installs `uname` in its normal `/usr/bin` directory. Install Git 2.52 under `/usr`, replacing the older packaged Git and preserving that standard layout. Verify `Git.is_cygwin()` immediately after installing Python dependencies so a setup regression fails before the long test suite. The detector and its missing-`uname` behavior remain unchanged. YAML parsing, extracted Bash syntax, ShellCheck, and `git diff --check` pass locally. The preceding Cygwin performance suite also passed all six tests; native validation of the corrected installation runs in CI.
…ckout Exercise a current high-download GitPython consumer with its unchanged upstream tests. Add a shared `uv` runner that resolves the latest PyPI release, retrieves verified source, installs a private environment, and replaces the released GitPython dependency with this editable checkout. Verify the imported `git` module before testing and retain source provenance, frozen requirements, and JUnit results for diagnosis and reproduction. Clear inherited Git repository/configuration settings and Python import paths so upstream commits and resets use their own fixtures. Use pytest's long `--override-ini` spelling because Bandit's CLI tests interpret `-o` as a forbidden Bandit output option. Reject successful runs with no passing tests, including entirely skipped suites. Add a CI job for the latest Bandit release, with Git 2.52 or newer, and local usage and download-ranking documentation. Bandit 1.9.4 passes all 12 selected tests against this checkout on Python 3.12 and Git 2.54, even with deliberately invalid inherited Git directory, index, and config settings. Ruff, workflow YAML parsing, and whitespace checks also pass. No GitPython compatibility changes were needed.
Add MLflow's released Git project and context tests to the shared local
runner and CI matrix. Rank the project by `mlflow-skinny` downloads without
summing overlapping distributions, resolve its current PyPI release, and
check out the matching `v{version}` source tag for unchanged upstream tests.
Install the matching full `mlflow` package because upstream global fixtures
need its server and SQLite support. Clear `CI` and `GITHUB_ACTIONS` inside
the isolated run to avoid unrelated upstream wheel builds and conda
cleanup. Disable telemetry and keep the venv first on `PATH` so project
subprocesses use the same editable GitPython checkout.
The shared runner passes all 47 selected tests for MLflow 3.16.1 on Python
3.12 and Git 2.54, including 31 repository/project/model-versioning cases and 16 Git
context or credential-redaction contract cases. Validation started with
both CI variables set, exercising the CI isolation. Public example clones
and a localhost HTTP server are required; no cloud services or models are
needed. The model-versioning cases also cover staged/unstaged diffs and
dirty-state handling. No GitPython compatibility changes were necessary.
Include `langchain-community` as a current runtime GitPython user: its
published `GitLoader` requests a manual GitPython installation even though
it is absent from `Requires-Dist`. Its September download count ranks
above the other selected consumers.
Resolve the latest PyPI release and run unchanged tests from the matching
`libs/community/v{version}` tag against the editable GitPython checkout.
Use upstream test-plugin ranges and `--only-extended` so missing integration
dependencies fail collection instead of silently skipping the tests.
Add the same profile to CI and document the runtime-use selection.
The shared `uv` runner passes both GitLoader tests for release 0.4.2 on
Python 3.12 and Git 2.54. They exercise real local clones, commits,
checkout, tree traversal, ignored files, repeated loads, and remote URL
validation. No network services are used during testing and no GitPython
compatibility changes were required.
SWE-bench is a current high-download GitPython user, but its latest 5.0.2 release has no tests for the GitPython inference helpers and no matching Git release tag. Retrieve the verified PyPI source and explicitly document that this profile runs a GitPython-authored supplemental integration test, rather than misrepresent unrelated upstream tests as compatibility coverage. Import the real `AutoContextManager` with only its required `chardet` and GitPython dependencies. Exercise clone, commit checkout, reset, untracked cleanup, directory restoration, and clone reuse for SHA-1/SHA-256 with files/reftable. Route its normal URL to a local fixture and permit only file transport during the test. Do not patch production modules or mock GitPython. BM25's Java/Pyserini helpers remain outside this focused check. Add the profile to the local runner and CI. Expand pytest-option paths, cut off unrelated ancestor conftests, and use importlib mode so the supplemental filename cannot shadow the installed upstream package. All four cases pass through the shared runner on Python 3.12 and Git 2.54; they also passed separately on minimum Git 2.52. Ruff and whitespace checks pass. No GitPython compatibility changes were required.
Complete the five-current-user compatibility matrix with `acryl-datahub`. Resolve the latest PyPI release and retrieve its tag from `acryldata/datahub`, which publishes patch tags missing from the repository named by package metadata. Run the unchanged Git integration file against the installed release and this editable GitPython checkout. Exclude unrelated SQL/docker conftests, disable telemetry, and clear the private SSH test credential variable. The selected tests still exercise a real public GitLab clone and fixed-commit checkout, a localhost SSH timeout, GitCommandError handling, password redaction, and source configuration. The upstream private-clone test retains its credential-dependent skip. Document all five current users and their September 2026 download ranking, including optional runtime integrations, distribution deduplication, and projects whose latest releases dropped GitPython. Add an all-project local command and the fifth CI matrix entry. The shared runner passes 7 tests with 1 upstream skip for DataHub 1.7.0.14 on Python 3.12 and Git 2.54. All five profiles have now passed locally, with 68 upstream passes plus 4 SWE-bench supplemental cases. Ruff, Python syntax, workflow YAML/matrix consistency, Bash syntax, ShellCheck, and whitespace checks pass. No GitPython compatibility fixes were required. Also clear inherited pytest options and plugins: a caller's `-k` filter could otherwise leave only unrelated contract tests and yield a misleading pass. All four SWE-bench cases still pass with a deliberately nonmatching inherited filter and a nonexistent plugin, verifying their removal.
Add `GitPython[gix]` with the published `GixPython==0.1.0` dependency and select it when the `gix` module is installed. Start with an empty native dispatch table so library-managed commands retain their CLI behavior. Keep the existing safety boundary and distinguish unsupported calls from errors after a native write. Add per-operation reporting and a test runner that creates its historical fixture in a disposable local clone. The installation test verifies backend selection, and the tutorial fixture no longer clones GitHub. Let the optional tox environment resolve its dependencies from the package index rather than requiring an unpublished local wheel. Tests themselves retain offline package installation using cached wheels. Pin the official initial release so the optional backend has a reproducible API baseline. Restrict that dependency to Python 3.11 or newer, matching its published interpreter requirement while allowing universal resolution of GitPython extras on the existing Python 3.8+ support range. The published Apple Silicon wheel was downloaded from PyPI and verified against its SHA-256 digest; it replaces the previous local artifact with the same version number. Validation on CPython 3.12/macOS: the SHA-1/SHA-256 backend smoke checks and fresh extra-installation check passed (3 tests). Ruff lint and formatting pass. Universal `uv sync --all-extras --all-groups --dry-run` initially rejected the unmarked dependency for Python 3.8–3.10; the Python-version marker fixes that resolution failure when dynamic package metadata is refreshed.
Use GixPython's header lookup for `Git.get_object_header` and ODB metadata. Preserve the existing object-ID, kind and size tuple, missing-object behavior, and CLI diagnostics for unsupported storage or revision syntax. Keep the persistent-process lifecycle regressions explicitly exercising the CLI fallback. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 10 passed in 4.34s.
Read object contents through GixPython and return independent byte streams from `Git.stream_object_data`. Partially reading one object must not corrupt another read. Retain CLI streaming above 8 MiB because the current native lookup buffers an entire object (GIX-2). The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 9 passed in 2.40s.
Handle verified revisions and bound repository metadata with GixPython. Keep Git's initial repository validation and unsupported discovery options. Message searches, dirty suffixes and describe-shaped names fall back because the native grammar differs in regex handling and exact-tag precedence (GIX-14/17). The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 26 passed in 18.03s.
Implement the managed full-tree NUL listing with native tree entries. Preserve Git's octal modes, object kinds, IDs and raw filename bytes for the existing tree parser. Other listing options continue through Git. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 26 passed in 3.86s.
Use native reference lookup for quiet, nonrecursive symbolic-target reads. Preserve detached-reference exit status and let Git supply missing-reference diagnostics. Symbolic mutations retain their existing reflog and validation behavior through the CLI. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 13 passed, 24 subtests passed in 3.81s.
List native reference names in Git's order for the managed `for-each-ref` format and literal prefixes. Keep glob patterns, pseudo refs and other formats on the CLI rather than interpreting them as plain prefixes. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 6 passed in 1.88s.
Serve simple managed `config --get` requests from the native configuration snapshot, including implicit booleans and missing-key status. Open configuration queries without synthetic safety settings so they report the user's actual values. Files, streams, enumeration and mutations stay on Git (GIX-12). The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 23 passed in 1.11s.
Build the managed porcelain worktree inventory from native main/linked repositories, including branch, detached, unborn and lock metadata. Keep prunable entries and linked worktrees of bare repositories on Git because their classification/diagnostics differ in the current engine (GIX-14). The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 5 passed in 1.24s.
Use the native graph operations for two-revision merge bases, all bases and ancestry checks. Preserve the existing result format and no-base/non-ancestor status. Octopus, fork-point and other options retain the CLI implementation. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 6 passed in 1.10s.
Add a closable native history helper and use it for `Commit.count`, including skip, limit and first-parent selection. Counting does not depend on the known general walk-order difference. Keep path filters and other revision options on Git; iteration errors preserve the public Git command error type. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 5 passed in 1.01s.
Connect `Commit.iter_items` to the native history helper for first-parent walks and a single unskipped tip. Preserve Git's default traversal for general history because native ordering diverges on the repository fixture (GIX-8). Existing revision and option checks still run before dispatch. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 6 passed in 1.05s.
Serialize native index entries into the existing staged NUL-listing format, preserving stage, mode, object ID, assume-valid and skip-worktree flags. Custom, relative and sparse index paths fall back because the current binding cannot load or expand them with the required semantics (GIX-3/4). The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 5 passed in 1.10s.
Use the native index version for `IndexFile.version`'s managed query. Reuse the index capability guards; actual `update-index` mutations continue to use Git. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 16 passed in 4.01s.
Implement managed object hashing/writing for blobs, trees, commits and tags. Validate the native serialization in an in-memory ODB before persisting bytes, and preserve the input position on fallback. Limit native input to 8 MiB. Once a native write starts, convert its error without retrying via Git. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 8 passed in 1.20s.
Implement managed `commit-tree` with explicit author/committer signatures and the native commit writer. Verify tree and parent kinds before mutation and preserve message bytes. Other encodings, duplicate parents, signing and unsupported options keep Git's behavior. Add combined object/tree/index/commit checks that reject subprocesses and compare native object IDs with Git. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 55 passed in 9.23s.
Implement reflog existence and GitPython's exact NUL-delimited read format with native log iteration. Preserve newest-first ordering, identity and raw timestamps while filtering unavailable/non-commit targets as Git does. Orphan log lookup, other formats and log writing remain with Git. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 42 passed in 9.86s.
Use native identity resolution for `GIT_COMMITTER_IDENT`, including per-command name, email and raw date overrides. Preserve Git formatting and fall back for unsupported normalization or incomplete identity. Add native/CLI byte comparisons for the supported managed query commands. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 20 passed in 3.64s.
Use native ref edits for compatible unconstrained target changes and non-dereferencing deletions. Keep strict create/CAS, active-branch HEAD reflogs, symbolic detachment and unchanged-target cases on Git because the current native semantics differ (GIX-9/10). Normalize log-message whitespace before mutation. Keep the CLI operand-boundary regression and add native history/ref/log parity checks. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 64 passed, 24 subtests passed in 11.54s.
Construct the existing DiffIndex from native tree changes, including reversed comparisons, type changes and unambiguous exact renames. Preserve Git's result order. Keep patches, paths, index/worktree comparisons and inexact/ambiguous rename pairing on Git until their contracts match (GIX-11). The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 52 passed in 9.45s.
Count changed lines with native Myers blob diffs against the first-parent or empty tree, and retain the existing Stats representation. Cover root, binary, rename, type-change and empty-commit parity. Keep diff attributes, other algorithms, gitlinks, quoted paths and large blobs on Git to preserve its attribute-source and formatting semantics (GIX-2/18). The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 29 passed in 7.68s.
Use GixPython's directory walk for `Repo.untracked_files`, preserving raw names, nested-repository directory markers and sorting. Keep extra status options, unsupported index storage and non-root command working directories on the existing CLI path. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 28 passed in 7.17s.
Use native exclude matching for `Repo.ignored`, retaining tracked-file suppression, directory modes and the caller's path spelling. Preserve CLI diagnostics for unsupported normalization and paths traversing symlinks or tracked gitlinks. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 28 passed in 7.03s.
Use native tree/index/worktree status for `Repo.is_dirty`, preserving its flags, pathspecs, untracked selection and configured submodule checks. Do not write native status refreshes back to the index. Add CLI parity and no-subprocess checks for status, ignores, untracked files and dirty submodules. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 34 passed in 18.88s.
Include `gix-requirements.txt` explicitly alongside the other requirement files so source distributions can resolve the optional backend with every supported setuptools version. Keep tox's wheel builder in `.pkg-gix`, whose package installation is restricted to the local wheel directory. Built a source archive locally, verified that it contains the new requirements file, and rebuilt its wheel without accessing a package index. The rebuilt wheel declares `Provides-Extra: gix` and the conditional `GixPython` dependency. Tox itself was not available locally; the interpreter-based test runner and its offline installation test were exercised instead.
`for-each-ref` omits symbolic references whose targets do not exist, while GixPython's reference iterator yields them. This made `Repo.heads` include branches that the CLI backend would omit. Resolve symbolic targets while retaining their original names. A dangling target uses the existing read-error fallback, allowing Git to handle its filtering and diagnostics. Valid symbolic aliases still use native enumeration. The regression failed for both SHA-1 and SHA-256 before the fix. All six selected enumeration, query, and reference tests now pass, as do Ruff lint and format checks.
GixPython opens repositories with `extensions.compatObjectFormat`, but native translation and object-write compatibility have not been validated. The local Git build reports that compatibility hash support requires Rust, so its failed translation probe does not establish a defect in GixPython's object mappings. Reject this capability in the shared native repository adapter before reads or mutations. Git retains responsibility for repositories using the extension, and the coverage report identifies the conservative guard as `GIX-19`. The new SHA-1 and SHA-256 regression cases failed before this guard. They verify CLI dispatch for reads and writes without depending on Git's optional compatibility support, and forbid any native write attempt. All 36 focused backend tests and Ruff lint/format checks pass.
The optional backend needs reproducible installation instructions and an explicit record of supported operations and upstream limitations. Add `doc/gix-backend.md` and link it from the README. Explain automatic extra selection, installation of the official `GixPython==0.1.0` PyPI release with an existing interpreter, offline installation tests using cached wheels, separate CLI/native environments, operation reporting, and remaining CLI cases. Record 19 GixPython/Gitoxide follow-ups with reproduction details, distinguishing confirmed mismatches from conservative capability guards. Preserve the original local-artifact results as historical evidence and identify the published wheel separately. Its SHA-256 matches PyPI's digest, and the backend-selection smoke and fresh installation checks passed (3 tests). The full native suite results recorded for the earlier local artifact do not claim validation of the published wheel. Clarify the backend docstring: unsupported capabilities fall back before mutation; native mutation failures must never trigger a second CLI mutation. Validation: the dependency change and all 31 descendants passed fast QA on existing CPython 3.12.14/macOS: Ruff lint/format, pre-commit, mypy, basedpyright, universal dependency resolution, and fatal lint checks for bundled dependencies. The user deferred builds, documentation builds, full suites, and downstream integration jobs after the initial thorough run proved slow. No additional Python interpreters were installed. Record future native-state and fixture-reuse optimizations without changing runtime behavior or tests. Native repository opens currently happen per operation; repeated-read measurements show the value of retaining a handle, while a focused sample attributes most elapsed time to Git subprocesses. Preserve invalidation, safety and test-isolation requirements in the ledger. A source audit of 1,729 parameterized cases identifies 256 repository-sharing candidates after preparation and 166 cases whose repository setup is unnecessary. Record the fixture groups, repeated historical dependency setup, and the limits of this conservative inventory. Subsequent validation of the official wheel on existing CPython 3.12.14/macOS: 1,649 tests and 38 subtests passed with coverage, 79 skipped, one expected failure; total wall-clock time was 757.85 seconds. Update the published-release record so it no longer says the full native run remains outstanding.
`TestActor` and `TExc` inherited `TestBase`, which opens a repository and reconstructs two historical dependency sources for assertions that only exercise actor parsing and exception formatting. Use the existing `TestCase` base instead. The 166 cases retain their assertions while avoiding repository creation, checkout and garbage collection during class setup. Validation on existing CPython 3.12.14/macOS: all 166 affected tests passed with official GixPython 0.1.0 first, then with the CLI installation. Ruff passes. The GixPython test bodies completed in 0.06 seconds.
`TestBase` reconstructed both merged dependency repositories for every class, even when its tests never cloned either source. Move reconstruction to a lazy session fixture and retain the existing URL helpers for consumers. The prepared sources are shared; consuming tests still clone their own writable repositories. Pytest owns source cleanup after all classes finish. Affected clone, tutorial, repository, and submodule consumers passed with GixPython first: 206 passed, 6 skipped, 1 expected failure and 14 subtests in 90.89 seconds wall time. CLI then produced the same results in 160.82 seconds. Ruff lint and formatting checks passed.
`movable_submodule` repeatedly initialized and committed the same source and parent before each rejection or mutation test. Prepare each logical name once per module and copy the parent with `shutil.copytree` instead. Relative gitfiles and `core.worktree` settings remain usable after copying; only the immutable source URL is shared. Every case gets fresh wrappers and independent refs, index, config, objects, and worktree files. Add an isolation check for distinct writable files and baseline preservation after edits and branch creation. Existing security snapshots remain intact. All 333 top-level submodule cases passed with GixPython first in 88.63 seconds wall time, then CLI in 178.75 seconds. Ruff checks passed.
Repeated rejection cases recreated nested metadata, separate Git directories, symlink layouts, and retained modules before every operation. Prepare each layout once and restore a complete filesystem copy at its original path. Keeping that path preserves absolute gitfiles, linked-worktree registration, and symlink targets without patching Git metadata. Remove each active copy in fixture cleanup, including after a failed assertion. Move preparation out of the six test bodies while preserving their security snapshots and destination assertions. A deliberate-mutation check verifies that restoration removes extra files and restores content through an absolute symlink. The 51 variants passed on GixPython first in 12.34 seconds, then CLI in 26.23 seconds wall time; the restoration check passed on both. Ruff passed. The `test-cygwin` fast job also exposed `PermissionError` when Python 3.9 copied symlink metadata. Copy ordinary files with `shutil.copytree` and recreate directory links afterward, retaining link targets without metadata operations. The restoration regression simulates this denial. All 52 affected cases passed with GixPython first (13.98 s), then CLI (27.38 s); Ruff passed.
Revision-query tests rebuilt the same four commits, tags, branches, index, and reflogs for every case. Build that graph once per module and copy its complete repository for each test. Return fresh repository, branch, and commit wrappers so tests that add tags or commits remain independent. Add a check that private tag, index, worktree, and commit changes preserve the prepared graph. All 23 revision cases passed with GixPython first in 4.39 seconds wall time, then CLI in 7.57 seconds. Ruff checks passed.
Eight lookup tests cloned and checked out `0.3.2.1` only to read paths from its tree. Resolve that historical tree through the existing class repository instead. Each assertion still uses a fresh tree wrapper and checks the same blob or tree IDs, pathlike operands, and missing-path errors. All 22 tree tests passed with GixPython first in 1.66 seconds wall time, then CLI in 3.86 seconds. Ruff lint and formatting checks passed.
No-fetch tests repeatedly built the same two-commit source and initialized submodule. Prepare them once per module and copy both repositories per case, because tests also advance and modify the source. Relocate the private source URL in `.gitmodules`, parent config, and the module remote; record that URL in parent history so `RootModule` compares the correct previous source. Re-enumerate the submodule after config edits to obtain normalized caches. An isolation check advances the private source while verifying the prepared source is unchanged. All 69 no-fetch tests passed with GixPython first in 56.27 seconds wall time, then CLI in 117.52 seconds. Ruff checks passed.
Update the performance journal now that repeated test setup has been reduced in seven independent commits. Describe copied writable state, stable-path restoration for absolute Git links, lazy historical sources, and direct tree reads, retaining native repository reuse as future work. Record affected GixPython-first and CLI-second validation, isolation checks, and the complete coverage-enabled runs at `c8ee26c7`: GixPython passed in 404.61 seconds versus the previous 757.85 seconds (46.6% less time); CLI passed in 755.78 seconds. Distinguish this local before/after observation from a statistical benchmark and explain the differing backend test counts. Repository-wide Ruff lint/format, mypy, and pyright checks passed.
Full-suite timings mix repository operations with repeated fixture setup and coverage. Add a benchmark-only `pyperf` harness against a pinned existing GitPython checkout, with one warm `Repo` per worker and fresh high-level wrappers per invocation. Measure the complete read-only journey and eight named public-API operations, including a patch operation that uses fallback. Separately time direct `Repo` construction and discovery from `git/objects`, creating and closing a repository per invocation. Keep those lifecycle costs outside the already-open journey so future native-handle reuse has a visible effect on operation timings. `MEASUREMENTS` makes additions join individual timings, the journey and result parity checks. Retain environment/revision metadata, result digests and one invocation's native/fallback decisions in raw `pyperf` results. The comparison reports both backend means and standard deviations and rejects mismatched results, installations or workload metadata. Document reproducible local setup without downloading another interpreter or modifying the measured repository. All eleven workload paths passed GixPython first and CLI second on the fixed fixture, with matching result digests. Four comparison regression cases passed with both installations. Ruff lint/format and `git diff --check` passed.
Add a separate `Backend benchmark` job using one CPython 3.12 interpreter and two installations, with official `GixPython==0.1.0` only in the native one. Prepare a fixed SHA-1/files checkout of GitPython 3.1.45 with one branch, an untracked file and an ignored directory before timing. Keep local/global Git configuration and optional locks from changing the measured workload. Run the calibrated `pyperf` suite with GixPython first, then CLI on the same runner. Include separate direct repository opening and nested discovery while keeping the operation journey on a retained `Repo`. Validate result digests and workload metadata, publish all means and standard deviations plus `pyperf` significance reporting in the job summary, and retain raw JSON and comparison artifacts even on failure. Timings are observational on shared runners; execution and parity errors fail the job, without a noisy performance threshold. New `MEASUREMENTS` entries automatically participate in this job. The final fixed workload passed all eleven GixPython-first and CLI-second smoke measurements locally with matching result digests. Four comparison regression cases passed both installations. Workflow YAML parsing, Ruff lint/format and `git diff --check` passed. Hosted execution will follow the branch push.
Document the new `pyperf` workload and local measurements independently of full-suite setup/coverage timings. Record all eleven means and standard deviations, fixture/source revisions and environment details: direct opening and nested discovery favor GixPython, but the already-open journey is about 13% slower because history metadata and commit statistics offset other gains. Explain retained `Repo` ownership during operation measurements, separate open/discovery lifecycle costs, fresh high-level wrappers, parity checks and the extensible CI artifact workflow. Native handle reuse remains future work; these measurements expose its potential benefit without implementing it. Both complete calibrated runs finished on existing CPython 3.12.14 with GixPython first and CLI second. All eleven result digests match; the comparison and `pyperf` significance table completed successfully. Record the stability warnings rather than treating one machine's warm-cache values as universal. The initial dedicated CI benchmark job also passed. `git diff --check` passed.
Backend fallback decisions do not measure process launches: raw `repo.git` calls bypass dispatch reporting, persistent processes serve repeated requests, and one fallback can cause multiple commands. Record successful `Git.execute` launches through the existing thread-safe backend counters, including raw calls and commands that fail after spawning, without recording failed creation. Direct test subprocesses and child processes started by Git remain outside this process-local counter. Report pytest session totals by setup/call/teardown phases, preserve the JSON record-list format, and add optional session/test-call ceilings that fail on increases while permitting reductions. Explain that `unittest` setup and teardown run inside pytest's call phase and xdist is not aggregated. Record warm per-invocation launches in `pyperf` metadata and comparison tables. The fixed journey launches 23 processes with CLI versus one with GixPython; opening uses 11 versus four and nested discovery 13 versus six. Enforce checked-in per-measurement Gix ceilings in the existing benchmark CI comparison, including zero for converted operations and one for patch fallback. Lower ceilings as conversions land instead of silently increasing them. The full GixPython suite passed 1,674 tests and 38 subtests without coverage in 408.81 seconds, recording 30,898 session launches (6,523 setup, 24,375 call) versus 21,409 fallback decisions. Subsequent affected CLI command tests passed 119 cases with one skip. Thirteen counter/ceiling/comparison cases passed Gix first and CLI second; an end-to-end three-launch test correctly failed a two-launch ceiling. All eleven two-sample benchmark results matched and met the Gix budget. Ruff lint/format, mypy, basedpyright, YAML/JSON parsing and `git diff --check` passed. Only the existing CPython 3.12.14 was used.
The native adapter previously called `gix.open_opts()` for every operation, repeating configuration parsing and losing shared index/object-store state. Each `Repo` now owns a native handle, associated with `Git` through a weak reference. `close()` releases it and pickling excludes native resources. Reuse preserves storage/environment guards. Metadata and environment changes, and successful raw CLI launches, invalidate the configuration view and trigger `reload()` on the retained handle. Config queries keep a separate fresh view without synthetic safety settings. Includes reload conservatively because GixPython does not expose their source paths. A per-repository lock prevents concurrent refresh races; Gix handles read sharing and index/ODB refresh. Validated Gix first: 260 affected tests and 14 subtests passed, with 3 skips; 38 native regressions passed again after the metadata guard was completed. CLI compatibility: 222 affected tests and 14 subtests passed, with 3 skips. Ruff lint/format, mypy and basedpyright passed.
Opening previously launched Git for discovery, worktree resolution and per-instance version checks even when GixPython could provide all metadata. The constructor now opens each candidate with `gix.open_opts()` and derives storage, worktree, format and empty-tree metadata from the retained handle. Parent traversal remains controlled by `search_parent_directories` and a malformed `.git` entry never causes ascent into another repository. Native HEAD decoding preserves invalid-reference rejection. Ambiguous common metadata and unsupported storage/environment/formats keep Git diagnostics. Linked worktrees of bare main repositories are classified as worktrees both at construction and in managed bare queries. Symlink path spelling remains compatible. Native operations no longer probe Git's version; every actual managed CLI fallback still enforces Git 2.52 or newer. Regression checks forbid CLI execution while opening SHA-1/SHA-256 worktrees, bare repositories, nested discovery, linked worktree administrative paths, linked worktrees of bare repositories and explicit `GIT_DIR`. The benchmark opening/discovery CLI ceilings are reduced from four/six to zero. Command mock tests explicitly prepare the Git version cache they previously obtained as a side effect of opening. Validation: affected Gix tests passed before CLI compatibility tests; 304 Gix and 264 CLI tests passed, with 3 skips and 14 subtests each. Command guards passed 136 tests per backend; positional guards passed 31 per backend. Process-count and benchmark-budget checks passed 13 per backend. Ruff lint/format, mypy and basedpyright passed. Full-suite results and measured performance are recorded in the following performance journal update.
Record sequential `pyperf` results for the pinned repository after native handle reuse and opening/discovery changes at `c49bbec0`. Both installations used official GixPython 0.1.0 or CLI on the same existing CPython 3.12.14, with three workers and three values each. All eleven result digests and CLI launch ceilings passed, including zero for opening and discovery. Opening measured 69.28 ms CLI versus 1.00 ms Gix; nested discovery measured 81.68 ms versus 0.90 ms. The complete already-open journey remains slower with Gix (263.25 ms versus 209.44 ms), so document the lifecycle gains and remaining history/statistics costs separately, including sampling limits. The complete Gix suite passed 1,678 tests and 38 subtests in 337.51 seconds, with 79 skips and one expected failure. It launched 22,319 CLI processes, 27.8% below the prior local baseline; runtime was 17.4% lower. Preserve the phase counts, affected CLI validation and static-check results in the journal. Documentation diff whitespace checks passed; no implementation changed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tasks
Created by Codex on behalf of Byron. Byron will review before this is ready to merge.
Summary
Important
This PR will become GitPython 3.3 some weeks before Git 3 is officially released in March.
Users can already test and run their software against this branch. Please report compatibility problems and provide feedback here in this PR.
To try it with Git 2.52 or newer installed:
python -m pip install --upgrade "git+https://github.com/gitpython-developers/GitPython.git@gix-backend"GitPython now delegates repository discovery, configuration, references, reflogs, object storage, tree construction, index operations, and revision parsing to guarded Git commands. The default backend supports SHA-1 and SHA-256 objects with either files or reftable reference storage, without interpreting Git's binary storage formats in Python.
The deprecated
GitDBremains explicitly selectable, and rawrepo.gitcommand access remains available. The migration notes inchanges.rstdescribe the API changes and differences in Git CLI behavior.Changes
1. Git CLI implementation of the GitPython API surface
The ordinary installation implements the supported GitPython API through guarded
git.cmd.Gitcalls. Git owns repository storage and format handling, including SHA-1/SHA-256 objects and files/reftable references. This removes Python storage-format parsing from the default backend while retaining common documented workflows, rawrepo.gitaccess, and the explicitly selectable deprecatedGitDB. The existing context and migration notes below describe safety controls and low-level API changes.2. GixPython implementation for better performance
The optional
GitPython[gix]installation uses the publishedGixPython==0.1.0release on CPython 3.11+. GitPython selects it automatically when thegixmodule is importable; the ordinary installation continues to support Python 3.8+. To try the native backend:python -m pip install --upgrade "GitPython[gix] @ git+https://github.com/gitpython-developers/GitPython.git@gix-backend"Supported library-managed operations use Rust-backed object reads/writes, revision and graph queries, tree enumeration and construction, index reads/edits, references and reflogs, commit creation/statistics, tree diffs, status, untracked files, and ignore matching. Unsupported formats and options retain the guarded CLI implementation. In particular, reftable and
extensions.compatObjectFormatrepositories currently use Git. Native mutation failures are reported without attempting a second CLI mutation. Publicrepo.gitcommands retain their CLI behavior.Recorded local measurements with official GixPython 0.1.0 on CPython 3.12.14, macOS arm64:
These are individual local validation runs, not statistical benchmarks or a guarantee for every application. The full suites collect different backend-specific tests; the focused selections overlap. See the backend guide and performance journal for conversion coverage, remaining CLI cases, reproduction details, and the 19-item GixPython/Gitoxide follow-up ledger. Native repository-handle reuse remains future work and is not included in these gains.
3. Significantly faster CI test execution
Repeated repository construction now happens once where safe: historical dependency sources are prepared lazily, and submodule/revision fixtures copy prepared baselines into independent writable repositories. Actor/exception tests avoid unnecessary repository setup, and historical tree reads avoid cloning and checking out a worktree. Tests retain fresh wrappers, independent mutable Git metadata, existing security snapshots, and explicit isolation regressions.
On the same local coverage-enabled GixPython suite, fixture optimizations reduced wall time from 757.85 s (12m38s) to 404.61 s (6m45s): 353.24 s saved, 46.6% less time, about 1.87× faster. CLI fallback decisions decreased from 39,002 to 21,402; these counters do not count every Git subprocess. This is the before/after fixture improvement, separate from the backend comparison above. The changes reduce repeated work in the CI suites; the quoted timings are local measurements, not a hosted GitHub Actions before/after benchmark. No matching pre-optimization full CLI run was recorded.
The branch also adds released downstream compatibility checks for LangChain, MLflow, Bandit, SWE-bench, and DataHub, plus reproducible local test execution and backend-operation reports.
Context
The goal is to support Git's object and reference formats while reducing the attack surface of Python implementations and limiting argument injection and unintended CLI side effects.
git.cmd.Git, existing unsafe option/protocol checks, validated operands, and framed stdin records. Shell execution and option reordering past protective flags are rejected. Persistentcat-filerequests use NUL framing.git hook run. Signing/editor options and custom archive format commands require an unsafe-options opt-in.GitCommandErrorwith Git's status and diagnostics.oldhexsha, precompressed object streams and custom object-output writers,Submodule.rename(), and directRepo.alternatesmutation. Configuration follows Git syntax and no longer subclassesRawConfigParser. Detailed replacements and limitations are in the changelog.The change also updates documentation, format/backend and injection regressions, the minimum-Git CI job, and fuzz harnesses. Existing performance benchmarks use bounded samples to keep subprocess-based runs practical. The release announcement above describes the planned release; this PR does not change
VERSIONyet.CI follow-up handles platform path conventions and rejects unsupported index filenames before changing the original index. Submodule updates now close their internally opened repositories, preventing retained log records from keeping Windows checkouts open through Git processes. Cygwin builds the supported Git version in its normal installation layout and verifies detection before testing.
User Prompts