test(performance): generate history-independent benchmark inputs

The backend benchmark fetched an old GitPython commit from the source
checkout and used its current Git revision to identify the code under
test. Both dependencies prevented running from a source archive and
required a full-history CI checkout.

Generate a fixed 30-commit repository with an annotated tag, nested tree,
and submodule through `gix.testtools`. Identify tested sources by a digest
of `VERSION` and package Python files, so local code edits are included
without requiring Git metadata. Reject source-digest mismatches during
comparison and cover changed and relocated source trees in tests.

Update the fixture revision in the budget while retaining every existing
CLI-call ceiling. Remove source `.git` before generating the benchmark
fixture, drop the full-depth benchmark checkout, and document
that old-history timing results cannot be compared to the new workload.

Validation: minimal-sample CLI and Gix benchmark runs from a source copy
without `.git` pass result parity and all CLI-call ceilings. The benchmark
comparison tests, Ruff lint/format, ShellCheck, and YAML parsing pass.
These runs validate the harness; they are not rigorous timing measurements.

Agent-Model: gpt-6-astra

Author: codex

.github/workflows/backend-benchmark.yml

@@ -16,7 +16,6 @@ jobs:
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
-          fetch-depth: 0
           persist-credentials: false
       - uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
         with:
@@ -50,7 +49,8 @@ jobs:
           .bench-gix/bin/python -m pip install '.[gix]' -r test/performance/benchmark-requirements.txt
       - name: Prepare the shared read-only benchmark fixture
         run: |
-          .bench-cli/bin/python -c 'from pathlib import Path; from gix import testtools; print("BENCH_REPO=" + str((testtools.scripted_fixture_read_only_with_args("benchmark.sh", [str(Path.cwd())]) / "repo").resolve()))' >> "$GITHUB_ENV"
+          rm -rf .git
+          .bench-cli/bin/python -c 'from gix import testtools; print("BENCH_REPO=" + str((testtools.scripted_fixture_read_only_with_args("benchmark.sh", []) / "repo").resolve()))' >> "$GITHUB_ENV"
           mkdir benchmark-results
       - name: Measure GixPython operations, opening and discovery first
         env:

test/performance/README.md

@@ -23,13 +23,13 @@ uv pip install --python .tox/gix/bin/python -r test/performance/benchmark-requir
 uv pip install --python .venv/bin/python -r test/performance/benchmark-requirements.txt
 ```
 
-Prepare the same fixed repository used by CI (GitPython 3.1.45) through
-`gix.testtools`, outside the measured operations. Its Bash script lives in
-`tests/fixtures/benchmark.sh`; no remote access is needed if that revision
-already exists locally:
+Prepare the same generated repository used by CI through `gix.testtools`,
+outside the measured operations. `tests/fixtures/benchmark.sh` creates 30
+commits, an annotated tag, a nested directory, and a submodule from scratch.
+It requires neither network access nor source Git metadata:
 
 ```sh
-BENCH_REPO=$(.venv/bin/python -c 'from pathlib import Path; from gix import testtools; print((testtools.scripted_fixture_read_only_with_args("benchmark.sh", [str(Path.cwd())]) / "repo").resolve())')
+BENCH_REPO=$(.venv/bin/python -c 'from gix import testtools; print((testtools.scripted_fixture_read_only_with_args("benchmark.sh", []) / "repo").resolve())')
 ```
 
 Run from the source root, sequentially to avoid competing for CPU and disk:
@@ -48,7 +48,7 @@ The default uses six worker processes, three measured values per process and
 one warmup, with calibrated loop counts and a minimum value duration of 100 ms.
 Pass standard `pyperf` options such as `--rigorous` for more samples or
 `--debug-single-value` for a smoke check. Raw JSON retains sample distributions,
-environment details, fixture/source revisions, result digests, and one untimed
+environment details, fixture revisions, source content digests, result digests, and one untimed
 invocation's native/fallback decisions. The comparison rejects differing
 results, revisions, Python/Git versions and storage formats. The harness also
 rejects the wrong backend rather than silently comparing CLI against itself.
@@ -64,7 +64,7 @@ mutates that worktree, and ignores ambient Git environment/config overrides.
 The journey also enumerates submodules, resolves tree/index revisions with
 their paths and modes, and reads raw commit/tree bytes through the object
 database. The last operation covers serialization readback without rewriting
-objects; the pinned fixture's signed commit remains intact. These three
+objects. The generated fixture includes a submodule for the inventory check. These three
 measurements have zero-CLI ceilings and join the existing CI job automatically.
 
 Add a function returning JSON-compatible results to `MEASUREMENTS` in
@@ -98,4 +98,7 @@ Timing is observational: shared CI runners and local activity introduce noise.
 The CI job publishes all means, standard deviations and ratios, retains raw
 artifacts, and uses `pyperf compare_to` for significance reporting. It fails on
 execution/parity errors, without a noisy speed threshold. Compare like-for-like
-fixture and source revisions when tracking performance across backend changes.
+fixture revisions and source content digests when tracking performance across backend changes.
+The source digest covers `VERSION` and the Python package files, including local
+edits. Measurements from the old GitPython-history fixture are not comparable
+with this generated workload.

test/performance/bench_repository.py

@@ -124,6 +124,13 @@ def worker_arguments(cmd, args):
     cmd.extend(["--repo", str(args.repo), "--expect-backend", args.expect_backend])
 
 
+def source_digest(root):
+    digest = sha256()
+    for path in sorted([root / "VERSION", *(root / "git").rglob("*.py")]):
+        digest.update(path.relative_to(root).as_posix().encode() + b"\0" + path.read_bytes() + b"\0")
+    return digest.hexdigest()
+
+
 def main():
     runner = pyperf.Runner(
         processes=6,
@@ -156,7 +163,7 @@ def git(*arguments, cwd=args.repo):
         backend=_backend.name,
         gixpython=version("GixPython"),
         git_version=git("version"),
-        gitpython_revision=git("rev-parse", "HEAD", cwd=Path.cwd()),
+        gitpython_source=source_digest(Path(__file__).resolve().parents[2]),
         fixture_revision=git("rev-parse", "HEAD"),
         fixture_tree=git("rev-parse", "HEAD^{tree}"),
         fixture_status=git("status", "--porcelain") or "clean",

test/performance/cli-budget.json

@@ -1,5 +1,5 @@
 {
-  "fixture_revision": "6ba2c0a2f9ee7feffd7e079621c4845820180c9a",
+  "fixture_revision": "bd49b71d9f760b7e1a0753c42a6c0ad0a6c48175",
   "max_gix_cli_processes": {
     "journey": 1,
     "inventory": 0,

test/performance/compare_backends.py

@@ -19,7 +19,7 @@ def compare(cli_path, gix_path, cli_budget=None):
         if (lm["backend"], rm["backend"]) != ("cli", "gix"):
             raise ValueError("Expected CLI results followed by GixPython results")
         for key in (
-            "gitpython_revision",
+            "gitpython_source",
             "fixture_revision",
             "fixture_tree",
             "fixture_status",

test/performance/test_benchmark_comparison.py

@@ -7,7 +7,9 @@
 pytest.importorskip("pyperf")
 
 
-@pytest.mark.parametrize("changed", [None, "result_digest", "fixture_revision", "backend", "cli_processes"])
+@pytest.mark.parametrize(
+    "changed", [None, "result_digest", "fixture_revision", "gitpython_source", "backend", "cli_processes"]
+)
 def test_comparison_requires_parity(capsys, changed):
     from test.performance.compare_backends import compare
 
@@ -20,3 +22,11 @@ def test_comparison_requires_parity(capsys, changed):
         compare(*paths, cli_budget=fixture / "budget.json")
         assert "journey | 150.00" in capsys.readouterr().out
         compare(*paths, cli_budget=fixture / "relaxed-budget.json")
+
+
+def test_source_digest_detects_code_changes_without_git_metadata():
+    from test.performance.bench_repository import source_digest
+
+    fixture = read_only_fixture("benchmark-source.sh")
+    assert source_digest(fixture / "source") != source_digest(fixture / "copy")
+    assert source_digest(fixture / "source") == source_digest(fixture / "moved")

tests/fixtures/benchmark-comparison.sh

@@ -3,12 +3,14 @@ set -eu
 for name in cli gix; do
     backend=$name
     fixture_revision=fixture
+    gitpython_source=source
     result_digest=equal
     cli_processes=0
     if test "$name" = gix; then
         case "$1" in
         backend) backend=different ;;
         fixture_revision) fixture_revision=different ;;
+        gitpython_source) gitpython_source=different ;;
         result_digest) result_digest=different ;;
         cli_processes) cli_processes=1 ;;
         equal) ;;
@@ -20,7 +22,7 @@ for name in cli gix; do
   "version": "1.0",
   "benchmarks": [{"runs": [{"values": [0.1, 0.2]}]}],
   "metadata": {
-    "gitpython_revision": "source", "fixture_revision": "$fixture_revision",
+    "gitpython_source": "$gitpython_source", "fixture_revision": "$fixture_revision",
     "fixture_tree": "tree", "fixture_status": "clean", "fixture_object_format": "sha1",
     "fixture_ref_format": "files", "git_version": "git version 2.52.0",
     "python_version": "3.12.14", "result_digest": "$result_digest", "backend_decisions": "[]",

tests/fixtures/benchmark-source.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -eu
+mkdir -p source/git
+printf '1.0\n' > source/VERSION
+printf 'value = 1\n' > source/git/module.py
+cp -R source copy
+cp -R source moved
+printf 'value = 2\n' > copy/git/module.py

tests/fixtures/benchmark.sh

@@ -1,9 +1,23 @@
 #!/usr/bin/env bash
 set -eu
 
-git init --object-format=sha1 --ref-format=files repo
-git -C repo fetch --no-tags "$1" 6ba2c0a2f9ee7feffd7e079621c4845820180c9a
-git -C repo checkout -b benchmark FETCH_HEAD
+export GIT_AUTHOR_NAME='Benchmark' GIT_AUTHOR_EMAIL='benchmark@example.invalid'
+export GIT_COMMITTER_NAME="$GIT_AUTHOR_NAME" GIT_COMMITTER_EMAIL="$GIT_AUTHOR_EMAIL"
+export GIT_AUTHOR_DATE='2001-01-01T00:00:00 +0000'
+export GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE"
+git init --object-format=sha1 --ref-format=files -b main dependency
+git -C dependency commit --allow-empty -m 'Dependency'
+git init --object-format=sha1 --ref-format=files -b benchmark repo
+mkdir -p repo/git/objects repo/dependency
+printf 'Sample objects\n' > repo/git/objects/readme
+printf '[submodule "dependency"]\n\tpath = dependency\n\turl = ../dependency\n' > repo/.gitmodules
+git -C repo update-index --add --cacheinfo "160000,$(git -C dependency rev-parse HEAD),dependency"
+for n in $(seq 1 30); do
+    printf 'Revision %s\n' "$n" >> repo/README.md
+    git -C repo add .
+    git -C repo commit -m "Revision $n"
+done
+git -C repo tag -a v1.0 HEAD~10 -m 'Sample release'
 git -C repo config core.excludesFile /dev/null
 git -C repo config core.attributesFile /dev/null
 git -C repo config core.fsmonitor false