Skip to content

Pin PYTHONHASHSEED in the benchmark workflow - #6993

Merged
masenf merged 1 commit into
reflex-dev:mainfrom
benedikt-bartscher:pin-pythonhashseed-benchmarks
Aug 28, 2026
Merged

Pin PYTHONHASHSEED in the benchmark workflow#6993
masenf merged 1 commit into
reflex-dev:mainfrom
benedikt-bartscher:pin-pythonhashseed-benchmarks

Conversation

@benedikt-bartscher

@benedikt-bartscher benedikt-bartscher commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Python randomizes string hashing per process, so dict probe sequences differ between two runs of the same code. The instrumented benchmarks are sensitive enough to pick that up as a few percent of drift on a commit that never touched the measured code path, which makes real regressions harder to spot.

Pinning PYTHONHASHSEED: 0 for the workflow removes that source of run-to-run variation.

CodSpeed doesn't call out PYTHONHASHSEED specifically, but this is the same idea as their Keep Benchmarks Deterministic guidance, applied to the interpreter rather than to the benchmark body. Their write-up on unrelated benchmark regressions covers the wider class of environment-driven drift this belongs to.

Only the workflow env changes; no package source is touched.

Review in cubic

Randomized string hashing changes dict probe sequences from run to run,
so the instrumented benchmarks can report a few percent of drift on a
commit that never touched the measured code path.
@benedikt-bartscher
benedikt-bartscher requested a review from a team as a code owner August 28, 2026 19:30
@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Pins PYTHONHASHSEED to make performance results deterministic across workflow runs.

  • Adds a workflow-level fixed Python hash seed.
  • Documents how randomized dictionary probe sequences can introduce benchmark drift.

Confidence Score: 5/5

The PR appears safe to merge with no actionable defects identified.

The fixed hash seed consistently applies to benchmark runs as intended, while no workflow consumer was found that requires randomized Python hashing.

Important Files Changed

Filename Overview
.github/workflows/performance.yml Adds a documented deterministic Python hash seed without introducing a concrete workflow failure or violating repository rules.

Reviews (1): Last reviewed commit: "Pin PYTHONHASHSEED in the benchmark work..." | Re-trigger Greptile

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

@codspeed-hq

codspeed-hq Bot commented Aug 28, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 32 untouched benchmarks
⏩ 8 skipped benchmarks1


Comparing benedikt-bartscher:pin-pythonhashseed-benchmarks (7aaeb1f) with main (7427617)

Open in CodSpeed

Footnotes

  1. 8 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@masenf
masenf merged commit 4bc9678 into reflex-dev:main Aug 28, 2026
111 checks passed
masenf pushed a commit that referenced this pull request Aug 28, 2026
…-context-refactor-jv3pig

Picks up #6987, #6990, #6989, #6992 and #6993. Clean auto-merge; main's
``templates.py`` edit is again in the Vite config template, away from
``render_iterable_tag``.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants