Repository navigation
feat: EP / graphical baselines + analytic parity cell (search-extensibility B4b) - #5
Merged
Merged
Conversation
…sibility B4b)
Port autofit_workspace_developer/{ep,graphical} as scripts/gaussian_1d_ep/
{ep,graphical}/baseline.py (shared harness, verbatim config) and add
scripts/analytic_gaussian/parity/benchmark.py, which runs
autofit_workspace_test's analytic_gaussian.py parity script unchanged and
records its verdicts. Reproduced N3/N10/N30 on the B4a laptop host
(DESKTOP-H143S82, PyAutoFit df6d218f1): every truth logL sum matches the
committed value exactly, both N30 sanity FAILs reproduce, and every wall is
faster than the committed developer baseline beyond the 2x policy (improved).
build_catalogue.py reads two sibling result schemas
(autofit-profiling-baseline, autofit-profiling-parity) and exports a runtime
wall plus a memory host_peak_rss record per row on the existing axes; the
reproduction table is wiki/campaigns/ep_graphical_baselines.md.
Closes #4
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- wiki: the parity run used autofit_workspace_test 26877bc69 (after the 825362d seeding change), not 8b86eca55 (the earlier probe). - rows: provenance.host_contention records the observed load (another session shared the host); the exporter names it in validation.reason; cells now record load_average_before/after. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Merged
2 of 3 tasks
5 of 6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
B4b of the search-extensibility epic: ports the EP / graphical-model baselines from
autofit_workspace_developer/{ep,graphical}and adds a profiling cell for the analytic Gaussian parity benchmark, reproduced on the B4a laptop host. The exporter reads two sibling result schemas and exports aruntimewall plus amemoryhost_peak_rssrecord per row on the existingprofiling-summary@2axes (no Pulse registry row, no contract bump). Findings:wiki/campaigns/ep_graphical_baselines.md.Closes #4
Decision taken (flagged)
euclid-ral-gpu-2, A100) does not fit CPU Dynesty runs; B4b pins to the same laptop as B4a (DESKTOP-H143S82, WSL2,platform.node()in every row) so Pulse's same-host rule pairs these records with B4a's. Stated in the wiki page and the catalogue LIMITATIONS. Cross-machine reproduction on RAL is a later campaign.runtime-drift-2x-1msnote inreference, andcomparisonsstays empty.sigma_q95: the prompt says 12.18; the benchmark docstrings bank 12.15 and the closed form gives 12.1536. The cell checks against 12.15.Reproduction (PyAutoFit df6d218f1, JAX CPU float64, one fresh-process fit per cell, serial)
baseline.json; all six are faster (improved). Against the developer postfix re-runs, graphical N3 is flat (1.06x), graphical N10 and EP N3 still faster (0.48x, 0.49x). No regression.Analytic parity (seed 0, N = 5):
PARITY: PASS (41/41), both hard caps pass, wall 130.0 s, peak RSS 332 MiB, closed-formsigma_q9512.1536.analytic_gaussian_priors.pyis recorded as parked NEEDS_FIX, not run.Scripts Changed
scripts/gaussian_1d_ep/shared.py: shared simulator, sanity rules, harness and row writer, ported from the developersimulator/util/sanity.py.scripts/gaussian_1d_ep/{ep,graphical}/baseline.py: the EP and graphical N3/N10/N30 cells.scripts/gaussian_1d_ep/config/: verbatim developer config.scripts/analytic_gaussian/parity/benchmark.py: runs autofit_workspace_test'sanalytic_gaussian.pyunchanged and records its verdicts.scripts/misc/tooling/build_catalogue.py: reads theautofit-profiling-baselineand-parityschemas; adds runtime + memory records, HTML sections and the host-pin and drift LIMITATIONS.scripts/misc/test/test_build_catalogue.py: 5 new tests (16 total).results/**,dashboard/**,wiki/campaigns/ep_graphical_baselines.md,README.md,AGENTS.md.No PyAutoFit change.
Test Plan
ruff check .,ruff format --check .pytest -q scripts/misc/test: 16 passedbuild_catalogue.py --check --validate-with ../PyAutoPulse: current,okValidation checklist (--auto run, plan was not pre-approved)
provenance.host_contentionand the catalogue names it invalidation.reason; cells now record load averages). Its claim dispositions are all basis-cited: port faithful (side-by-side +diff -rconfig), truth logL bit-exact, N30 FAILs reproduced, ratios recomputed, deviations (a)–(d) disclosed honestly; Pulse re-ingest is pending sequencing · Heart STALE 85, acknowledged reason list: "release validation incomplete: no rehearsal for current source". STALE passes this leg. An earlier RED ("PyAutoFit: CI failure") was a timing flake intest_emcee_call_path_runs_at_jitted_speedon the 3.12 leg; the failed jobs were rerun green, thenpyauto-heart tick🤖 Generated with Claude Code