Skip to content

feat: B4a search.fit breakdown cell, profiling-summary@2 exporter, Pages board, epic-1 table - #3

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/search-ext-b4a-breakdown-exporter
Oct 14, 2026
Merged

Jammy2211 merged 1 commit into
mainfrom
feature/search-ext-b4a-breakdown-exporter

Conversation

@Jammy2211

Copy link
Copy Markdown
Contributor

Summary

Search-extensibility B4a: autofit_profiling gets its first measurement and becomes a PyAutoPulse producer. The Nautilus search.fit anatomy from autofit_workspace_developer/searches_minimal/ is re-homed as a breakdown cell, run on the laptop CPU, published as a thin profiling-summary@2 catalogue with a Pages board and a pulse-refresh dispatch, and epic 1's ranked bottleneck table is written up with likelihood_share from autofit_inference.

Closes #2. Merge order: this PR first, then PyAutoPulse (the fit registry row reads the published catalogue), then the autofit_workspace_developer deletion PR.

Per step

  1. Contract read — Pulse REFERENCE.md v2 section, pulse/summary.py::classify, autolens_profiling build_catalogue.py (shape only) and pages_dashboard.yml, and the Mind record complete/2026/10/pulse-refresh-project-identity.md.

  2. Breakdown cell — scripts/gaussian_x1/nautilus/search_fit_breakdown.py. Monkey-patches the nautilus internals (NeuralNetworkEmulator.train, NautilusBound.compute, Sampler.sample_shell/evaluate_likelihood/write/write_shell_update/run) and PyAutoFit's output path (SearchUpdater._save_samples/_compute_latent_samples/_profile_and_summarize/visualize, NonLinearSearch.pre_fit_output/post_fit_output). It charges exclusive wall time through a stack timer (scripts/misc/profiling/exclusive_timer.py), so nested calls are never double-counted and other = wall − instrumented. There are two variants:

    • search_fit: the full af.Nautilus(...).fit, written to results/gaussian_x1/nautilus/breakdown.json.
    • bare_sampler: the findings page's own bare-nautilus.Sampler measurement re-run as the reproduction gate, written to breakdown_bare_sampler.json. The cell exits 1 if NN training, bound geometry or likelihood fall outside ±10 pp of the findings page.

    Both use seed 1 and are deterministic in calls (7 501 and 16 400 likelihood calls in every repeat). Each is the median of 3 fresh fits. The findings prose is in wiki/campaigns/search_fit_breakdown.md, with a new §0 for the re-measurement.

  3. Exporter — scripts/misc/tooling/build_catalogue.py (stdlib, about 300 lines). It turns results/**/*.json into dashboard/catalogue.json, index.html and state.json, with one setup per variant, one breakdown record per component plus a runtime wall record, and an explicit selection for each. A zero checkpoint_io (the bare sampler writes no checkpoint) becomes an inapplicable slot, not a zero record. Every record is unreviewed. Rendering is deterministic because the stamp is the newest measured_at. Use --check for staleness and --validate-with PULSE to run Pulse's own classify. Tests are in scripts/misc/test/. lint.yml now runs pytest plus build_catalogue.py --check --validate-with _pulse (PyAutoPulse checked out after ruff).

  4. Pages + dispatch — .github/workflows/pages_dashboard.yml mirrors autolens_profiling's workflow. It validates the catalogue with Pulse and the badge feed with Brain board/_state.py, deploys dashboard/, then sends repository_dispatch: pulse-refresh (repo, sha, dashboard/catalogue.json) using PAT_PYAUTOLABS. Without the secret it warns and exits 0. state.json uses organ: autofit_profiling, following the project-feed identity in the Mind record.

  5. (PyAutoPulse PR, separate.)

  6. Epic-1 table — wiki/campaigns/search_fit_bottlenecks.md. Epic 2 is not opened.

  7. (autofit_workspace_developer PR, separate.)

Measured breakdown (laptop CPU, DESKTOP-H143S82, WSL2 8 cores, numpy fp64, PyAutoFit 911fe05e1 = 2026.10.12.1+1, nautilus 1.0.5, n_live 200, n_networks 4, 1 core)

Reproduction (bare sampler): wall 22.5 s (repeats 23.1/22.5/21.8). log Z −57.455, the same as the findings reference.

component findings §1 measured Δ
NN training 51.7 % 53.8 % +2.1 pp
bound geometry 25.2 % 25.3 % +0.1 pp
likelihood 14.3 % 11.6 % −2.7 pp
proposal sampling 5.2 % 5.7 % +0.5 pp

All three gated shares are within ±10 pp, so the cell exits 0.

search.fit (epic 1): wall 28.4 s (25.8/28.4/28.9), 7 501 calls. Shares of wall:

component share
NN training 43.4 %
checkpoint I/O (nautilus HDF5, 103 writes, about 69 ms each) 24.5 %
bound geometry 20.4 %
visualisation 4.1 %
sampler bookkeeping 2.3 %
samples output 1.7 %
likelihood 1.1 %
other 1.0 %
proposal sampling 0.8 %

likelihood_share cross-check: B3 Nautilus numpy has a median of 1.9 % (n=11), and Nautilus JAX CPU about 0.01 %.

Pages status

Pages is enabled (gh api repos/PyAutoLabs/autofit_profiling/pages -X POST -f build_type=workflow returned build_type: workflow, html_url: https://pyautolabs.github.io/autofit_profiling/). It is not live yet: the URL returns 404 until pages_dashboard.yml runs on main after this merges, because it triggers on dashboard/** pushes to main. I could not verify that the org secret PAT_PYAUTOLABS is visible to this new repo (reading org secrets needs admin:org). If it isn't, the deploy still succeeds and the dispatch step only warns. In that case, run Pulse's dashboard_refresh.yml by hand.

Validator output (tail)

pulse validator: ok (profiling-summary@2)
catalogue: current

Decisions taken

  • Two variants, not one row. The findings page measured a bare nautilus.Sampler, while a PyAutoFit search.fit differs in three ways: it checkpoints (24.5 %), stops at n_eff=500 (7 501 calls against 16 400), and has a cheaper per-call likelihood (about 43 µs through Fitness against about 160 µs through the bare script's instance_from_vector). Its likelihood share (1.2 % sampler-internal) therefore cannot reproduce 14.3 %. The reproduction gate runs on the faithful bare-sampler variant, and the search_fit row records its comparison as information only (gates_reproduction: false).
  • Tolerance ±10 pp absolute per share. The findings page states ±40 % session variance in wall time, and shares are the stable quantity.
  • The fast 10-d scenario was not ported. It is not a PyAutoFit search.fit. Its numbers stay quoted on the findings page.
  • likelihood_share source. catalogue/search_catalogue.json has no likelihood_share field on main 900648b5 or on feature/search-ext-b3-postmerge-fixes 637713b2 (it aggregates legs by task). The per-run values come from autofit_inference dashboard/summary.json (inference-summary@1) at 637713b253ab149f87a08ea0b02fcb009b4aa24e. The values are identical on main.
  • Library output defaults. config/general.yaml only disables the workspace version check, so the cell measures what a user's search.fit pays. Visualisation is PyAutoFit's search-level default; there is no custom Analysis visualiser.
  • Every record is validation: unreviewed, with qualified: true because provenance is complete. Pulse never judges, and the profiling conductor has not reviewed these numbers.
  • producer_revision: null. The catalogue is committed alongside the rows it describes, so no self-referencing SHA is possible. generated_at is the newest measured_at, which keeps --check deterministic.
  • The wall is a runtime record (metric: wall) and components are breakdown records. Shares are not catalogue records, because a breakdown unit must be s. They are shown on index.html and in the result rows.
  • Project badge status: green with a neutral headline. It reads "baseline only, not a verdict", and the Brain cockpit reads the Pulse organ feed, not this one.

Tests

  • pytest -q scripts/misc/test: 11 passed. The exclusive timer has 4 tests. The exporter has 7, including the committed catalogue being current, the committed rows validating with the Pulse reader, and the Pulse reader refusing a broken unit.
  • ruff check . and ruff format --check . are clean.
  • build_catalogue.py --check --validate-with ../PyAutoPulse reports ok.
  • Brain board/_state.py dashboard/state.json reports state: ok.

🤖 Generated with Claude Code

…board (B4a)

- scripts/gaussian_x1/nautilus/search_fit_breakdown.py: re-homes
  autofit_workspace_developer/searches_minimal/nautilus_profile.py as an
  axis: breakdown cell. Times one af.Nautilus search.fit (exclusive
  component wall: NN training, bound geometry, proposal sampling,
  likelihood, checkpoint I/O, sampler bookkeeping, samples output,
  visualisation, other) and re-runs the bare-sampler measurement as the
  reproduction gate against the findings page (+-10 pp).
- results/gaussian_x1/nautilus/breakdown{,_bare_sampler}.json: laptop CPU
  rows, median of 3 fresh fits.
- scripts/misc/tooling/build_catalogue.py: thin stdlib exporter to
  dashboard/catalogue.json (profiling-summary@2), index.html and the
  project badge feed state.json; --check and --validate-with PULSE.
- scripts/misc/test: stdlib-only tests; lint.yml runs them plus the
  catalogue up-to-date + Pulse-validator check.
- .github/workflows/pages_dashboard.yml: publish dashboard/ to Pages and
  dispatch pulse-refresh to PyAutoPulse.
- wiki/campaigns/search_fit_breakdown.md (findings, re-homed) and
  search_fit_bottlenecks.md (epic-1 ranked table + likelihood_share).

Closes #2

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: B4a search.fit breakdown cell, profiling-summary@2 exporter, Pulse fit row, epic-1 table

1 participant