perf(index): substantially accelerate text catch-up - #721
Merged
ScriptedAlchemy merged 71 commits intoAug 25, 2026
Merged
Conversation
|
…plan-reopened' into codex/code-index-catchup-pipeline
…plan-reopened' into codex/code-index-catchup-pipeline
`fix(semantic): build activation candidate in daemon` removed
`wait_for_evaluated_semantic_profile_current` from this module but left its
only call site in `advanced_workflow_journey_test.rs` untouched, so the
auto-discovered `daemon_suite` target no longer compiles:
error[E0425]: cannot find function
`wait_for_evaluated_semantic_profile_current` in module `task_session`
The helper is restored verbatim from the last revision that carried it, which
also puts the module's now-unused `ApplicationConfigurationGet` import back to
work. It proves the evaluated profile is selected through the public
configuration authority and ready through the mounted runtime authority, and
that the readiness receipt authorizes the exact configuration the SDK selected.
ScriptedAlchemy
marked this pull request as ready for review
August 25, 2026 23:55
ScriptedAlchemy
merged commit Aug 25, 2026
8b1afa2
into
codex/tracedecay-total-redesign-plan-reopened
5 checks passed
This was referenced Aug 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary\n\n- batch authenticated lexical source pages with exact cursor rollback and atomic bulk commits\n- parallelize bounded relational preparation and publish real-time Code/Observatory progress\n- add feature-gated, low-cardinality Hotpath instrumentation plus repository/Codex MCP configuration\n- continue the production optimization until the cold text artifact is Ready within 300 seconds and below 8 GiB RSS\n\n## Current verified evidence\n\n- source batch tests: 6/6\n- disk artifact tests: 27/27 feature-off and 27/27 feature-on\n- sealed-source integration: 2/2\n- Hotpath reader lock regression: 1/1 feature-on\n- microbenchmark: 240 commits reduced to 15; 7.46x ingestion and 5.96x end-to-end; byte-identical digests\n- Rust library check, generated-contract check, dashboard typecheck/build, and focused Code/Observatory DOM suites green at the latest checkpoint\n\n## Truthful production RED baseline\n\nThe preserved external fixture contains 4,851 tracked files (4,399 admitted sealed files); the historical “10,592 files” count included 5,741 .git internals and is not the indexing denominator.\n\nFirst feature-off production run:\n- no Ready receipt after 414.821s\n- 92 pages / 11,324 chunks / 88 of 4,399 files committed\n- 26.96 MiB of 1.029 GiB lexical bytes complete; ETA ~1,565s\n- peak RSS 9,235,228 KiB (8.807 GiB)\n- 20.55 MiB source payload produced 27.59 GiB userspace writes and 3.59 GiB physical writes\n\nConfirmed remaining causes:\n- mutable worktree wakes cancel/restart immutable 1.47 GiB sealed-source verification\n- full active-generation decode retains multiple corpus-sized wire/object representations\n- rejected batches are restaged and one refusal keeps later batches artificially narrow\n- 33 per-row SQLite epoch triggers, online secondary indexes, and row-at-a-time derived stats roughly double five million mutations and amplify writes\n\n## Ongoing acceptance\n\n- generation-owned text cancellation with reconcile fairness\n- accepted-prefix batching without replay\n- zero-copy/streaming generation decode and a lightweight text owner\n- authenticated SQLite bulk-load followed by set-based stats and deferred indexes\n- feature-off and feature-on cold journeys under 300s / 8 GiB with identical generation/artifact receipts\n- live Chrome verification of Code and Observatory one-second progress through scan, prepare, commit, finalization, and Ready\n