fix(sessions): block cover-past on observation commit failures - #528
Conversation
|
e4d59ab to
e782c3b
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e782c3b7e6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
3f5b0ea to
94b39dd
Compare
Live daemon.err.log carries ~32k 'admission refused; covering past it' WARNs (provider=codex, reason=observation_commit_failed, ~8630/min) with a 2.8G user-sessions.db and 415M WAL. The JSONL admission seam treated every non-retryable capture error as a content refusal and wrote AdmissionRefused coverage from a stale in-memory cursor over a persist that had already committed and advanced the source cursor. Cover-past now has exactly two outcomes: deterministic content refusals (invalid_observation_contract, privacy_boundary_failed) still cover past so the stream converges; store commit/read-back failures and retryable races surface as typed HostAdmission blocks so the frontier never advances over a record whose durable fate is unknown. A committed persist whose immediate read-back misses reports the Queued status persist just wrote instead of failing the capture as PersistedObservationUnavailable. Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
94b39dd to
fcc5a76
Compare
9430898
into
codex/tracedecay-total-redesign-plan
Live symptom
daemon.err.log(7.9MB) is dominated by ~32k WARNs at ~8630/min:matching a 2.8G
user-sessions.db+ 415M WAL. Cursor hits the same path. Root cause:ActiveAdmission::capturetreated every non-retryable capture error as a deterministic content refusal and wroteAdmissionRefusedcoverage from a stale in-memoryexpected_cursor— after a persist that had already committed the row and advanced the source cursor. Each frame produced a successful persist plus a conflicting cover-past cursor write, 32k times, while a live reader held the WAL open.Cover-past contract (two outcomes)
TranscriptIngestError::HostAdmissionblock, no retry, the frontier does not advance.observation_commit_failed,authority_write_failed, andobservation_persisted_value_unavailableare store write/read failures, not content refusals — they now take the typed-block arm. Only deterministic content refusals (invalid_observation_contract,privacy_boundary_failed) still cover past so the stream converges. Retryable races (cursor_conflict, still-mounting authority, cancellation) keep their own verdict viahost_admission_errorinstead of being laundered into a terminalNonDurableRecord.A committed persist whose immediate read-back returns
None(missed read-your-writes under a busy WAL) now reports theQueuedstatus persist just wrote instead of failing the capture asPersistedObservationUnavailable;exact_duplicate_reports_authoritative_projection_statusstill passes when the read succeeds.Isolated tests (this crate only, no live home dirs)
observation_test.rs: persist commits, read-back misses → capture staysPersistedwithQueued.runtime/jsonl_observation_admission/tests.rs(tempfile rollouts +MemoryHostAdmissionspy):HostAdmissionblock,advance_non_durable_source_cursornever called, frontier stays put;cursor_conflictkeepsretryable: true;invalid_observation_contractstill covers past withAdmissionRefusedand the stream converges (second pass reads 0 bytes, no re-written coverage);cargo test -p tracedecay-sessions(all targets) andcargo clippy -p tracedecay-sessions --all-targetsare green. Onlycrates/tracedecay-sessionsis touched; sessions stays robust to the usecasesclassify_errormapping without changing it.