perf(usage): cut warm usage scans from seconds to milliseconds on large histories

[t3dotgg]

The Usage page was slow on machines with a large transcript history. On zbook (800k cached records, a 50 MB cache), each warm scan took 3 to 18 s. File reads were cheap, because the cache works. The time went to work repeated on every scan:

• Every scan aggregated all 90 days of saved records, even for a 1-day view.
• Each Codex record got a new dedupe key string through Schema JSON encoding, which was then hashed twice.
• Each record called Intl.DateTimeFormat (about 2 µs) and built a bucket key string.
• The whole 50 MB cache was encoded again on almost every scan.
• The walk called stat for each file one at a time.

How

• Skip saved entries last written before the window. The walk already uses this filter for live files. A removed directory with no saved records in the window now reports missing instead of ok, so it cannot claim the source from another environment that still reads it.
• Build Codex dedupe keys only for sessions that span more than one file (a moved rollout). Keys include the session and an occurrence count per file, so they can only collide across files of the same session.
• Aggregator: find the day once per 15-minute slot. A slot that spans midnight, which is possible only for fixed offsets such as +00:01, is formatted per record. The aggregator also reuses the previous record's bucket, adds totals in place, and memoizes rate lookups per rate table.
• Persist: encode only the entries that changed since the last write. The file format is the same. Writes now run one at a time, and a change made during a write marks the cache dirty again.
• Walk: list directories as before, then stat the candidates with a fixed pool of 32 workers. Output keeps readdir order.
• Read up to 4 transcripts at once. Fresh entries are stored in walk order, because saved records dedupe first-come. When two scans race on one file, the read that saw the later file wins. So a slow wide scan cannot replace newer usage with older.

The one-time v5 cache upgrade from #15101 still re-parses Codex history in full, so fast-tier pricing stays correct. Parallel reads make it about 2x faster.

Proof

I ran the real UsageService from main and from this branch against a frozen snapshot of zbook's transcripts (44 GB). I used 30d, 7d, 90d, and 24h hourly windows, both cold from its old cache and warm. Buckets and sources were identical in every case.

zbook, load ~20	main	this branch
Warm scan (all windows)	1.7 to 3.2 s	54 to 246 ms
First view after restart, 30d	2.8 s	0.5 s
v5 upgrade re-parse, 90d	174 s	60 to 81 s

New tests cover these cases:

• Saved history outside the window.
• Which duplicate copy wins after cleanup.
• Both orders of two scans racing on one file. These use FIFO gates, so they are deterministic, and each fails without its fix.
• The incremental cache writer.
• Midnight in quarter-hour and fixed-offset zones.

Reviewed with sol-loop: 7 rounds with GPT-6.1-Sol on high.

Created with Claude Opus 5.5 in Claude Code.

🤖 Generated with Claude Code

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR substantially restructures the production usage scan and metering path with new filtering, deduplication, concurrency, shared cache state, and incremental persistence. The breadth and runtime sensitivity of those changes warrant human review despite the accompanying tests and performance-focused intent.

^{You can add or adjust custom eligibility rules. Learn more.}

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

⚠️ The thread fixture changed, so impact percentages are not directly comparable to the main baseline.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	—	4.9 KiB	—	6.8 KiB	✅
Codex	Thread snapshot wire	—	3.7 KiB	—	4.9 KiB	✅
Codex	Live turn WebSocket wire	—	1.2 KiB	—	2.0 KiB	✅
Codex	Live turn WebSocket decoded	—	20.4 KiB	—	29.3 KiB	✅
Codex	Live turn messages	—	2	—	8	✅
Claude	Total thread wire	—	4.9 KiB	—	6.8 KiB	✅
Claude	Thread snapshot wire	—	3.7 KiB	—	4.9 KiB	✅
Claude	Live turn WebSocket wire	—	1.2 KiB	—	2.0 KiB	✅
Claude	Live turn WebSocket decoded	—	20.8 KiB	—	29.3 KiB	✅
Claude	Live turn messages	—	2	—	8	✅

Baseline: 408ff8a · PR result: 5d38fd0 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 106.1 KiB
• Claude decoded thread snapshot: 106.4 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: pingdotgg/t3code/.coderabbit.yaml
• Review profile: CHILL
• Plan: Team
• Run ID: e2d88972-dd0c-4ca8-bd11-935a4218af71

📥 Commits

Reviewing files that changed from the base of the PR and between 31a9da1 and 5d38fd0.

📒 Files selected for processing (9)

• apps/server/src/usage/UsageService.test.ts
• apps/server/src/usage/UsageService.ts
• apps/server/src/usage/usageAggregation.test.ts
• apps/server/src/usage/usageAggregation.ts
• apps/server/src/usage/usagePricing.ts
• apps/server/src/usage/usageScanCache.test.ts
• apps/server/src/usage/usageScanCache.ts
• apps/server/src/usage/usageTranscriptReader.ts
• apps/server/src/usage/usageTranscripts.ts

💤 Files with no reviewable changes (1)

• apps/server/src/usage/usageTranscripts.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

---

📝 Walkthrough

Walkthrough

The usage pipeline now supports concurrent transcript discovery and reads while preserving traversal order. Cache serialization and persistence are updated. Retained-file filtering, Codex deduplication, aggregation bucketing, and rate lookup behavior also change.

Changes

Usage pipeline

Layer / File(s)	Summary
Transcript discovery and concurrent reads apps/server/src/usage/usageTranscriptReader.ts, apps/server/src/usage/UsageService.ts, apps/server/src/usage/UsageService.test.ts	Transcript files are statted with up to 32 workers, with traversal order preserved. UsageService reads up to four files concurrently and applies accepted cache updates in walk order. Tests cover competing scan completion order.
Scan-cache serialization and persistence apps/server/src/usage/usageScanCache.ts, apps/server/src/usage/usageScanCache.test.ts, apps/server/src/usage/UsageService.ts	The cache writer memoizes serialized file fragments by object identity and emits the current tables and entries. Cache writes are serialized, and failed writes restore the dirty flag.
Retained transcripts and Codex deduplication apps/server/src/usage/UsageService.ts, apps/server/src/usage/UsageService.test.ts	Cached files are filtered against both the requested window and retention cutoff. Codex deduplication is limited to sessions found in multiple files. Tests cover duplicate transcript records and removed directories.
Usage bucketing and rate lookup apps/server/src/usage/usageAggregation.ts, apps/server/src/usage/usageAggregation.test.ts, apps/server/src/usage/usagePricing.ts, apps/server/src/usage/usageTranscripts.ts	Aggregation caches day formatting and bucket lookups, updates token totals in place, and copies totals into output. Rate lookups are memoized. Tests cover day boundaries in quarter-hour time zones.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Refactor

Sequence Diagram(s)

sequenceDiagram
  participant UsageService
  participant TranscriptReader
  participant ScanCacheWriter
  UsageService->>TranscriptReader: listTranscriptFiles(sinceMs)
  TranscriptReader-->>UsageService: ordered transcript paths
  UsageService->>UsageService: read up to four transcripts concurrently
  UsageService->>UsageService: apply accepted cache updates in walk order
  UsageService->>ScanCacheWriter: serialize cache snapshot

Loading

Possibly related PRs

• pingdotgg/t3code#9018: Changes usage aggregation and transcript-cache records that feed the same usage pipeline.
• pingdotgg/t3code#9308: Changes transcript parsing, deduplication, scan-cache fields, and aggregation semantics in the same pipeline.
• pingdotgg/t3code#9019: Changes scan-cache persistence and UsageService refresh behavior.

Suggested reviewers: juliusmarminge

Merge Risk: 🔵 Low · up to 5d38f

The change is mergeable with awareness of a narrow race: simultaneous scans of a shorter transcript replacement can retain old usage if the file is deleted before another scan.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 46.15% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 8 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	⚠️ Warning	The description gives a clear problem statement, implementation details, benchmark results, and test coverage. It omits the required Scope and approval information, and does not state whether the new …	Add a Scope and approval section that links the triaged issue or approved discussion, or explains why this focused fix qualifies without prior approval. In Verification, state whether the new tests were run and report their observed results…

✅ Passed checks (3 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly identifies the main change: improving warm usage-scan performance on large histories.

Full details: Description check

Explanation

The description gives a clear problem statement, implementation details, benchmark results, and test coverage. It omits the required Scope and approval information, and does not state whether the new tests were run or their results.

Resolution

Add a Scope and approval section that links the triaged issue or approved discussion, or explains why this focused fix qualifies without prior approval. In Verification, state whether the new tests were run and report their observed results, including any checks that could not be completed.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}