Skip to content

perf(db): index ai_usage_events(status, created_at) for the daily-budget query - #657

Merged
JSONbored merged 1 commit into
mainfrom
fix/ai-usage-budget-index
Jun 13, 2026
Merged

perf(db): index ai_usage_events(status, created_at) for the daily-budget query#657
JSONbored merged 1 commit into
mainfrom
fix/ai-usage-budget-index

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Investigation: are hot-path queries doing full table scans?

Audited the prod D1 index inventory against every query the code runs. Conclusion: the per-webhook / gate hot path is fully indexedissues/pull_requests/pull_request_files/bounties by repo_full_name, official_miner_detections by login (PK), repositories/repository_settings by PK. The recent prod D1 overload is therefore capacity / DB-size related (~2.87 GB, blob-heavy tables), not a missing-index problem on the hot path.

A few things I checked and ruled out:

  • webhook_events (39,780 rows) — the app only reads it by delivery_id (PK). The 40k-row scans I saw earlier were my own ad-hoc diagnostic queries, not the app. No index needed.
  • audit_events (15,876 rows) — both app reads filter by event_type first, covered by audit_events_type_created_idx. No index needed.

The one real gap

sumAiEstimatedNeuronsSince (the daily AI-neuron budget check) runs:

SELECT sum(estimated_neurons) FROM ai_usage_events WHERE status = 'ok' AND created_at >= ?

EXPLAIN QUERY PLAN on prod confirms a full SCAN ai_usage_events — the existing indexes are (feature, created_at) and (actor, created_at), neither of which covers a status-only filter. This query runs on every AI review/summary (it's the budget gate), and ai_usage_events grows with each AI call, so the scan cost grows unbounded once AI is enabled — which #652 (AI review) is about to do. It also serves the existing ai-summaries budget query on main.

Fix

Add a covering index ai_usage_events_status_created_idx (status, created_at) (schema + migration 0028). A test asserts via EXPLAIN QUERY PLAN that the budget query now SEARCHes via the index instead of scanning. Full suite green.

Migration numbered 0028 to avoid colliding with 0026/0027 reserved by #652; the D1 runner applies un-applied migrations in order and tolerates the gap.

Out of scope (flagged separately)

The actual prod D1 overload looks like capacity/size, not query plans — worth watching D1 metrics / considering retention pruning on the large append-only + blob tables, but that's an ops decision, not a code fix.

…get query

sumAiEstimatedNeuronsSince runs SELECT sum(estimated_neurons) WHERE status='ok'
AND created_at >= ? — verified via EXPLAIN QUERY PLAN on prod to be a full
`SCAN ai_usage_events` (no covering index; existing indexes are on feature/actor).
It runs on every AI review/summary (the daily-budget check) and the table grows
with each AI call, so the scan cost grows unbounded once AI is enabled.

Add the covering index (status equality + created_at range). EXPLAIN confirms
the query now SEARCHes via it. Investigation found this is the ONLY un-indexed
read in the codebase — the per-webhook/gate hot path (issues/pull_requests/
files/bounties by repo, official_miner_detections by login, settings/repos by
PK) is already fully indexed, so the recent prod D1 overload is capacity/DB-size
related, not a missing-index problem on the hot path.

Migration numbered 0028 to avoid colliding with 0026/0027 reserved by #652.
@ghost

ghost commented Jun 13, 2026

Copy link
Copy Markdown

gittensory · advisory review

Reviewed 3 changed file(s) — two independent AI reviewers.

Suggested action:Safe to merge — both reviewers found no blocking issues.

Reviewer A · gpt-oss-120b — recommends ✅ merge
This PR adds a new SQLite index to support the daily AI usage budget query, updates the schema to reflect the index, and includes a unit test verifying the index exists and is used by the query plan. The changes are straightforward, safe, and well‑documented.

Suggestions

  • Confirm that the migration runner includes the new 0028 file in production deployments.
  • Consider adding a benchmark test to ensure the index provides the expected performance improvement under realistic data volumes.

Worth double-checking

  • The new index will add write overhead on inserts/updates to ai_usage_events; monitor impact on write latency.
  • If the migration order changes, the index could be created after dependent code runs; ensure migrations are applied before the app starts.

Reviewer B · nemotron-3-120b-a12b — recommends ✅ merge
The PR adds a performance index on ai_usage_events(status, created_at) to optimize the daily-budget query (sumAiEstimatedNeuronsSince) used in AI reviews/summaries. It includes a migration (0028), schema synchronization, and a test verifying index usage via EXPLAIN QUERY PLAN. The change is safe, well-documented, and does not affect public/private boundaries or introduce forbidden terms.

Worth double-checking

  • Write performance on ai_usage_events may slightly decrease due to index maintenance (acceptable trade-off for read-heavy workload).
  • Confirm migration number 0028 does not conflict with existing migrations in main branch (author notes 0026/0027 reserved for PR feat(ai-review): dual-AI maintainer review + BYOK (Phase C) #652).

@ghost ghost added the gittensory-review label Jun 13, 2026
@ghost

ghost commented Jun 13, 2026

Copy link
Copy Markdown

Note

Gittensory Gate skipped

PR closed before full evaluation. No late first comment was created.

Signal Result Evidence Action
Gate result ⚠️ Skipped #657 is no longer open. No action.

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

@ghost ghost added the gittensory:reviewed label Jun 13, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored
JSONbored merged commit 956333f into main Jun 13, 2026
10 checks passed
@JSONbored
JSONbored deleted the fix/ai-usage-budget-index branch June 13, 2026 23:09
@github-project-automation github-project-automation Bot moved this from Todo to Done in gittensory - v1 roadmap Jun 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant