perf(database): index BatchTaskRun on (runtimeEnvironmentId, createdAt, id) for the batches list - #4361
Merged
Merged
Conversation
…t, id) for the batches list The batches list page orders by createdAt DESC, id DESC filtered by environment and a created-at window, but the only matching index was (runtimeEnvironmentId, id), which cannot satisfy the createdAt ordering. On environments with many batches the query fell back to a full table scan and in-memory sort, which could run long enough to hit the statement timeout. Adds (runtimeEnvironmentId, createdAt DESC, id DESC) so the query reads straight from the index in order. Created with CONCURRENTLY IF NOT EXISTS, so it takes no table lock and is a no-op if already present.
|
ericallam
marked this pull request as ready for review
July 24, 2026 10:29
d-cs
approved these changes
Jul 24, 2026
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Contributor
WalkthroughAdds a composite descending index on 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Merged
ericallam
added a commit
that referenced
this pull request
Jul 27, 2026
…e dedicated schema The batches list orders by (createdAt DESC, id DESC), which is why #4361 added a matching index on BatchTaskRun. That index only landed in @trigger.dev/database. The dedicated run-ops database has its own migration history, so it never received the index and the page kept falling back to a scan and in-memory sort for environments whose batches live there. The list reads both databases, so the slow one stayed on the hot path. Adds the index there, and a schema-parity test comparing the physical shape of every shared model across the two schemas: scalar fields with their attributes plus @@index, @@unique, @@id and @@Map, ignoring relation navigation fields since the run-ops schema deliberately drops relations that would cross databases. A run-graph change that lands in one schema and not the other is now a failing test rather than a silent miss.
carderne
pushed a commit
that referenced
this pull request
Jul 27, 2026
## Summary 2 new features, 9 improvements, 3 bug fixes. ## Highlights - Allow additional environment API keys to create scoped public access tokens through the Trigger.dev API. Use server-issued public access tokens for batch operations so environment-scoped API keys can read batch results. ([#4387](#4387)) ## Improvements - Preserve the partial assistant message when a chat turn's model stream fails mid-response. `chat.agent` now passes the recovered partial to `onTurnComplete`, and `chat.createSession`'s `turn.complete()` keeps it before rethrowing, instead of dropping the streamed-so-far output. ([#4348](#4348)) ## Server changes These changes affect the self-hosted Docker image and Trigger.dev Cloud: - Favorite any dashboard page to a new Favorites section in the side menu, and customize the sidebar by renaming favorites, hiding items, and reordering items and sections. ([#4375](#4375)) - List API endpoints now clamp the page size to a maximum of 100. Requests asking for a larger page size return up to 100 items and keep paginating, rather than pulling an unbounded page. ([#4360](#4360)) - Organizations without billing alerts now get default spend alert thresholds, so you're notified before usage grows unexpectedly. The billing limit page no longer pre-selects an option before you've set a limit and prompts you to configure one. Alert previews now update immediately after you change your billing limit. ([#4328](#4328)) - When you create a Personal Access Token, the generated token now shows its first and last few characters instead of being fully hidden, so you can confirm you copied the right value. ([#4363](#4363)) - Add metrics to the realtime backend that measure how often a single changed run is served to multiple subscriptions in one batch. ([#4341](#4341)) - Realtime run subscriptions can now be configured to read run data straight from the primary database, so a run's latest state is never served from a lagging replica. Off by default; replica reads are unchanged unless you turn it on. ([#4378](#4378)) - SSO and Directory Sync are no longer restricted to Enterprise plans — get in touch and we can turn them on for your organization whatever plan you're on. ([#4393](#4393)) - Improved supervisor observability: it now reports metrics for its outbound requests, making failed calls to upstream services easier to monitor. ([#4350](#4350)) - The runs list on a task's page now updates live — run statuses change and newly triggered runs appear without a manual refresh, matching the main Runs page. ([#4377](#4377)) - Speed up the Batches list page for environments with a large number of batches, which could previously time out while loading. ([#4361](#4361)) - Container startup no longer prints database and ClickHouse connection strings (with credentials) to the logs. ([#4346](#4346)) - The tasks page no longer runs two queries whose results were never displayed, cutting wasted work on every page load and removing a source of hidden server errors ([#4380](#4380)) <details> <summary>Raw changeset output</summary> # Releases ## @trigger.dev/build@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.8` ## trigger.dev@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.8` - `@trigger.dev/build@4.5.8` - `@trigger.dev/schema-to-json@4.5.8` ## @trigger.dev/core@4.5.8 ### Patch Changes - Allow additional environment API keys to create scoped public access tokens through the Trigger.dev API. Use server-issued public access tokens for batch operations so environment-scoped API keys can read batch results. ([#4387](#4387)) ## @trigger.dev/python@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/sdk@4.5.8` - `@trigger.dev/core@4.5.8` - `@trigger.dev/build@4.5.8` ## @trigger.dev/react-hooks@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.8` ## @trigger.dev/redis-worker@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.8` ## @trigger.dev/rsc@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.8` ## @trigger.dev/schema-to-json@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.8` ## @trigger.dev/sdk@4.5.8 ### Patch Changes - Preserve the partial assistant message when a chat turn's model stream fails mid-response. `chat.agent` now passes the recovered partial to `onTurnComplete`, and `chat.createSession`'s `turn.complete()` keeps it before rethrowing, instead of dropping the streamed-so-far output. ([#4348](#4348)) - Allow additional environment API keys to create scoped public access tokens through the Trigger.dev API. Use server-issued public access tokens for batch operations so environment-scoped API keys can read batch results. ([#4387](#4387)) - Updated dependencies: - `@trigger.dev/core@4.5.8` </details> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
ericallam
added a commit
that referenced
this pull request
Jul 27, 2026
…e dedicated schema (#4396) ## Summary The batches list page orders by `(createdAt DESC, id DESC)`, which is why [#4361](#4361) added a matching index on `BatchTaskRun`. That index only landed in `@trigger.dev/database`. The dedicated run-ops database has its own migration history, so it never received the index. `BatchListPresenter` reads both databases and merges, so for environments whose batches live in the dedicated database the page kept falling back to a scan and in-memory sort, which is the exact behaviour #4361 set out to fix. ## Fix Adds the index to the run-ops schema with its own migration. `CREATE INDEX CONCURRENTLY IF NOT EXISTS`, so it is a no-op where the index already exists and still records its ledger row. The second half is the interesting part. Because the two packages own separate migration histories, a run-graph schema change has to be authored twice, and nothing made the miss visible: the run-ops status check truthfully reports "up to date" against its own history, so the apply step just skips. `schemaParity.test.ts` compares the physical shape of every model the run-ops schema declares against its counterpart in `@trigger.dev/database`: scalar fields with their attributes, plus `@@index`, `@@unique`, `@@id` and `@@map`. Relation navigation fields are excluded, since the run-ops schema deliberately drops relations that would cross a database boundary while keeping the scalar FK column. A field counts as a relation when its type resolves to a model name, which keeps enum-typed columns in scope. Two models are listed as run-ops-only: `CompletedWaitpoint` and `WaitpointRunConnection`, both explicit FK-free replacements for a control-plane implicit many-to-many, since an implicit m2m carries a foreign key that cannot resolve across databases. The test also asserts that exception list is exhaustive, so a new unpaired model fails rather than being silently skipped. Confirmed the guard actually fails: reverting the index turns `BatchTaskRun` red with the missing `@@index` named in the diff.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The batches list page orders by
createdAt DESC, id DESCfiltered by environment and a created-at window, but the only supporting index onBatchTaskRunwas(runtimeEnvironmentId, id). That index can't satisfy thecreatedAtordering, so on environments with a large number of batches the query fell back to a full table scan and in-memory sort, which could run long enough to hit the statement timeout.Fix
Adds
(runtimeEnvironmentId, createdAt DESC, id DESC)onBatchTaskRun. The query now reads straight from the index in order with no sort step, returning a page with only a handful of heap fetches instead of scanning the whole environment slice.The migration uses
CREATE INDEX CONCURRENTLY IF NOT EXISTS, so it takes no table lock and is a no-op if the index already exists.