Skip to content

perf(knowledge): lock the knowledge base row FOR NO KEY UPDATE on document writes - #8909

Merged
waleedlatif1 merged 2 commits into
stagingfrom
perf/kb-lock
Oct 11, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
perf/kb-lock

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Document uploads, document hard deletes, connector creation, and knowledge base restore locked the parent knowledge_base row FOR UPDATE. Every embedding insert takes FOR KEY SHARE on that row through its foreign key for the whole indexing transaction, and FOR UPDATE conflicts with it, so these writes queued behind any indexing pass in the same knowledge base (and a hard delete could deadlock against an indexing pass on one of its own documents).
  • Switch those sites to FOR NO KEY UPDATE. It still conflicts with itself, FOR SHARE (connector sync), FOR UPDATE (KB move, tag mutations) and any UPDATE/DELETE of the row, so every serialization these locks provided is kept; only the foreign-key KEY SHARE conflict goes away.
  • KB move and tag mutations keep FOR UPDATE (they do need to exclude child inserts). Matches soft delete and connector cleanup, which already use FOR NO KEY UPDATE.

Type of Change

  • Performance improvement

Testing

  • Verified every lock site, the work done under each lock, and every other locker/writer of the row; nothing under these locks writes knowledge_base.id (the only key column) or enforces a count-based limit
  • New integration case: single and bulk uploads, then a hard delete, each must commit while another transaction holds FOR KEY SHARE on the KB row (as an in-flight embedding insert does); with FOR UPDATE they wait on it and the case times out
  • biome on changed files; CI runs type-check, audits and the integration suite

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…ument writes

Document uploads, document hard deletes, connector creation, and knowledge
base restore locked the parent knowledge_base row FOR UPDATE. Every embedding
insert takes FOR KEY SHARE on that row through its foreign key and holds it
for the whole indexing transaction, and FOR UPDATE conflicts with KEY SHARE,
so these writes queued behind any indexing pass in the same knowledge base
(and vice versa). A hard delete could also deadlock against an indexing pass
on one of its own documents.

FOR NO KEY UPDATE still conflicts with itself, FOR SHARE (connector sync
saves and completion), FOR UPDATE (KB moves, tag mutations), and the
UPDATE/DELETE of the row, so every serialization these locks provided is
kept. No path under these locks writes knowledge_base.id, the only key
column (the other unique indexes are partial). This matches soft deletion
and connector cleanup, which already use FOR NO KEY UPDATE.
@vercel

vercel Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Oct 11, 2026 4:02am UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[High impact] The PR appears safe to merge; no actionable regression was found.

Summary

Changes five parent-row locks from FOR UPDATE to FOR NO KEY UPDATE across document uploads, hard deletes, connector creation, and knowledge base restore.

  • Document uploads can proceed while embedding inserts are indexing.
  • Hard deletes no longer wait for embedding inserts to finish.
  • Connector creation and restore can run alongside embedding indexing.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A["Uploads, hard deletes, connector creation, restore"] --> B["knowledge_base: FOR NO KEY UPDATE"]
  C["Embedding inserts"] --> D["knowledge_base: KEY SHARE"]
  B --- E["Compatible: both can proceed"]
  D --- E
  F["Archive, move, sync, tag writes"] --> G["Conflicting parent locks"]
  G --> H["Still wait for the changed writes"]
  B --> H
Loading

Reviews (1) · Last reviewed commit: "perf(knowledge): lock the knowledge base..." · Reviewed by Greptile

@waleedlatif1
waleedlatif1 merged commit fd91c62 into staging Oct 11, 2026
46 of 47 checks passed
@waleedlatif1
waleedlatif1 deleted the perf/kb-lock branch October 11, 2026 05:20

This branch was previously deployed

1 inactive deployment
Preview — 09adc679 Deployed Oct 11, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant