Skip to content

feat(gate): detect same-table/same-column collisions across differently-numbered migrations - #2607

Merged
JSONbored merged 2 commits into
mainfrom
feat/migration-column-collision-check
Jul 2, 2026
Merged

feat(gate): detect same-table/same-column collisions across differently-numbered migrations#2607
JSONbored merged 2 commits into
mainfrom
feat/migration-column-collision-check

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • scripts/check-migrations.mjs's collision logic only grouped migration files BY FILENAME NUMBER — it never parsed the SQL body to detect two DIFFERENT, individually-valid numbers adding the SAME column to the SAME table.
  • repository_settings alone has taken 5+ independent ALTER TABLE ... ADD COLUMN migrations under unique filenames, confirming this is the hottest actual collision surface in the schema.
  • Two concurrent PRs each independently picking the same column/table combination under different numbers would both pass CI and show mergeable_state: clean (different files, no git conflict), only failing at actual wrangler d1 migrations apply deploy time — after merge, with zero CI signal.

Scope

  • src/db/migration-column-extraction.ts — new pure, fs-free module. Replays every migration file's schema-affecting statements (CREATE TABLE column lists, ALTER TABLE ADD/DROP/RENAME COLUMN) IN MIGRATION-NUMBER ORDER and flags any (table, column) pair defined by more than one file. A DROP TABLE event clears every column tracked for that table so far, so migrations/0060_orb_fleet_collector.sql's documented DROP+CREATE recreate (SQLite can't ALTER away a table-level UNIQUE constraint) correctly does not read as colliding with the table it replaces — the first real false positive found while building this, along with a second bug where an inline trailing column comment's own comma fooled the top-level clause splitter into extracting comment words as fake columns. Both fixed by stripping comments before matching.
  • scripts/check-migrations.mjs — wired in; no new CI job needed, it already runs via the existing db:migrations:check step in test:ci.
  • Tests: 26 new unit tests for the extraction module (100% line/branch coverage) plus 3 new CLI-level tests in test/unit/check-migrations-script.test.ts.

Validation

  • Verified zero false positives against the full 95-file migrations/ directory (this is the acceptance criterion the issue calls out explicitly).
  • npx vitest run test/unit/migration-column-extraction.test.ts test/unit/check-migrations-script.test.ts — 58/58 passing, 100% coverage on the new module
  • npm run typecheck — clean
  • npm run test:ci — full local gate green
  • npm audit --audit-level=moderate — 0 vulnerabilities

Safety

  • Read-only static analysis over already-committed SQL text; no runtime/deploy behavior changes.
  • Bare-identifier SQL parsing only (verified against the actual corpus — no quoted identifiers, no triggers exist in migrations/**), not a general-purpose SQL parser — scoped to what this repo's migrations actually contain, per the issue's own guidance.

Closes #2551

@dosubot dosubot Bot added the size:L label Jul 2, 2026
@loopover-orb

loopover-orb Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-02 13:51:25 UTC

4 files · 1 AI reviewer · no blockers · readiness 80/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • Touches a guarded path — held for manual review

Review summary
The change adds a pure migration SQL event extractor and wires it into the existing migration check so duplicate table/column definitions are caught across numbered migration files. The replay model correctly handles statement order, comments, DROP TABLE recreates, and DROP/RENAME COLUMN removal before flagging later definitions. The visible tests cover the important production path and the CLI wiring, so the change is safe enough to proceed with only parser-scope hardening left.

Nits — 6 non-blocking
  • nit: src/db/migration-column-extraction.ts:146 only matches `ALTER TABLE ... ADD COLUMN ...`; SQLite also accepts `ALTER TABLE ... ADD ...`, so this checker should either support the optional `COLUMN` keyword or document that repo migrations must use the explicit form.
  • nit: src/db/migration-column-extraction.ts:139 and src/db/migration-column-extraction.ts:146 only support bare `\w+` identifiers, which matches the current corpus but will silently ignore future quoted identifiers instead of failing closed.
  • src/db/migration-column-extraction.ts:146: change the ADD regex to make `COLUMN` optional, e.g. `\bALTER\s+TABLE\s+(\w+)\s+ADD\s+(?:COLUMN\s+)?(\w+)`, and add one unit test for that accepted SQLite spelling.
  • src/db/migration-column-extraction.ts:139: consider adding an explicit unsupported-identifier test or fail-closed path if a migration introduces quoted table/column names, since a silent skip weakens this gate.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #2551
Related work ⚠️ 2 scoped overlaps Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (size label size:L; 1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 65 registered-repo PR(s), 55 merged, 536 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 65 PR(s), 536 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Review context
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Review top overlaps.
  • Add a concise scope and risk note.
  • No action.
  • Check active issues and PRs before submitting.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added gittensor gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. labels Jul 2, 2026
@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.04%. Comparing base (c4b797b) to head (2820773).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2607      +/-   ##
==========================================
+ Coverage   96.02%   96.04%   +0.02%     
==========================================
  Files         233      234       +1     
  Lines       26077    26209     +132     
  Branches     9474     9510      +36     
==========================================
+ Hits        25041    25173     +132     
  Misses        425      425              
  Partials      611      611              
Files with missing lines Coverage Δ
src/db/migration-column-extraction.ts 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

JSONbored added 2 commits July 2, 2026 05:58
…ly-numbered migrations

scripts/check-migrations.mjs's collision logic only grouped migration
files BY FILENAME NUMBER -- it never parsed the SQL body to detect two
DIFFERENT, individually-valid numbers adding the SAME column to the
SAME table. repository_settings alone has taken 5+ independent ALTER
TABLE ... ADD COLUMN migrations under unique filenames, confirming this
is the hottest actual collision surface in the schema. Two concurrent
PRs each independently picking the same column/table combination under
different numbers would both pass CI and show mergeable_state: clean
(different files, no git conflict), only failing at actual
wrangler d1 migrations apply deploy time -- after merge, with zero CI
signal.

Added src/db/migration-column-extraction.ts: a pure, fs-free module
that replays every migration file's schema-affecting statements
(CREATE TABLE column lists, ALTER TABLE ADD/DROP/RENAME COLUMN) IN
MIGRATION-NUMBER ORDER and flags any (table, column) pair defined by
more than one file. A DROP TABLE event clears every column tracked for
that table so far, so migrations/0060_orb_fleet_collector.sql's
documented DROP+CREATE recreate (SQLite can't ALTER away a table-level
UNIQUE constraint) correctly does not read as colliding with the table
it replaces -- this was the first real false positive found while
building the check, along with a second bug where an inline trailing
column comment containing its own comma fooled the top-level clause
splitter into extracting comment words as fake columns. Both are fixed
by stripping comments before matching. Verified zero false positives
against the full 95-file migrations/ directory.

Wired into scripts/check-migrations.mjs (no new CI job -- already runs
via the existing db:migrations:check step in test:ci). 26 new unit
tests for the extraction module (100% line/branch coverage) plus 3
new CLI-level tests in check-migrations-script.test.ts.

Closes #2551
…ase it

Gate review (gittensory-orb) flagged a real defect: the collision
detector only checked its tracking map for files.size > 1 at the very
end, after replaying every event -- so CREATE TABLE t (c INT);
ALTER TABLE t ADD COLUMN c INT; DROP TABLE t; was silently accepted,
even though the ADD COLUMN would already fail at real migration
execution time (SQLite runs statements strictly in order) well before
the DROP TABLE is ever reached. A later drop_table event was clearing
evidence of a collision that had already happened.

Fixed by recording a collision into a separate, permanent map the
moment a redefinition is detected against the live tracking state --
before drop_table's clearing logic runs on any subsequent statement.
The existing DROP+CREATE-recreate false-positive fix is unaffected
(verified against the full 95-file migrations/ directory and all
existing tests): a table dropped with no prior collision still clears
cleanly, since nothing was ever recorded for it.

Added a regression test for the exact scenario the gate flagged.
100% branch coverage maintained.
@JSONbored
JSONbored force-pushed the feat/migration-column-collision-check branch from d036cf9 to 2820773 Compare July 2, 2026 13:00
@JSONbored
JSONbored merged commit 6ff084d into main Jul 2, 2026
12 checks passed
@JSONbored
JSONbored deleted the feat/migration-column-collision-check branch July 2, 2026 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(gate): detect same-table/same-column collisions across differently-numbered migrations

1 participant