Skip to content

Commit 63dc3ab

Browse files
committed
fix(table): run batch unique check inside import transaction
`checkBatchUniqueConstraintsDb` queried the global `db` connection, so inside a single import transaction (one tx wrapping all batches) the constraint lookup couldn't see uncommitted rows from prior batches — duplicates that crossed `CSV_MAX_BATCH_SIZE` boundaries slipped through. Accept an optional executor and pass `trx` from `batchInsertRowsWithTx` so the lookup observes the in-flight transaction state.
1 parent 553ae99 commit 63dc3ab

2 files changed

Lines changed: 20 additions & 3 deletions

File tree

apps/sim/lib/table/service.ts

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -849,7 +849,12 @@ export async function batchInsertRowsWithTx(
849849

850850
const uniqueColumns = getUniqueColumns(table.schema)
851851
if (uniqueColumns.length > 0) {
852-
const uniqueResult = await checkBatchUniqueConstraintsDb(data.tableId, data.rows, table.schema)
852+
const uniqueResult = await checkBatchUniqueConstraintsDb(
853+
data.tableId,
854+
data.rows,
855+
table.schema,
856+
trx
857+
)
853858
if (!uniqueResult.valid) {
854859
const errorMessages = uniqueResult.errors
855860
.map((e) => `Row ${e.row + 1}: ${e.errors.join(', ')}`)

apps/sim/lib/table/validation.ts

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -379,14 +379,26 @@ export async function checkUniqueConstraintsDb(
379379
return { valid: errors.length === 0, errors }
380380
}
381381

382+
/**
383+
* Minimal executor surface needed by unique-constraint checks. Both `db` and a
384+
* drizzle transaction (`trx`) satisfy this, letting callers run the lookup
385+
* inside an open transaction so it observes uncommitted prior-batch inserts.
386+
*/
387+
type UniqueCheckExecutor = Pick<typeof db, 'select'>
388+
382389
/**
383390
* Checks unique constraints for a batch of rows using targeted database queries.
384391
* Validates both against existing database rows and within the batch itself.
392+
*
393+
* Pass a transaction as `executor` when running inside an open tx so the lookup
394+
* sees rows inserted by earlier batches in the same transaction; otherwise the
395+
* default `db` connection only observes committed state.
385396
*/
386397
export async function checkBatchUniqueConstraintsDb(
387398
tableId: string,
388399
rows: RowData[],
389-
schema: TableSchema
400+
schema: TableSchema,
401+
executor: UniqueCheckExecutor = db
390402
): Promise<{ valid: boolean; errors: Array<{ row: number; errors: string[] }> }> {
391403
const uniqueColumns = getUniqueColumns(schema)
392404
const rowErrors: Array<{ row: number; errors: string[] }> = []
@@ -458,7 +470,7 @@ export async function checkBatchUniqueConstraintsDb(
458470
return sql`(${userTableRows.data}->${sql.raw(`'${columnName}'`)})::jsonb = ${normalizedValue}::jsonb`
459471
})
460472

461-
const conflictingRows = await db
473+
const conflictingRows = await executor
462474
.select({
463475
id: userTableRows.id,
464476
data: userTableRows.data,

0 commit comments

Comments
 (0)