Skip to content

orb(queue): boot recovery re-pends ALL processing rows with no lease guard — double-processing on overlapping deploys #9023

Description

@JSONbored

Verified in source: recoverProcessingJobs (src/selfhost/pg-queue.ts ~561-577) does SELECT ... WHERE status='processing' and flips every row to pending with no lease-age check and no AND status='processing' guard on the UPDATE — while the file header advertises multi-instance FOR UPDATE SKIP LOCKED safety.

A new instance booting while the old one is still mid-job (any overlapped or rolling deploy) re-pends the old instance's in-flight jobs. Both then run the same PR pass concurrently — and per #9013 the public-surface publish has no mutex, so this produces duplicate gate check-runs and verdict thrash.

Related, same file: reclaimExpiredProcessingJobs (~1609-1629) skips only ids in this instance's in-memory activeJobIds, so a genuinely long (>30 min) pass can be reclaimed and double-run by a sibling. It also never increments attempts, so a job that reliably wedges past the lease loops forever without ever dead-lettering.

Fix

  1. At boot, recover only rows older than the lease (reuse reclaimExpiredProcessingJobs's cutoff), or skip boot recovery on Postgres entirely and rely on the runtime reaper.
  2. Heartbeat the lease from the worker and increment attempts on reclaim so chronic wedgers dead-letter.

Refs #9007, #9013.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.orbGittensory Orb related - maintainer self-hosting analytics.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions