Skip to content

fix(orb): ack the GitHub webhook before the relay forward (#1623) - #1624

Merged
JSONbored merged 1 commit into
mainfrom
fix/orb-ack-before-relay-forward
Jun 27, 2026
Merged

fix(orb): ack the GitHub webhook before the relay forward (#1623)#1624
JSONbored merged 1 commit into
mainfrom
fix/orb-ack-before-relay-forward

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

The central Orb App (gittensory-api) was returning GitHub webhook delivery errors (visible in the App's Recent Deliveries) on bursty events like check_suite/check_run, even though every event was recorded status:"received" in orb_webhook_events. Root cause: handleOrbWebhook awaited forwardOrbEvent before sending the 202 it owes GitHub. In push mode that forward POSTs to the brokered self-host's relay URL with a 10s timeout; a slow path (e.g. a tailnet/Tailscale-Funnel container) delays the response past GitHub's ~10s delivery deadline, so GitHub marks the delivery failed — despite the Orb having received + queued the event.

Fix: ACK GitHub immediately and run the relay forward after the response via executionCtx.waitUntil (with a fire-and-forget fallback where no execution context exists). The forward + its failure-persistence for the retry-orb-relay cron are extracted into a self-contained, fail-safe relayForward helper.

  • No event is lost: the forward still runs (waitUntil keeps the Worker alive), failures still persist to orb_relay_failures for the retry cron, and the self-host pull-sweep remains a backstop.
  • Behavior for the GitHub ACK is now decoupled from downstream container latency.

Closes #1623.

Scope

  • Conventional Commit title; focused (one handler + its helper + tests).
  • No site//CNAME/Pages; follows CONTRIBUTING.md.
  • Linked issue.

Validation

  • git diff --check · npm run actionlint · npm run typecheck
  • npm run test:coveragerelayForward covered both arms (failed→persist orb_relay_failures; skipped→no-op, never throws) + the handler defers via waitUntil (202 returned, forward scheduled not awaited). scheduleAfterResponse both branches covered (waitUntil present in the new test; absent in the existing handler tests → fire-and-forget).
  • npm run test:workers · build:mcp · test:mcp-pack · ui:* · npm audit --audit-level=moderate

If any required check was skipped, explain why:

  • No migration/OpenAPI/cf-typegen: handler control-flow change only.

Safety

  • No secrets/wallets/etc. exposed.
  • Auth/CORS/GitHub-App change includes negative-path tests — N/A to auth (this is webhook ACK timing); the signature/dedup/size guards are unchanged and still covered.
  • No public GitHub text change.

Notes

  • This handler runs on the cloud gittensory-api (the central Orb receiver), which deploys via Cloudflare Workers Builds on merge. The self-host engine runs handleOrbRelay (the receiver), not this path, so no self-host rebuild is needed.
  • Follow-up (separate): a Tailscale/tailnet self-host is arguably better served by pull mode (the engine drains, no inbound push) than push mode — registerOrbRelayTarget currently defaults to push. waitUntil fixes the GitHub-timeout regardless of mode; pull-mode is a robustness improvement to evaluate next.

handleOrbWebhook awaited forwardOrbEvent before returning the 202 it owes
GitHub. In push mode that forward POSTs to the brokered self-host's relay URL
with a 10s timeout; a slow path (e.g. a Tailscale-Funnel container) delays the
response past GitHub's ~10s delivery deadline, so GitHub marks the delivery
FAILED even though the Orb received + queued the event (rows still recorded
status='received').

Extract the forward + its failure-persistence (for the retry-orb-relay cron)
into a self-contained, fail-safe relayForward helper, and run it AFTER the
response via executionCtx.waitUntil (fire-and-forget fallback where no execution
context exists). No event is lost; the GitHub ACK is decoupled from downstream
container latency.
@dosubot dosubot Bot added the size:M label Jun 27, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jun 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.52%. Comparing base (031cb7b) to head (303ecc3).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1624      +/-   ##
==========================================
+ Coverage   95.51%   95.52%   +0.01%     
==========================================
  Files         204      204              
  Lines       22051    22056       +5     
  Branches     7966     7967       +1     
==========================================
+ Hits        21062    21069       +7     
+ Misses        413      412       -1     
+ Partials      576      575       -1     
Files with missing lines Coverage Δ
src/orb/webhook.ts 100.00% <100.00%> (+3.33%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored
JSONbored merged commit 2a0d644 into main Jun 27, 2026
18 checks passed
@JSONbored
JSONbored deleted the fix/orb-ack-before-relay-forward branch June 27, 2026 22:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Orb webhook delivery timeouts: handler blocks the 202 on the relay forward

1 participant