Skip to content

fix(sdk): recover mid-stream provider 5xx/429 like severed connections - #1162

Closed
nordicnode wants to merge 9045 commits into
CodebuffAI:mainfrom
nordicnode:oss/recover-provider-5xx-midstream-1155
Closed

fix(sdk): recover mid-stream provider 5xx/429 like severed connections#1162
nordicnode wants to merge 9045 commits into
CodebuffAI:mainfrom
nordicnode:oss/recover-provider-5xx-midstream-1155

Conversation

@nordicnode

Copy link
Copy Markdown

Summary

  • a provider-reported 500/429 arriving mid-stream — the openai-compatible shim enqueues it as an error part with finishReason='error' — no longer ends the entire run with an error
  • extend classifyThrownStreamRecovery (already consulted for both the thrown-iterator and error-chunk shapes) so retryable APICallErrors (429, any 5xx) take the same capped recovery path as a severed body: note injected into the conversation, retry step forced, capped at MAX_CONSECUTIVE_STREAM_RECOVERIES
  • the recovery message names the HTTP status instead of claiming the connection dropped
  • client-error statuses (400/401/402/403) are deterministic — retrying cannot help — so they stay fatal and still propagate to the run's error handling, including the existing 402 rethrow
  • add regression tests: 500 and 429 recover with a status-naming message, a wrapped 503 behind a RetryError cause chain recovers, client-error statuses and post-cancel 5xx stay fatal

This addresses the run-killing part of #1155: the recoverable class was "the connection failed to speak" (isTransientNetworkError, a socket-level allowlist) and the fatal class was "the provider reported a failure", which is backwards for flaky endpoints, where both are the same transient event.

Validation

  • bun test sdk/src/__tests__/stream-interruption.test.ts (19 pass, 5 new)
  • bun test sdk (493 pass; the 100 failures are network/env-dependent and pre-exist on main — same count with this change stashed)
  • bunx tsc --noEmit -p sdk/tsconfig.json clean

Refs #1155

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants