Skip to content

fix(selfhost): enforce structured JSON output for OpenAI-compatible review calls, and stop retrying a deterministic-identical response #8790

Description

@JSONbored

Context

Confirmed production incident (same 2026-07-26 window as the bail-misfire issue): with `AI_PROVIDER=claude-code,ollama`, every ollama fallback attempt for the PR review returned well-formed MARKDOWN prose ("### Review of Code Changes…") instead of the required JSON — `hasJsonObject: false`, byte-identical `responseChars: 2814` across all 3 attempts — so the retry budget was pure waste and the review went inconclusive. The fallback provider is currently structurally incapable of rescuing a failed primary review, which converts EVERY primary bail/failure into a manual-review hold.

Requirements

  • `createOpenAiCompatibleAi` (src/selfhost/ai.ts) should request structured output for review-shaped calls — `response_format: { type: "json_object" }` (supported by Ollama's OpenAI-compatible endpoint and by vLLM/OpenAI) — with a graceful fallback when a server rejects the parameter.
  • Early-stop on identical retries: when attempt N returns byte-identical output to attempt N-1 (deterministic generation), further same-model retries are provably useless — skip them (mirror the fix(review): short-circuit the retry loop on a deliberate INCOHERENT_DIFF_ASSESSMENT bail #7518 dont-retry-deliberate-bail precedent) and fall through to the next model/provider immediately.
  • Consider logging a distinct event when a fallback provider produces 0 parseable outputs across a full review, so a structurally-useless fallback is visible before the next incident.

Deliverables

  • response_format on the OpenAI-compatible chat call (+ rejection fallback).
  • Identical-output early-stop with test.
  • Tests covering both.

Test Coverage Requirements

99%+ patch coverage, branch-counted.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions