Skip to content

feat(review): wire a live AI-vision caller for confirmed visual regressions - #4141

Merged
JSONbored merged 1 commit into
mainfrom
feat/ai-vision-live-caller-4111
Jul 8, 2026
Merged

feat(review): wire a live AI-vision caller for confirmed visual regressions#4141
JSONbored merged 1 commit into
mainfrom
feat/ai-vision-live-caller-4111

Conversation

@JSONbored

@JSONbored JSONbored commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Summary

  • Part of the visual-capture convergence epic (Epic: extend automated visual-capture convergence (#577) to self-host + metagraphed #3607); wires the deliberately-deferred feat(review): advisory-only AI-vision analysis of before/after visual captures #4111 follow-up. src/review/visual/visual-findings.ts already shipped the gating/prompt/parse logic but explicitly deferred a live caller — this PR is that caller.
  • New exported runVisualVisionForAdvisory in src/queue/processors.ts, called right after the visual-capture block (independent try/catch, so a vision failure can never affect the "Visual preview" section that block already rendered): resolves submitter reputation + the maintainer's BYOK provider key (mirrors runAiReviewForAdvisory's own resolution, same re-resolve-per-caller convention), runs evaluateVisualVisionGate, fetches the before/after PNGs for the routes it selects, and calls the real BYOK vision provider (Anthropic/OpenAI). A parsed finding becomes an advisory visual_regression_finding — strictly advisory, can never become a gate blocker (see the file's own header).
  • New fetchShotContentBlock in src/review/visual/capture.ts — fetches an already-captured shot URL and returns it as an AiContentBlock, degrading to undefined on any failure so one broken image never sinks the call.
  • Exported the pre-existing (already-tested) base64Encode helper from src/utils/crypto.ts for reuse rather than duplicating byte→base64 logic.
  • Extracted the wiring into its own exported function (rather than inlining it in the giant maybePublishPrPublicSurface) specifically so it's directly unit-testable without driving the full webhook pipeline — see the new test/unit/visual-vision-wiring.test.ts.

Scope

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage — 100% branch/line on every changed hunk in all 3 touched src/** files (verified via scoped --coverage.include runs before and after adding tests)
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate — 0 vulnerabilities
  • New/changed behavior has unit tests for every branch: all 3 gate-decline reasons, the desktop-vs-mobile viewport selection, a missing-URL route (skipped), a failed image fetch (degrades to fewer images), all-images-failed (no provider call at all), a successful call with a published finding, an empty-findings response, an unparseable response, a non-2xx provider response, a thrown error from the BYOK lookup (caught, logged, never escapes), and a null author.
  • Full local gate (npm run test:ci) green end-to-end: 11,920 tests passed, 0 failed.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized — the vision system prompt explicitly instructs the model never to mention rewards/payouts/wallets/hotkeys/coldkeys/trust scores, mirroring the existing text-review prompt's boundary.
  • N/A — no auth/cookie/CORS/GitHub App/Cloudflare/session changes.
  • N/A — no API/OpenAPI/MCP surface touched.
  • N/A — no UI changes.
  • N/A — no docs/changelog changes needed.

Notes

  • Deliberately does NOT change the automated capture's viewport count/dimensions (still desktop + mobile, no theme forcing) — shot.ts's own #4109 comment already explicitly considered and rejected widening to a 3rd viewport by default (50% render-cost increase for every repo, every review), in favor of a future per-repo opt-in if ever needed. Out of scope here.

…ssions

Wires the deliberately-deferred #4111 follow-up: a route the pixel-diff pipeline
already flagged changed now gets one more look from a real vision-capable BYOK
model (Anthropic/OpenAI), publishing an advisory "visual_regression_finding" when
it spots a genuine layout break. Strictly advisory -- can never become a gate
blocker. Mirrors runAiReviewForAdvisory's own resolve-gate-call-parse shape as a
new exported runVisualVisionForAdvisory function so it's directly unit-testable
without driving the full webhook pipeline.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.72%. Comparing base (b8a1ba4) to head (8d7df44).
⚠️ Report is 5 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4141   +/-   ##
=======================================
  Coverage   93.71%   93.72%           
=======================================
  Files         385      385           
  Lines       36209    36240   +31     
  Branches    13266    13281   +15     
=======================================
+ Hits        33934    33965   +31     
  Misses       1618     1618           
  Partials      657      657           
Files with missing lines Coverage Δ
src/queue/processors.ts 95.22% <100.00%> (+0.03%) ⬆️
src/review/visual/capture.ts 91.39% <100.00%> (+0.35%) ⬆️
src/utils/crypto.ts 100.00% <ø> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored JSONbored self-assigned this Jul 8, 2026
@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 8, 2026
@loopover-orb

loopover-orb Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Important

🟪🟪🟪🟪🟪🟪🟪🟪🟪🟪🟪🟪

🔍 Gittensory is reviewing…

AI analysis is in progress. This comment will update when the review is complete.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed · 🟪 Reviewing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Development

Successfully merging this pull request may close these issues.

1 participant