Skip to content

feat(slop): BYOK frontier model for the AI slop advisory - #728

Merged
JSONbored merged 1 commit into
mainfrom
feat/ai-slop-byok
Jun 14, 2026
Merged

feat(slop): BYOK frontier model for the AI slop advisory#728
JSONbored merged 1 commit into
mainfrom
feat/ai-slop-byok

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

What

Extends the AI slop advisory (#724) to optionally use the maintainer's own frontier model (Anthropic/OpenAI) instead of free Workers AI — reusing the existing per-repo encrypted key + aiReviewByok flag (no new key or setting). One BYOK key serves both AI features. Advisory-only either way; BYOK never changes whether the slop pass can block (it can't).

How

  • ai-review.ts: extract callAiProvider (shared BYOK HTTP/timeout/error handling) — runProviderReview now wraps it. Export ProviderFailure + DEFAULT_BYOK_DAILY_REPO_LIMIT.
  • repositories.ts: generalize countByokAiReviewEventsForRepoSincecountByokAiEventsForRepoSince — one shared per-repo/day BYOK cap across all AI features (review + slop), so enabling more capabilities can't multiply frontier spend.
  • ai-slop.ts: the providerKey path skips the free neuron budget, checks the shared BYOK cap, calls callAiProvider + parseSlopOpinion, and records byok:<provider> usage. Workers-AI fallback unchanged. Fail-safe on every path.
  • processors runAiSlopForAdvisory: decrypt the repo key (a declared provider override must match the stored key) and pass it through.

Tests

BYOK provider path (provider used, Workers AI not) + the shared BYOK cap quota; existing Workers-AI gating/fail-safe still green. Typecheck + UI lint + OpenAPI clean; 97% coverage held.

Extends the AI slop advisory (#724) to optionally use the maintainer's own
frontier model (Anthropic/OpenAI) instead of free Workers AI, reusing the
existing per-repo encrypted key + aiReviewByok flag — one BYOK key serves
both AI features. Advisory-only either way; BYOK never changes whether the
slop pass can block (it can't).

- ai-review.ts: extract callAiProvider (shared BYOK HTTP/timeout/error
  handling) — runProviderReview now wraps it; export ProviderFailure +
  DEFAULT_BYOK_DAILY_REPO_LIMIT.
- repositories.ts: generalize countByokAiReviewEventsForRepoSince ->
  countByokAiEventsForRepoSince — one shared per-repo/day BYOK cap across
  ALL AI features (review + slop), so enabling more capabilities cannot
  multiply frontier spend.
- ai-slop.ts: providerKey path skips the free neuron budget, checks the
  shared BYOK cap, calls callAiProvider + parseSlopOpinion, records
  byok:<provider> usage. Workers-AI fallback unchanged. Fail-safe.
- processors runAiSlopForAdvisory: decrypt the repo key (a declared
  provider override must match) and pass it through.

Tests: BYOK provider path (provider used, Workers AI not) + shared BYOK cap
quota; existing Workers-AI gating/fail-safe still green. 97% coverage held.
@ghost

ghost commented Jun 14, 2026

Copy link
Copy Markdown

Note

Gittensory Gate skipped

PR closed before full evaluation. No late first comment was created.

Signal Result Evidence Action
Gate result ⚠️ Skipped #728 is no longer open. No action.

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

@ghost ghost added the gittensory:reviewed label Jun 14, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@ghost

ghost commented Jun 14, 2026

Copy link
Copy Markdown

gittensory · advisory review

Reviewed 5 changed file(s) — two independent AI reviewers.

Suggested action: 🛠️ Request changes. (reviewers split: request changes / merge) Address the suggestions below before merging.

Reviewer A · gpt-oss-120b — recommends 🛠️ request changes
The PR adds BYOK (maintainer‑paid) AI support to the slop advisory flow, shares counting logic with the review path, and updates related DB queries and tests. The intent is clear and the implementation largely follows the existing patterns.

Suggestions

  • Import RepositorySettings (and getDecryptedRepositoryAiKey if not already imported) at the top of src/queue/processors.ts to satisfy TypeScript.
  • Verify that no other file still imports the old countByokAiReviewEventsForRepoSince name; replace any leftover imports with the new countByokAiEventsForRepoSince.
  • Consider adding a unit test for the case where providerKey is supplied but the stored key’s provider mismatches, ensuring the fallback to Workers‑AI is exercised.

Worth double-checking

  • Missing imports could cause the build to fail, preventing deployment.
  • Changing the DB query to count all BYOK events removes the feature filter; ensure no downstream logic relied on that filter.

Reviewer B · nemotron-3-120b-a12b — recommends ✅ merge
The PR adds BYOK (Bring Your Own Key) support for the AI slop advisory feature, allowing maintainers to use their own frontier models for generating slop advisories while respecting a shared per-repo/day BYOK quota across all AI features. It refactors shared BYOK logic between ai-review and ai-slop services, updates repository functions to count BYOK events universally, and includes comprehensive tests. The changes maintain the public/private boundary by keeping advisory generation internal and only outputting sanitized findings.

Suggestions

  • Consider adding a comment in src/services/ai-slop.ts explaining why freeCalls is set to 0 for BYOK (to clarify it doesn't consume the free neuron budget).
  • In test/unit/ai-slop.test.ts, the new BYOK usage test could assert that the fetched model matches the stored key's model (if specified) for stronger validation.
  • Ensure the DEFAULT_BYOK_DAILY_REPO_LIMIT export in ai-review.ts is actually used elsewhere (it is, via env var fallback) - no action needed.

Worth double-checking

  • The shared BYOK daily repo limit means enabling both AI review and slop advisory will consume from the same quota - verify this aligns with product expectations (it appears intentional).
  • The BYOK key decryption path (getDecryptedRepositoryAiKey) relies on existing encryption infrastructure; confirm key rotation/secrets management is handled elsewhere (pre-existing).
  • Unit test for BYOK cap seeds a prior event but doesn't verify the exact count logic - though the test passes, adding an assertion on byokUsed would increase confidence.

@ghost ghost added the gittensory-review label Jun 14, 2026
@JSONbored
JSONbored merged commit d018a13 into main Jun 14, 2026
10 checks passed
@JSONbored
JSONbored deleted the feat/ai-slop-byok branch June 14, 2026 16:06
@github-project-automation github-project-automation Bot moved this from Todo to Done in gittensory - v1 roadmap Jun 14, 2026
JSONbored added a commit that referenced this pull request Jun 14, 2026
Completes the BYOK-only gating in this PR by closing the remaining cost
vector: the AI slop advisory was still invoking FREE Workers AI for any
PR author (the shared daily neuron budget), even though the AI review
path (runAiReviewForAdvisory) already no-ops for unconfirmed authors.

- runAiSlopForAdvisory now returns early when `!confirmedContributor`
  (matching the AI review path exactly): no AI spend — free OR BYOK — on
  a PR from an unconfirmed/untrusted author. The deterministic slop core
  still runs for everyone; only the paid AI layer is gated. Fail-closed:
  if the contributor can't be confirmed, no AI runs.
- The BYOK key lookup no longer needs its own `&& confirmedContributor`
  guard (the early return already guarantees it).

Adopts the full-gate approach from the sibling PR #727 (which also
spotted this vector) while preserving this branch's BYOK integration
(#728). Deliberately does NOT adopt #727's neuron-budget bump (1 -> 6):
the AI review path estimates by LOGICAL free-call count, not worst-case
retries, so a 6x reservation would be inconsistent and over-reserve,
causing premature quota_exceeded.
JSONbored added a commit that referenced this pull request Jun 14, 2026
…free + BYOK) (#737)

* fix: gate BYOK slop advisory by contributor

* fix(slop): fully gate the AI slop advisory on confirmed contributor

Completes the BYOK-only gating in this PR by closing the remaining cost
vector: the AI slop advisory was still invoking FREE Workers AI for any
PR author (the shared daily neuron budget), even though the AI review
path (runAiReviewForAdvisory) already no-ops for unconfirmed authors.

- runAiSlopForAdvisory now returns early when `!confirmedContributor`
  (matching the AI review path exactly): no AI spend — free OR BYOK — on
  a PR from an unconfirmed/untrusted author. The deterministic slop core
  still runs for everyone; only the paid AI layer is gated. Fail-closed:
  if the contributor can't be confirmed, no AI runs.
- The BYOK key lookup no longer needs its own `&& confirmedContributor`
  guard (the early return already guarantees it).

Adopts the full-gate approach from the sibling PR #727 (which also
spotted this vector) while preserving this branch's BYOK integration
(#728). Deliberately does NOT adopt #727's neuron-budget bump (1 -> 6):
the AI review path estimates by LOGICAL free-call count, not worst-case
retries, so a 6x reservation would be inconsistent and over-reserve,
causing premature quota_exceeded.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant