Skip to content

Epic: AMS + ORB combined-use synergy features (both installed on the same repo) #6206

Description

@JSONbored

Problem

ORB (self-hosted PR review) and AMS (the autonomous miner) are architected as fully independent products — confirmed via this session's research: ORB reviews any PR regardless of who authored it, and AMS's self-review-context.js reconstructs gate predictions from a target repo's own .loopover.yml without needing ORB actually installed there. That independence is correct and should be preserved; neither product should ever require the other.

But today, when a repo owner runs both on the same repo, the only thing that actually connects them is observability — AMS's SQLite ledgers get exported into ORB's Grafana panels in fleet mode (docs.self-hosting-unified-ams-orb.tsx). There is no product-feature-level synergy. Two concrete gaps found during this session's research:

  1. Two disconnected reputation systems. ORB's src/review/submitter-reputation.ts derives a private per-submitter quality signal purely from ORB's own review history for that repo. AMS has its own, completely separate calibration/track-record store (prediction-ledger.js, calibration.js, track-record-summary.ts). A PR from a contributor with a long, high-quality AMS track record is invisible to ORB's reputation gate — and vice versa.
  2. AMS's gate-prediction always uses the static fallback, even when ORB is actually live. self-review-context.js's GitHub-only reconstruction is the correct standalone fallback, but when ORB genuinely is installed and running on the target repo, AMS could instead query ORB's live, self-tuned gate config (the same data src/review/auto-apply.ts's self-tune loop already produces) for a materially more accurate prediction than reconstructing from the static .loopover.yml file.

A third, lower-risk opportunity: a maintainer running ORB whose repo attracts AMS contributors currently has no visibility into that — "what fraction of my merged PRs come from AMS miners, and how do they perform" is a real, answerable question with data ORB already has (GitHub logins on every PR record) that nobody has built a view for.

Area

Cross-cutting: src/review/submitter-reputation.ts (ORB), AMS's calibration stack (packages/loopover-engine/src/*calibration*.ts, packages/loopover-miner/lib/prediction-ledger.js), src/review/auto-apply.ts (ORB self-tune), and a new maintainer-dashboard analytics surface (apps/loopover-ui).

Proposal

Three sub-specs, each requiring its own design pass before any contributor-facing work is carved out — none of these should be unlocked for contributors until a maintainer has decided the actual mechanism (see each sub-issue for the specific open questions):

  1. Bridge ORB's submitter-reputation signal with AMS's calibration/track-record data, so a repo running both products treats a known-good AMS contributor more favorably than an anonymous first-time submitter, without ever making the signal public (matches submitter-reputation.ts's existing "STRICTLY INTERNAL" contract).
  2. Upgrade AMS's predicted-gate to prefer ORB's live self-tuned config over the static-file reconstruction when both are installed on the same repo, while keeping the static-file path as the correct standalone fallback.
  3. A maintainer-facing "contributor mix" analytics view showing what fraction of activity on a repo comes from identifiable AMS miners and how they perform, built on data ORB already has.

Deliverables

  • Sub-spec: reputation bridge (see linked sub-issue).
  • Sub-spec: live-gate-prediction upgrade (see linked sub-issue).
  • Sub-spec: contributor-mix analytics (see linked sub-issue).

Resources

  • src/review/submitter-reputation.ts, src/review/auto-apply.ts
  • packages/loopover-miner/lib/self-review-context.js, prediction-ledger.js, calibration.js
  • apps/loopover-ui/src/routes/docs.self-hosting-unified-ams-orb.tsx (today's only real synergy: observability)

Boundaries

  • Neither product may ever require the other to function — every synergy feature here is additive-only, active exclusively when both happen to be present on the same repo, and must degrade to today's fully-independent behavior otherwise.
  • No public exposure of any reputation/calibration signal — matches submitter-reputation.ts's existing strict-internal contract.
  • None of the three sub-specs should be unlocked as contributor-open work until the maintainer has made the actual design decision each one raises.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.roadmapOn the Wave-2 agent-layer roadmap board (project 9)

Projects

Status
Done
Status
Done

Relationships

None yet

Development

No branches or pull requests

Issue actions