Skip to content

feature(review): surface computeRuleGateEval's per-project rows on the operator dashboard - #8928

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
oktofeesh1:claude/issue-8906-rule-gate-eval-per-project
Jul 26, 2026
Merged

feature(review): surface computeRuleGateEval's per-project rows on the operator dashboard#8928
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
oktofeesh1:claude/issue-8906-rule-gate-eval-per-project

Conversation

@oktofeesh1

Copy link
Copy Markdown
Contributor

Summary

  • `computeBlendedRuleGateEval` (pooled by `ruleCode` alone) is fully wired into the operator dashboard and the auto-tune breaker. Its finer-grained sibling `computeRuleGateEval`, keyed by `(project, ruleCode)`, had zero consumers outside its own test.
  • Adds `projectRulesBelowClosePrecisionFloor` — the per-project counterpart to the existing `rulesBelowClosePrecisionFloor` — using the exact same floor/minDecided check, over `computeRuleGateEval`'s rows instead of the blended ones.
  • Wires `computeRuleGateEval` into `buildOperatorDashboardPayload` as a new "Rules below close-precision floor (per-project)" tile, following the exact same fetch → fold → tile pattern already used for the blended view (and for `contributor-gate-eval.ts`'s own per-project/blended pair).

Fixes #8906.

Test plan

  • New unit tests for `projectRulesBelowClosePrecisionFloor` in `test/unit/rule-gate-eval.test.ts`, mirroring the existing `rulesBelowClosePrecisionFloor` suite (below-floor-flagged, insufficient-sample, healthy, null-precision, at-floor-boundary), plus a dedicated test proving it isolates a rule broken on one project from a second project's healthy rows.
  • New end-to-end test in `test/unit/operator-dashboard.test.ts` seeding real `review_audit` rows across two repos for the same rule code — one repo genuinely broken (0% precision), one healthy — asserting the blended tile does NOT flag it (pooled precision ≈0.88, above the 0.85 floor) while the new per-project tile DOES flag the specific broken repo, proving the finer-grained view surfaces something the already-wired blended view misses.
  • `npx tsc --noEmit` — 0 errors
  • `npx vitest run test/unit/rule-gate-eval.test.ts test/unit/operator-dashboard.test.ts` — 48/48 passing
  • `npm run test:ci` (full local gate) — green
  • `npm audit --audit-level=moderate` — pre-existing dev-only eslint/minimatch advisory only, unrelated to this change (no lockfile diff)

…e operator dashboard

computeBlendedRuleGateEval (pooled by ruleCode alone) is fully wired
into the operator dashboard and the auto-tune breaker; its
finer-grained sibling computeRuleGateEval, keyed by (project,
ruleCode), had zero consumers outside its own test.

Adds projectRulesBelowClosePrecisionFloor -- the per-project
counterpart to the existing rulesBelowClosePrecisionFloor -- and
wires computeRuleGateEval into buildOperatorDashboardPayload as a new
"Rules below close-precision floor (per-project)" tile, so a rule
broken on one specific repo is now visible even when a second repo's
healthy usage pools the blended figure back above the floor.

Refs JSONbored#8906.
@oktofeesh1
oktofeesh1 requested a review from JSONbored as a code owner July 26, 2026 13:41
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.60%. Comparing base (898cdcb) to head (a7faaf3).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8928      +/-   ##
==========================================
+ Coverage   90.56%   90.60%   +0.04%     
==========================================
  Files          96       98       +2     
  Lines       22490    22612     +122     
  Branches     3884     3926      +42     
==========================================
+ Hits        20367    20487     +120     
  Misses       1945     1945              
- Partials      178      180       +2     
Flag Coverage Δ
backend 98.36% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/review/rule-gate-eval.ts 97.40% <100.00%> (ø)
src/services/operator-dashboard.ts 100.00% <100.00%> (ø)

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 26, 2026
@loopover-orb

loopover-orb Bot commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Tip

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-26 13:50:24 UTC

4 files · 1 AI reviewer · no blockers · readiness 77/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This PR wires the already-existing but unused computeRuleGateEval into the operator dashboard as a new per-(project, ruleCode) tile, adding projectRulesBelowClosePrecisionFloor as a straight structural copy of the existing rulesBelowClosePrecisionFloor over RuleGateEvalRow instead of BlendedRuleGateEvalRow. The wiring in operator-dashboard.ts follows the exact same fetch->fold->tile pattern already used for the blended view, and the e2e test genuinely demonstrates the value proposition (seeds a rule broken on one repo but pooled healthy blended, asserting the blended tile misses it while the new per-project tile catches it). This is a narrowly-scoped, well-tested, correctly-wired feature closing a linked issue (#8906).

Nits — 3 non-blocking
  • src/review/rule-gate-eval.ts:261-274 — projectRulesBelowClosePrecisionFloor's body is byte-for-byte identical to rulesBelowClosePrecisionFloor's (only the row type differs); since RuleGateEvalRow and BlendedRuleGateEvalRow share the same wouldClose/weightedClosePrecision fields, consider a single generic helper (`function belowClosePrecisionFloor<T extends {wouldClose: number; weightedClosePrecision: number | null}>(rows: readonly T[], floor = ..., minDecided = ...): T[]`) to avoid maintaining two copies of the same filter logic.
  • The external brief's 'magic number 8906' flag is just the issue reference embedded in a code comment, not an actual unexplained numeric literal — not a real finding.
  • Consider the shared generic filter helper noted above in rule-gate-eval.ts to keep the two floor checks from drifting apart if the floor logic ever changes.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #8906
Related work ⚠️ 1 scoped overlap Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 44 registered-repo PR(s), 36 merged, 0 issue(s).
Contributor context ✅ Confirmed Gittensor contributor oktofeesh1; Gittensor profile; 44 PR(s), 0 issue(s).
Improvement ✅ Minor risk: clean · value: minor
Linked issue satisfaction

Addressed
The PR wires computeRuleGateEval into buildOperatorDashboardPayload via a new projectRulesBelowClosePrecisionFloor helper and exposes it as a new 'Rules below close-precision floor (per-project)' dashboard metric, matching the existing blended-view pattern as requested.

Review context
  • Author: oktofeesh1
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: JavaScript, MDX, Python, TypeScript
  • Official Gittensor activity: 44 PR(s), 0 issue(s).
  • Related work: Titles/paths share 6 meaningful terms. (PR #8929)
Contributor next steps
  • Start here: Review top overlaps.
  • Then work through the remaining 3 steps in the Signals table above.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

Decision record
  • action: merge · clause: success
  • config: 03a7f8b529a9 · pack: oss-anti-slop
  • record: e261c364c8f4 (schema v2, head a7faaf3)

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoopOver approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit f4cf7cc into JSONbored:main Jul 26, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feature(review): surface the per-(project, ruleCode) gate-accuracy breakdown computeRuleGateEval already computes

1 participant