feat(scoring): wire issue-discovery/issue-spam scoring constants into engine - #835
Conversation
… gate into scoring engine (JSONbored#808) Co-Authored-By: YB0y <YB0y@users.noreply.github.com>
|
Note Gittensory Gate skippedPR closed before full evaluation. No late first comment was created.
💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers. |
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
This comment has been minimized.
This comment has been minimized.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #835 +/- ##
=======================================
Coverage 96.60% 96.60%
=======================================
Files 109 109
Lines 14914 14921 +7
Branches 5390 5394 +4
=======================================
+ Hits 14408 14415 +7
Misses 104 104
Partials 402 402 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
JSONbored
left a comment
There was a problem hiding this comment.
Good:
- Correctly models #808 constants in scoring defaults and score-preview math.
- Adds useful tests for test-token weighting and open-issue threshold behavior.
- CI/test/ui/validate pass.
Bad:
- Codecov patch/project fail.
- Source/generated contract is stale: no OpenAPI schema/public artifact update for new score-preview fields/blocker.
- New open-issue gate is not wired into scenario cleanup projections, so “best reasonable case” cannot clear it.
Change requests:
- Add OpenAPI/schema + generated artifact coverage for openIssueMultiplier/openIssueThreshold/openIssueCount/open_issue_threshold.
- Fix Codecov patch/project failure.
- Either project open-issue cleanup in scenario previews or explicitly document/test why it is intentionally current-state only.
JSONbored
left a comment
There was a problem hiding this comment.
PR #835 — feat(scoring): wire issue-discovery/issue-spam constants into engine [YB0y]
Action: CHANGES REQUESTED
Issue #808: PARTIALLY CI: pass
Good:
- Models TEST_FILE_CONTRIBUTION_WEIGHT + an open-issue spam gate mirroring the open-PR gate; scopes out
the 5 validator-enforced floors with comments; clears 9 unmodeled-drift warnings. Good tests, and the
scenario-cleanup projection IS wired (the maintainer's "not wired" note is addressed in this head).
Flagged:
- #808 asks to wire modeled constants into score preview AND decision-pack AND reward-risk — only the
score preview is wired here; decision-pack/reward-risk untouched. - OpenAPI/generated contract stale for the new openIssueMultiplier/openIssueThreshold/open_issue_threshold
fields.
🔍 Maintainer reviewReviewed 4 file(s) — ✅ all CI green. Suggested action: ✅ Safe to merge — both reviewers found no blocking issues. Reviewer A · Suggestions
Worth double-checking
Reviewer B · No blocking issues spotted. |
Summary
Ten scoring constants were parsed and stored in
ScoringModelSnapshotRecordbut never applied anywhere in the scoring engine — causing silent drift between upstream gittensor and gittensory's preview outputs.This PR resolves the drift by:
TEST_FILE_CONTRIBUTION_WEIGHTintocomputeScoreCore()so test-file tokens are weighted at0.05×when computingtotalTokenScorefrom components.OPEN_ISSUE_SPAM_BASE_THRESHOLD,OPEN_ISSUE_SPAM_TOKEN_SCORE_PER_SLOT,MAX_OPEN_ISSUE_THRESHOLD) as a new scoring multiplier that mirrors the existing open-PR spam gate.PR_LOOKBACK_DAYS,MIN_VALID_MERGED_PRS,MIN_VALID_SOLVED_ISSUES,MIN_ISSUE_CREDIBILITY,MIN_TOKEN_SCORE_FOR_VALID_ISSUE) with inline comments explaining they are validator-enforced, not per-PR/issue scoring inputs.open_issue_thresholdblocker inScoreGateBlocker,ScoreGateDelta,ScorePreviewResult, andPUBLIC_BLOCKER_TEXTfor public-safe scenario summaries.Related Issue
Closes: #808
Change Type
Real Behavior Proof
TEST_FILE_CONTRIBUTION_WEIGHTwired:Open-issue spam gate blocks at threshold:
Threshold scales with token history:
MAX_OPEN_ISSUE_THRESHOLDcap enforced:All 9 previously unmodeled constants no longer flagged:
Test Output
New tests (describe:
"issue-discovery scoring constants (#808)"):TEST_FILE_CONTRIBUTION_WEIGHTreduces test-file token contributionexistingContributorTokenScoreMAX_OPEN_ISSUE_THRESHOLDcaps the threshold at 30Checklist
npm run typecheckpasses (0 errors)npm run test:unitpasses (1959/1959)ScorePreviewResultshape (new fields are additive)openIssueCountinput is optional (openIssueCount?: number) — defaults to 0scenario-summary.tsfor safe external rendering