Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/00_overview/DASHBOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ _Top-level index across MVP1 → GA v1+ as of **2026-05-21**. Click a release na

| Release | Theme | Progress | Status |
|---|---|---|---|
| [MVP1 / v0.1](MVP1_DASHBOARD.md) | The Loop | 56 / 57 scoped done · 6 remaining | **In progress** |
| [MVP1 / v0.1](MVP1_DASHBOARD.md) | The Loop | 57 / 57 scoped done · 5 remaining | **In progress** |
| [MVP2 / v0.2](MVP2_DASHBOARD.md) | Observable | 1 / 1 scoped done · 1 remaining | **In progress** |
| MVP3 / v0.3 | Production Stacks | — | **Not yet scoped** |
| MVP4 / v0.4 | Multi-tenant, Multi-LLM | — | **Not yet scoped** |
Expand Down
27 changes: 10 additions & 17 deletions docs/00_overview/MVP1_DASHBOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,30 +6,24 @@ _Reflects feature-folder state as of **2026-05-21** (latest mtime of any planned

## Next up

**[feat_pr_metric_confidence](../02_product/planned_features/feat_pr_metric_confidence/feature_spec.md)** — Feature, currently in **Plan**
All scoped MVP1 features shipped 🎉

> Approvers reading a study-backed PR see a "## Confidence" section directly between the existing "## Metric delta" and "## Config diff" sections.

Plan approved; run /impl-execute to ship

```bash
/impl-execute docs/02_product/planned_features/feat_pr_metric_confidence/implementation_plan.md --all
```
Pull from the Idea backlog or capture a new feature spec.

## MVP1 Progress

| Metric | Value |
|---|---|
| Scoped items done | **56 / 57** (98%) — feat_/infra_/chore_/epic_ past idea stage |
| Path to MVP1 | **6** items remaining (features + bugs + chores) |
| Scoped items done | **57 / 57** (100%) — feat_/infra_/chore_/epic_ past idea stage |
| Path to MVP1 | **5** items remaining (features + bugs + chores) |
| Open bugs | 0 |
| Open chores | 5 (idea-stage debt) |
| Backlog ideas | 4 idea-only feat/infra (not yet scoped into MVP1) |
| In flight | 0 feature(s) actively shipping |

## Pipeline

### Done (68)
### Done (69)

| Feature | Type | One-liner | Depends on | Status |
|---|---|---|---|---|
Expand All @@ -45,6 +39,7 @@ Plan approved; run /impl-execute to ship
| [feat_github_webhook](implemented_features/2026_05_12_feat_github_webhook/feature_spec.md) | Feature | GitHub posts to `POST /webhooks/github` with HMAC-SHA256 signature; the receiver verifies the signature, looks up the proposal by `pr_url`, updates `pr_state` and `pr_merged_at`. | `infra_foundation` `infra_adapter_elastic` `feat_github_pr_worker` | [PR #56](https://github.com/SoundMindsAI/relyloop/pull/56) merged 2026-05-12 |
| [feat_judgments_periodic_resume_sweep](implemented_features/2026_05_14_feat_judgments_periodic_resume_sweep/feature_spec.md) | Feature | A new Arq cron job `resume_stuck_judgment_lists` ticks every `RELYLOOP_JUDGMENTS_RESUME_SWEEP_MINUTES` minutes (default 15), re-enqueues every `judgment_lists.status='generating'` row via deterministi | — | [PR #104](https://github.com/SoundMindsAI/relyloop/pull/104) merged 2026-05-12 |
| [feat_llm_judgments](implemented_features/2026_05_11_feat_llm_judgments/feature_spec.md) | Feature | A relevance engineer selects a query set + cluster + target + rubric and the system runs the current template to fetch top-K hits per query, asks OpenAI to rate each (query, doc) on a 0–3 scale with r | `infra_foundation` `infra_adapter_elastic` `feat_study_lifecycle` | [PR #35](https://github.com/SoundMindsAI/relyloop/pull/35) merged 2026-05-11 |
| [feat_pr_metric_confidence](implemented_features/2026_05_21_feat_pr_metric_confidence/feature_spec.md) | Feature | Approvers reading a study-backed PR see a "## Confidence" section directly between the existing "## Metric delta" and "## Config diff" sections. | — | [PR #180](https://github.com/SoundMindsAI/relyloop/pull/180) merged 2026-05-21 |
| [feat_proposals_ui](implemented_features/2026_05_12_feat_proposals_ui/feature_spec.md) | Feature | Two routes — `/proposals` (filterable list) and `/proposals/{id}` (config diff + metric delta + "Open PR" button + post-open PR-state mirror) — plug into the existing `feat_studies_ui` Next.js app. | `feat_studies_ui` `feat_digest_proposal` `feat_github_pr_worker` `feat_github_webhook` | [PR #58](https://github.com/SoundMindsAI/relyloop/pull/58) merged 2026-05-12 |
| [feat_query_inline_crud](implemented_features/2026_05_14_feat_query_inline_crud/feature_spec.md) | Feature | A relevance engineer on the `/query-sets/[id]` page sees a paginated table of every query in the set with `query_text`, `reference_answer`, `query_metadata`, and a `judgment_count` derived field. | `infra_foundation` `infra_adapter_elastic` `feat_study_lifecycle` `feat_llm_judgments` `feat_studies_ui` | [PR #101](https://github.com/SoundMindsAI/relyloop/pull/101) merged 2026-05-14 |
| [feat_studies_ui](implemented_features/2026_05_12_feat_studies_ui/feature_spec.md) | Feature | A Next.js app provides 9 of the 11 MVP1 routes from [`ui-architecture.md` §"Routes (MVP1)"](../01_architecture/ui-architecture.md): dashboard, clusters list/detail, query sets list/detail, judgment re | `infra_foundation` `feat_study_lifecycle` `feat_digest_proposal` `feat_llm_judgments` `infra_adapter_elastic` | [PR #50](https://github.com/SoundMindsAI/relyloop/pull/50) merged 2026-05-12 |
Expand Down Expand Up @@ -106,11 +101,9 @@ Plan approved; run /impl-execute to ship

_None._

### Plan (1)
### Plan (0)

| Feature | Type | One-liner | Depends on | Status |
|---|---|---|---|---|
| [feat_pr_metric_confidence](../02_product/planned_features/feat_pr_metric_confidence/feature_spec.md) | Feature | Approvers reading a study-backed PR see a "## Confidence" section directly between the existing "## Metric delta" and "## Config diff" sections. | — | [PR #41](https://github.com/SoundMindsAI/relyloop/pull/41) merged 2026-05-11 |
_None._

### Spec (0)

Expand Down Expand Up @@ -141,8 +134,6 @@ graph LR
classDef plan fill:#fef9c3,stroke:#854d0e,color:#854d0e;
classDef spec fill:#dbeafe,stroke:#1e40af,color:#1e40af;
classDef idea fill:#f1f5f9,stroke:#334155,color:#334155;
feat_pr_metric_confidence["pr metric confidence"]
class feat_pr_metric_confidence plan;
infra_foundation["foundation"]
class infra_foundation done;
feat_study_lifecycle["study lifecycle"]
Expand Down Expand Up @@ -253,6 +244,8 @@ graph LR
class chore_precommit_node_path_resolution done;
feat_agent_propose_search_space["agent propose search space"]
class feat_agent_propose_search_space done;
feat_pr_metric_confidence["pr metric confidence"]
class feat_pr_metric_confidence done;
infra_uv_sync_drops_precommit["uv sync drops precommit"]
class infra_uv_sync_drops_precommit done;
feat_study_lifecycle --> feat_digest_proposal
Expand Down
2 changes: 1 addition & 1 deletion docs/00_overview/dashboard.html
Original file line number Diff line number Diff line change
Expand Up @@ -371,7 +371,7 @@ <h2>Releases</h2>
<div class="roadmap-row">
<div class="release-name"><a href="mvp1_dashboard.html">MVP1 / v0.1</a></div>
<div class="theme">The Loop</div>
<div class="progress">56 / 57 scoped done · 6 remaining</div>
<div class="progress">57 / 57 scoped done · 5 remaining</div>
<span class="state-pill in_progress">In progress</span>
</div>

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Implementation Plan — PR Metric Confidence (Phase 1)

**Date:** 2026-05-21
**Status:** Draft (pending GPT-5.5 cross-model review)
**Status:** Complete (PR #180, merged 2026-05-21 as squash `d0a8358`)
**Primary spec:** [`feature_spec.md`](feature_spec.md)
**Policy source(s):** [`CLAUDE.md`](../../../../CLAUDE.md), [`docs/01_architecture/data-model.md`](../../../01_architecture/data-model.md), [`docs/01_architecture/api-conventions.md`](../../../01_architecture/api-conventions.md), [`docs/01_architecture/optimization.md`](../../../01_architecture/optimization.md)

Expand Down Expand Up @@ -1061,7 +1061,7 @@ None planned. The feature is purely additive across all surfaces.
- [x] Story 2.2 — `<ConfidencePanel>` component + glossary + page mount
- [x] Story 2.3 — Playwright E2E
- [x] **Epic 2 gate**
- [ ] Final state.md + architecture.md update
- [x] Final state.md + architecture.md update

### Blocked items

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,14 @@
- Next: User approval, then `/impl-execute` on this branch.

## Implementation
- Status: Not started
- Status: Complete (PR #180, merged 2026-05-21 as squash `d0a8358`)
- Stories shipped: 9 of 9 across 2 epics
- CI: both `pr` + `secrets-defense` workflows green on the final commit
- Reviews:
- Gemini Code Assist: clean pass (no line-level comments, no review findings)
- GPT-5.5 cycle 1 (Epic 1 gate): 12 findings — 5 rejected with cited counter-evidence (truncated-diff false positives), 2 deferred (plan/code interface drift; full-worker integration test), 5 accepted + fixed inline in `0814700`
- GPT-5.5 final review (cumulative `main..HEAD`): 3 Low findings — all accepted + fixed inline in `6c34a5b`
- Adjudication summary posted at https://github.com/SoundMindsAI/relyloop/pull/180

## Branch context
- Working on: `feat_pr_metric_confidence` (branch created at spec-gen start)
- Carries: bug_e2e_target_dropdown_flake folder rename (uncommitted, from earlier preflight) + feat_pr_metric_confidence/idea.md preflight patch + feature_spec.md (new) + phase2_idea.md (new) + this file (new)
- Shipped via PR #180 (squash `d0a8358`); finalization docs branch is `docs/finalize-pr-metric-confidence`
57 changes: 29 additions & 28 deletions docs/00_overview/mvp1_dashboard.html
Original file line number Diff line number Diff line change
Expand Up @@ -369,28 +369,28 @@ <h1>RelyLoop MVP1 Dashboard</h1>
<main>

<section>
<div class="next-up">
<div class="eyebrow">Next up — Feature, currently in <strong>Plan</strong></div>
<div class="title"><a href="../../docs/02_product/planned_features/feat_pr_metric_confidence/feature_spec.md">Pr Metric Confidence</a></div>
<div class="one-liner">Approvers reading a study-backed PR see a &quot;## Confidence&quot; section directly between the existing &quot;## Metric delta&quot; and &quot;## Config diff&quot; sections.</div>
<div class="stage-hint">Plan approved; run /impl-execute to ship</div>
<code class="cmd">/impl-execute docs/02_product/planned_features/feat_pr_metric_confidence/implementation_plan.md --all</code>
<div class="next-up done">
<div class="eyebrow">Next up</div>
<div class="title">All scoped MVP1 features shipped 🎉</div>
<div class="one-liner">
Pull from the Idea backlog or capture a new feature spec.
</div>
</div>
</section>


<section>
<h2>MVP1 Progress</h2>
<div class="kpi-row">
<div class="kpi ">
<div class="kpi complete">
<div class="label">Scoped items done</div>
<div class="value">56 / 57</div>
<div class="sub">98% of feat_/infra_/chore_/epic_ items past idea stage</div>
<div class="bar"><span style="width:98%"></span></div>
<div class="value">57 / 57</div>
<div class="sub">100% of feat_/infra_/chore_/epic_ items past idea stage</div>
<div class="bar"><span style="width:100%"></span></div>
</div>
<div class="kpi warn">
<div class="label">Path to MVP1</div>
<div class="value">6</div>
<div class="value">5</div>
<div class="sub">items left = features + bugs + chores</div>
</div>
<div class="kpi ">
Expand Down Expand Up @@ -545,18 +545,7 @@ <h3>Spec <span class="count">0</span></h3>
</div>

<div class="col plan">
<h3>Plan <span class="count">1</span></h3>

<div class="card feat" data-prefix="feat">
<div class="name"><a href="../../docs/02_product/planned_features/feat_pr_metric_confidence/feature_spec.md">Pr Metric Confidence</a></div>
<div class="meta">
<span class="badge feat">Feature</span>
<a class="pr" href="https://github.com/SoundMindsAI/relyloop/pull/41">PR #41</a><span>merged 2026-05-11</span>
</div>
<div class="one-liner">Approvers reading a study-backed PR see a &quot;## Confidence&quot; section directly between the existing &quot;## Metric delta&quot; and &quot;## Config diff&quot; sections.</div>
<div class="deferred">deferred: Phase 2</div>

</div>
<h3>Plan <span class="count">0</span></h3>

</div>

Expand All @@ -566,7 +555,7 @@ <h3>Implementing <span class="count">0</span></h3>
</div>

<div class="col done">
<h3>Done <span class="count">68</span></h3>
<h3>Done <span class="count">69</span></h3>

<div class="card feat" data-prefix="feat">
<div class="name"><a href="../../docs/00_overview/implemented_features/2026_05_21_feat_agent_propose_search_space/feature_spec.md">Agent Propose Search Space</a></div>
Expand Down Expand Up @@ -712,6 +701,18 @@ <h3>Done <span class="count">68</span></h3>
</div>


<div class="card feat" data-prefix="feat">
<div class="name"><a href="../../docs/00_overview/implemented_features/2026_05_21_feat_pr_metric_confidence/feature_spec.md">Pr Metric Confidence</a></div>
<div class="meta">
<span class="badge feat">Feature</span>
<a class="pr" href="https://github.com/SoundMindsAI/relyloop/pull/180">PR #180</a><span>merged 2026-05-21</span>
</div>
<div class="one-liner">Approvers reading a study-backed PR see a &quot;## Confidence&quot; section directly between the existing &quot;## Metric delta&quot; and &quot;## Config diff&quot; sections.</div>


</div>


<div class="card feat" data-prefix="feat">
<div class="name"><a href="../../docs/00_overview/implemented_features/2026_05_12_feat_proposals_ui/feature_spec.md">Proposals Ui</a></div>
<div class="meta">
Expand Down Expand Up @@ -1396,8 +1397,6 @@ <h2>Dependency graph (feat_ + infra_)</h2>
classDef plan fill:#fef9c3,stroke:#854d0e,color:#854d0e;
classDef spec fill:#dbeafe,stroke:#1e40af,color:#1e40af;
classDef idea fill:#f1f5f9,stroke:#334155,color:#334155;
feat_pr_metric_confidence[&quot;pr metric confidence&quot;]
class feat_pr_metric_confidence plan;
infra_foundation[&quot;foundation&quot;]
class infra_foundation done;
feat_study_lifecycle[&quot;study lifecycle&quot;]
Expand Down Expand Up @@ -1508,6 +1507,8 @@ <h2>Dependency graph (feat_ + infra_)</h2>
class chore_precommit_node_path_resolution done;
feat_agent_propose_search_space[&quot;agent propose search space&quot;]
class feat_agent_propose_search_space done;
feat_pr_metric_confidence[&quot;pr metric confidence&quot;]
class feat_pr_metric_confidence done;
infra_uv_sync_drops_precommit[&quot;uv sync drops precommit&quot;]
class infra_uv_sync_drops_precommit done;
feat_study_lifecycle --&gt; feat_digest_proposal
Expand Down Expand Up @@ -1601,8 +1602,6 @@ <h2>Dependency graph (feat_ + infra_)</h2>
classDef plan fill:#fef9c3,stroke:#854d0e,color:#854d0e;
classDef spec fill:#dbeafe,stroke:#1e40af,color:#1e40af;
classDef idea fill:#f1f5f9,stroke:#334155,color:#334155;
feat_pr_metric_confidence[&quot;pr metric confidence&quot;]
class feat_pr_metric_confidence plan;
infra_foundation[&quot;foundation&quot;]
class infra_foundation done;
feat_study_lifecycle[&quot;study lifecycle&quot;]
Expand Down Expand Up @@ -1713,6 +1712,8 @@ <h2>Dependency graph (feat_ + infra_)</h2>
class chore_precommit_node_path_resolution done;
feat_agent_propose_search_space[&quot;agent propose search space&quot;]
class feat_agent_propose_search_space done;
feat_pr_metric_confidence[&quot;pr metric confidence&quot;]
class feat_pr_metric_confidence done;
infra_uv_sync_drops_precommit[&quot;uv sync drops precommit&quot;]
class infra_uv_sync_drops_precommit done;
feat_study_lifecycle --&gt; feat_digest_proposal
Expand Down
Loading