JSONbored · JSONbored · Jul 29, 2026 · Jul 29, 2026
@@ -11,16 +11,27 @@ website only build trust if a skeptic can check them without asking anyone's per
 page is the end-to-end walkthrough: export the same corpus snapshot the numbers come from, verify
 its checksum, replay the same scorer over it, and compare what you get against what is published.
 
-Everything below runs read-only against a database export and pure functions from
-`@loopover/engine`. Nothing posts anywhere, nothing needs an API key.
+Everything below runs read-only against a corpus export and pure functions from `@loopover/engine`.
+Nothing posts anywhere, and nothing needs a LoopOver API key.
+
+<Callout variant="warn">
+  **Step 1 is not yet runnable by a stranger.** Exporting the corpus reads the deployment's own
+  database — with `--remote` that means the operator's Cloudflare D1, which needs *their* Cloudflare
+  credentials, not a LoopOver key. So today steps 1–3 are reproducible by an operator or by anyone
+  running a self-host deployment against their own data, and step 4 is the part a third party can
+  check unauthenticated. No anonymous corpus download exists yet; until one does, this page marks
+  each step with who can actually run it rather than implying everyone can run all of them.
+</Callout>
 
 For the wider contract — every claim, its artifact, and its trust assumption, including the ones you
 cannot check — see [what you can verify](/docs/what-you-can-verify).
 
-## 1. Export the corpus snapshot
+## 1. Export the corpus snapshot *(operator / self-host)*
 
 Every rule's fired/override history exports as a versioned, checksummed JSON snapshot
-([backtest &amp; calibration](/docs/backtest-calibration) explains how that history is recorded):
+([backtest &amp; calibration](/docs/backtest-calibration) explains how that history is recorded).
+`--remote` shells out to `wrangler d1 execute … --remote`, so it reads the deployed database and
+requires that deployment's own Cloudflare credentials:
 
 ```bash
 npx tsx scripts/backtest-corpus-export.ts --rule-id linked_issue_scope_mismatch --output corpus.json --remote
@@ -34,10 +45,12 @@ npx tsx scripts/backtest-corpus-export.ts --rule-id linked_issue_scope_mismatch
 
 The snapshot's `checksum` field is a SHA-256 over the canonicalized cases (keys sorted, so
 property order can never change the hash). The fairness report's *reproducibility freeze point*
-shows the checksum of the corpus behind the latest persisted backtest run — an export of the same
-window reproduces the same checksum, byte for byte.
+shows that checksum for the **most recent persisted backtest run**, whichever single rule that run
+covered — re-exporting that rule over the same window reproduces the same checksum, byte for byte.
+It is one run's freeze point, not a per-rule commitment for every rule on the report, so compare it
+against an export of the rule the run actually covered.
 
-## 2. Verify the checksum
+## 2. Verify the checksum *(anyone, given a snapshot)*
 
 The manifest is self-verifying: recompute the hash over its own `cases` array and compare it to
 the recorded `checksum`. The canonicalization lives in `scripts/backtest-corpus-export-core.ts`
@@ -53,7 +66,7 @@ console.log(recomputed.checksum === saved.checksum ? "checksum OK" : "CHECKSUM M
 '
 ```
 
-## 3. Replay the scorer
+## 3. Replay the scorer *(anyone, given a snapshot)*
 
 The published precision comes from the same pure functions any Node script can import:
 `scoreBacktest` replays a classifier over the labeled cases, and `compareBacktestScores` applies
@@ -75,19 +88,42 @@ console.log(report);
 - **`null` is never `0`.** Precision and recall stay `null` below the decided-sample floor;
   the fairness report renders that as *insufficient data*, never as a zero.
 
-## 4. Compare against the published numbers
+## 4. Compare against the published numbers *(anyone)*
+
+The [fairness report](/fairness) renders each rule's decided-case count and measured precision from
+the public stats endpoint's `rulePrecision` block. That endpoint is served by the API host, not by
+this site, so fetch it absolutely — a bare `/v1/public/stats` resolves against `loopover.ai` and
+404s:
+
+```bash
+curl -s "https://api.loopover.ai/v1/public/stats" | jq '.rulePrecision'
+```
+
+The same per-rule numbers are also published as digest-committed
+[EvalScoreRecords](/docs/what-you-can-verify), each independently re-derivable without trusting the
+transport — recompute `recordDigest` over the record's own remaining fields and compare:
+
+```bash
+curl -s "https://api.loopover.ai/v1/public/eval-scores" | jq '.records'
+```
+
+That array is empty whenever the latest backtest run's corpus is empty. A checksum over zero cases
+is byte-identical for every rule and every window, so it commits to nothing a reader could re-derive
+the scores from — publishing a record against it would assert a reproducibility that does not exist.
+An empty `records` array therefore means *the numbers are not currently committed to a corpus*, never
+that the numbers are zero.
 
-The [fairness report](/fairness) renders each rule's decided-case count and measured precision
-from the public stats endpoint (`/v1/public/stats`, the `rulePrecision` block). The aggregated
-run history is also readable directly:
+The aggregated run history is readable directly too, again against the deployment's own database
+(operator / self-host):
 
 ```bash
 npx tsx scripts/backtest-track-record.ts --db loopover --remote
 ```
 
 Your replayed `confirmed / decided` for a rule should match the published precision for the same
-window; the freeze-point checksum ties the published numbers to the exact corpus you just
-verified.
+window. The freeze-point checksum ties the published numbers to the corpus you just verified only
+for the rule the latest run actually backtested — for any other rule it is a timestamped pointer to
+a different run, not a commitment to that rule's own cases.
 
 ## 5. Verify an attested run (when a run carries one)
 

@@ -176,7 +176,10 @@ replayable history — not hand-entered numbers.
 
 This is the walkthrough in [Verify this review](/docs/verify-this-review): export the checksummed
 corpus, verify its checksum, re-run the same public scoring functions from `@loopover/engine`, and
-compare. **Anyone** can do this for public repositories.
+compare. The checksum verification and the scorer replay are pure functions **anyone** can run over
+a snapshot they hold, and the published numbers themselves are fetchable unauthenticated — but the
+export step that produces the snapshot reads the deployment's own database and needs that
+deployment's credentials, so it is an operator / self-host step today rather than an anonymous one.
 
 <Callout variant="warn">
   **Private repositories are the exception.** A hosted tenant's review history cannot be published

@@ -4,7 +4,7 @@
 // It adds no new scoring and no new trust -- the numbers are the same ones `/v1/public/stats` already
 // publishes, just committed to a corpus checksum and made independently re-derivable per-record.
 import { canonicalJson, contentDigest } from "./decision-record";
-import type { PublicRulePrecision } from "./public-rule-precision";
+import { EMPTY_CORPUS_CHECKSUM, type PublicRulePrecision } from "./public-rule-precision";
 
 export const EVAL_SCORE_RECORD_SCHEMA_VERSION = 1 as const;
 
@@ -76,6 +76,14 @@ async function finalizeRecord(input: EvalScoreRecordDigestInput): Promise<EvalSc
  * a record whose commitments cannot be independently re-derived (no corpus checksum to point at) is not
  * publishable, so this deliberately emits nothing rather than a record with a placeholder commitment.
  *
+ * Also returns an empty array when the run's checksum is {@link EMPTY_CORPUS_CHECKSUM}. A hash over zero
+ * cases is the same 32 bytes for every rule, every window, and every deployment, so it points at nothing a
+ * consumer could re-derive the scores from -- it is a placeholder commitment wearing a real hash's clothes,
+ * and pairing it with a `reproducible` trust tier claims a reproducibility the artifact cannot support. The
+ * scores themselves come from a different dataset (live human-override events) and are unaffected by whether
+ * a corpus was exported, so an empty corpus never means the numbers are zero -- it means they are
+ * uncommitted, which is exactly the state #9215 says must not be published.
+ *
  * `recall` and `abstained` do not apply to this work-unit kind: ORB's gate rules fire deterministically (no
  * agent choosing to abstain) and this data measures precision, not a false-negative rate. `recall` is
  * `null` (genuinely inapplicable, never a misleading `0`); `abstained` is `0` (there is no abstention
@@ -85,6 +93,7 @@ async function finalizeRecord(input: EvalScoreRecordDigestInput): Promise<EvalSc
 export async function buildEvalScoreRecordsFromRulePrecision(precision: PublicRulePrecision, issuedAt: string): Promise<EvalScoreRecord[]> {
   if (!precision.latestBacktestRun) return [];
   const { corpusChecksum } = precision.latestBacktestRun;
+  if (corpusChecksum === EMPTY_CORPUS_CHECKSUM) return [];
   const windowStart = new Date(Date.parse(issuedAt) - precision.windowDays * 24 * 60 * 60 * 1000).toISOString();
 
   const records = await Promise.all(
@@ -145,3 +154,4 @@ export async function verifyEvalScoreRecordDigest(record: EvalScoreRecord): Prom
 // Re-exported so callers that only import this module never need a second import from decision-record.ts
 // just to canonicalize something alongside a record (e.g. logging, or a future signed-bundle wrapper).
 export { canonicalJson, contentDigest };
+export { EMPTY_CORPUS_CHECKSUM };
@@ -23,6 +23,15 @@ export const PUBLIC_PRECISION_MIN_DECIDED = 10;
 // duplication rule-calibration-trend.ts documents for its identical queries.
 const HUMAN_OVERRIDE_EVENT_TYPE_PREFIX = "signal.human_override:";
 
+/** `checksumCases([])` — SHA-256 over the canonicalized empty case list (the two-byte string `"[]"`), i.e. what
+ *  a corpus export produces for a rule with no labeled cases at all. A hash over zero cases is the same 32
+ *  bytes for every rule, every window and every deployment, so it is not the "independently-verifiable freeze
+ *  point" {@link PublicRulePrecision.latestBacktestRun} claims to be — it points at nothing a skeptic could
+ *  re-derive anything from. Hard-coded because the canonicalization that produces it
+ *  (`scripts/backtest-corpus-export-core.ts`) runs on `node:crypto` and is unimportable from the Workers
+ *  runtime; this module's own test re-derives it from `sha256Hex("[]")` so it can never drift. */
+export const EMPTY_CORPUS_CHECKSUM = "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945";
+
 export type PublicRulePrecisionRow = {
   ruleId: string;
   decided: number;
@@ -40,7 +49,9 @@ export type PublicRulePrecision = {
   /** All three reversal shapes counted over the window — the "counted against ourselves" number. */
   reversals: { reopened: number; reverted: number; superseded: number };
   /** The latest persisted backtest run carrying a corpus checksum — the independently-verifiable freeze
-   *  point — or null when no run has been recorded yet. */
+   *  point — or null when no run has been recorded yet, or when the latest run's corpus was empty (see
+   *  {@link EMPTY_CORPUS_CHECKSUM}: a commitment to nothing is not a freeze point, so it is reported as
+   *  absent rather than published as though it were verifiable). */
   latestBacktestRun: { corpusChecksum: string; at: string } | null;
 };
 
@@ -102,6 +113,9 @@ export async function loadPublicRulePrecision(env: Env, nowMs: number = Date.now
       reverted: reversalCount("reversal_reverted"),
       superseded: reversalCount("reversal_superseded"),
     },
-    latestBacktestRun: latest && typeof latest.checksum === "string" && latest.checksum !== "" ? { corpusChecksum: latest.checksum, at: latest.created_at } : null,
+    latestBacktestRun:
+      latest && typeof latest.checksum === "string" && latest.checksum !== "" && latest.checksum !== EMPTY_CORPUS_CHECKSUM
+        ? { corpusChecksum: latest.checksum, at: latest.created_at }
+        : null,
   };
 }
@@ -6,9 +6,10 @@ import {
   ORB_GATE_SUBJECT_ID,
   OUTCOME_CONFIRMED_PRECISION_SCORING_RULE_VERSION,
   verifyEvalScoreRecordDigest,
+  EMPTY_CORPUS_CHECKSUM,
   type EvalScoreRecord,
 } from "../../src/review/eval-score-records";
-import { contentDigest } from "../../src/review/decision-record";
+import { contentDigest, sha256Hex } from "../../src/review/decision-record";
 import type { PublicRulePrecision } from "../../src/review/public-rule-precision";
 
 const ISSUED_AT = "2026-07-27T12:00:00.000Z";
@@ -29,6 +30,25 @@ describe("buildEvalScoreRecordsFromRulePrecision (#9266)", () => {
     expect(records).toEqual([]);
   });
 
+  it("refuses to publish records whose freeze point commits to an empty corpus", async () => {
+    // Regression: production published decided=460/confirmed=287 alongside sha256("[]") -- a hash that is
+    // byte-identical for every rule and every window, so it committed to nothing a consumer could re-derive.
+    const records = await buildEvalScoreRecordsFromRulePrecision(
+      { ...PRECISION_WITH_FREEZE_POINT, latestBacktestRun: { corpusChecksum: EMPTY_CORPUS_CHECKSUM, at: "2026-07-27T10:00:00.000Z" } },
+      ISSUED_AT,
+    );
+    expect(records).toEqual([]);
+  });
+
+  it("EMPTY_CORPUS_CHECKSUM is the exporter's own checksum over zero cases", async () => {
+    // scripts/backtest-corpus-export-core.ts hashes `JSON.stringify(cases.map(canonicalizeCase))`, which for
+    // an empty list is the two-byte string "[]" -- re-derived here so the hard-coded constant cannot drift
+    // from the exporter that produces the value it guards against. (That exporter's canonicalization and
+    // canonicalJson coincide ONLY on the empty case, so this hashes the literal preimage rather than
+    // round-tripping [] through either one.)
+    expect(EMPTY_CORPUS_CHECKSUM).toBe(await sha256Hex("[]"));
+  });
+
   it("builds one record per rule, committed to the freeze point's corpus checksum", async () => {
     const records = await buildEvalScoreRecordsFromRulePrecision(PRECISION_WITH_FREEZE_POINT, ISSUED_AT);
     expect(records).toHaveLength(2);

@@ -3,7 +3,9 @@ import {
   loadPublicRulePrecision,
   PUBLIC_PRECISION_MIN_DECIDED,
   PUBLIC_PRECISION_WINDOW_DAYS,
+  EMPTY_CORPUS_CHECKSUM,
 } from "../../src/review/public-rule-precision";
+import { sha256Hex } from "../../src/review/decision-record";
 import { recordAuditEvent } from "../../src/db/repositories";
 import { createSignalStore } from "../../src/review/signal-tracking-wire";
 import { createTestEnv } from "../helpers/d1";
@@ -95,6 +97,33 @@ describe("loadPublicRulePrecision (#8230)", () => {
     expect(block.latestBacktestRun).toEqual({ corpusChecksum: "newest111", at: new Date(NOW - 30_000).toISOString() });
   });
 
+  it("reports no freeze point when the latest run's corpus was empty", async () => {
+    // Regression: production's latest run carried sha256("[]") -- the checksum of ZERO cases -- and the
+    // fairness page rendered it as a "reproducibility freeze point" while /v1/public/eval-scores published
+    // records committed to it. A hash over no cases is identical everywhere, so it verifies nothing.
+    const env = createTestEnv();
+    await seedVerdicts(env, "ai_consensus_defect", 15, 5);
+    await recordAuditEvent(env, {
+      eventType: "calibration.logic_backtest_run",
+      targetKey: "rule",
+      outcome: "completed",
+      metadata: { corpusChecksum: EMPTY_CORPUS_CHECKSUM, comparison: {} },
+      createdAt: new Date(NOW - 1000).toISOString(),
+    });
+
+    const block = await loadPublicRulePrecision(env, NOW);
+    expect(block.latestBacktestRun).toBeNull();
+    // The scores come from a different dataset (human-override events) and are unaffected -- an empty corpus
+    // means the numbers are uncommitted, never that they are zero.
+    expect(block.rules).toEqual([{ ruleId: "ai_consensus_defect", decided: 20, confirmed: 15, precision: 0.75 }]);
+  });
+
+  it("EMPTY_CORPUS_CHECKSUM is the exporter's own checksum over zero cases", async () => {
+    // scripts/backtest-corpus-export-core.ts hashes `JSON.stringify(cases.map(canonicalizeCase))`, which for
+    // an empty list is the two-byte string "[]" -- re-derived here so the constant cannot drift from it.
+    expect(EMPTY_CORPUS_CHECKSUM).toBe(await sha256Hex("[]"));
+  });
+
   it("degrades fail-safe on a broken store and reports null freeze point on a fresh ledger", async () => {
     const empty = await loadPublicRulePrecision(createTestEnv(), NOW);
     expect(empty).toEqual({