Skip to content

fix(Conformance): render a rule defect totally, capped at 500 characters - #272

Merged
thomasfosterau merged 2 commits into
trunkfrom
issue-246-rule-defect-followup-2
Oct 8, 2026
Merged

thomasfosterau merged 2 commits into
trunkfrom
issue-246-rule-defect-followup-2

Conversation

@thomasfosterau

@thomasfosterau thomasfosterau commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

Summary

Follow-up to #265. Refs #246.

The review follow-up to #265 (nothing in that review was blocking):

  1. The defect renderer is total (src/internal/conformance/core.ts, runRule). Rendering an Errored outcome's message could itself throw: Predicate.isError on a revoked Proxy throws TypeError: Cannot perform 'getPrototypeOf' on a proxy that has been revoked. That throw escaped Effect.catchDefect and killed the run the Errored outcome exists to keep. Rendering now runs under Result.try and falls back to the fixed string [unrenderable defect]. errorMessage renders name through a template too, so a non-string name that coerces to a string (a number, say) still renders; one that cannot coerce (a Symbol, or an object whose toString throws) throws there, and the message falls back to [unrenderable defect].
  2. Errored.message is capped at 500 UTF-16 code units, the … marker included. A longer message is cut to 499 code units plus …. The cut never splits a surrogate pair: a high surrogate that would end up last is dropped as well. defect itself is kept whole. The cap reaches assert's RuleDied message too, because that message is built from Errored.message.
  3. Fixture-throw behaviour is pinned with an it.each after "records a cleanup that fails". On trunk it matches the description for a thrown value the kit can render (the test throws a tagged error), so nothing changed: a create that throws skips every write rule with a reason naming writes["articles"].create threw; an update that throws skips only spec/status/update-response and every other write rule passes; a cleanup that throws leaves every creation except the one the kit DELETEd as cleanup-failed. None of the three produces an Errored outcome or a finding. A fixture member that throws a value the kit can't render (a revoked Proxy, Object.create(null)) still makes its rule Errored — that is bug: a write fixture that throws an unstringifiable value is reported as Errored, not Skipped #271.
  4. Changeset: dropped "and never was" from the pending .changeset/conformance-rule-defect-outcome.md and added the cap and the total rendering to it.

Second review round (commit 2, after rebasing onto trunk b1d774b)

  • "A write fixture that throws is not a defect" is qualified everywhere it appears. cleanup is included: runCleanup in write.ts calls Cause.pretty, which throws on a revoked Proxy, a throwing getter or a Symbol name. The sentence now reads "A write fixture that throws a value the kit can render is not a defect … One that throws a value the kit can't render (a revoked Proxy, say) is still reported as Errored until bug: a write fixture that throws an unstringifiable value is reported as Errored, not Skipped #271." It is changed in the changeset, the Conformance.Outcome JSDoc and README.md. Only that one README sentence changed, because other open work also edits README. docs/conformance.md's "A cleanup that fails is recorded, not propagated" is qualified the same way. Published files already link issues (README [#260](…), JSDoc {@link https://…/issues/122 #122}, changesets #237/#250), so the published text links bug: a write fixture that throws an unstringifiable value is reported as Errored, not Skipped #271 in the same style. write.ts is not touched here; bug: a write fixture that throws an unstringifiable value is reported as Errored, not Skipped #271 now covers cleanup too.
  • The errorMessage template is pinned, and its doc no longer over-claims. New rows in renders %s a rule dies with as itself: a tagged error whose name is 42, with an empty message (renders "42") and with a message (renders "42: refused"). A new test, marks an error whose name cannot become a string as unrenderable, uses a Symbol name and expects [unrenderable defect]. The doc now says that a coercible non-string name renders and a non-coercible one falls back to the marker. The test uses TransportRefused, not new Error, so that it adds no avoid-untagged-errors warning.
  • Conformance.assert JSDoc now notes that RuleDied's message is always rendered, while its cause holds the defect as it was thrown. A reporter that pretty-prints the cause can therefore throw on a defect that cannot be inspected. The cause contract is unchanged.
  • "500 UTF-16 code units" replaces "500 characters" in the public Outcome JSDoc, the changeset and the internal Errored doc.

Tests (written first)

  • reports a defect no rendering can read, %s, as Errored uses a revoked Proxy in three ways: thrown by the transport, passed to Effect.die by the transport, and thrown before the rule's Effect starts (a writes getter). Each must produce one Errored outcome with the message [unrenderable defect], keep the same defect, let the run complete and print the rule died: [unrenderable defect], and leave the store empty. To support this, verifyOver/run gained an optional client wrapper.
  • caps the message of a defect too long for a report line, does not split a surrogate pair when it caps a message, leaves a message at the cap as it is.
  • renders an error named by a number … (two rows) and marks an error whose name cannot become a string as unrenderable.

Mutation checks:

  • With trunk's core.ts restored, 5 tests fail: all 3 revoked-proxy cases, where the run itself rejects with the TypeError from Predicate.isError at core.ts:566, plus the cap test and the surrogate test.
  • With the surrogate guard removed from capped, only the surrogate test fails.
  • With errorMessage's empty-message branch untemplated (error.name instead of `${error.name}`), renders an error named by a number a rule dies with as itself fails: the number reaches capped, which throws, so the message becomes the marker instead of "42". The other 110 tests pass.
  • The fixture-throw it.each pins trunk's behaviour, so it passes on trunk by design.

Found, not done

#271: write.ts builds a fixture's skip reason with String(error), and runCleanup records a failed cleanup with Cause.pretty. A fixture that throws Object.create(null) or a revoked Proxy therefore makes its rule Errored instead of Skipped or cleanup-failed. The run still completes because of item 1 above. It sits outside this follow-up's scope, so it is filed rather than fixed here.

Verification

Each gate was run in AGENTS.md order on the head commit's tree (rebased onto trunk b1d774b). Every one passed.

  • pnpm install --frozen-lockfile: pass
  • vp run build: pass
  • vp check: pass, Found 0 errors and 4353 warnings. None of the warnings is on an added line: I checked the lint output against the diff's added line ranges.
  • vp run check (sources, type-level tests, examples, emitted declarations): pass
  • vp run -r test:unit: pass. Root: 39 files, 2266 tests. Examples: 42, 39 and 53 tests.
  • vp run docgen (every JSDoc @example compiled and executed): pass, "Docs generation succeeded!"
  • fallow audit --format json --quiet --explain --gate-marker agent: verdict pass
  • fallow audit --base origin/trunk: pass. "No issues in 6 changed files"; 11 inherited findings were excluded by the gate.

Changeset

  • A changeset under .changeset/ for every change to published behaviour. I amended the pending, unreleased .changeset/conformance-rule-defect-outcome.md (minor) rather than adding a new one: the cap, the total rendering and the qualified fixture-throw sentence all refine the same unreleased Errored outcome.

Notes

🤖 Generated with Claude Code


Generated by Claude Code

claude added 2 commits October 8, 2026 06:59
…cters (#246)

Follow-up to #265. Rendering an `Errored` outcome's message could itself
throw — `Predicate.isError` on a revoked `Proxy` throws — and the throw
escaped `Effect.catchDefect`, killing the run the outcome exists to keep.
The renderer now runs under `Result.try` and falls back to
`[unrenderable defect]`, and caps the message at 500 characters, ending a
longer one in `…` without splitting a surrogate pair.

Pins the existing fixture-throw behaviour: a `create` that throws skips
every write rule, an `update` that throws skips only `update-response`, and
a `cleanup` that throws is recorded as `cleanup-failed`; none is Errored.

Drops "and never was" from the pending changeset and adds the cap and the
total rendering to it.

Refs #246

Claude-Session: https://claude.ai/code/session_0187g2QpzaHqEzX8GJxiJBVc
Co-Authored-By: Claude <noreply@anthropic.com>
…ssage template (#246)

A write fixture that throws a value the kit can't render (a revoked
Proxy, say) still makes its rule Errored until #271, cleanup included;
the changeset, the Outcome JSDoc, the README and docs/conformance.md now
say so instead of promising it never does.

Pin errorMessage's template with an error whose name is a number (it
renders as "42"; untemplated, it would fall back to the marker) and one
whose name is a Symbol (the marker), and say exactly that in its doc.
Note on Conformance.assert that RuleDied's message is always rendered
while its cause holds the defect as thrown. Say "UTF-16 code units"
for the cap.

Claude-Session: https://claude.ai/code/session_0187g2QpzaHqEzX8GJxiJBVc
Co-Authored-By: Claude <noreply@anthropic.com>
@thomasfosterau
thomasfosterau force-pushed the issue-246-rule-defect-followup-2 branch from 6ea7e11 to 56a8c65 Compare October 8, 2026 07:04
@thomasfosterau
thomasfosterau merged commit d66fb2a into trunk Oct 8, 2026
1 check passed
@thomasfosterau
thomasfosterau deleted the issue-246-rule-defect-followup-2 branch October 8, 2026 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants