Repository navigation
feat(Conformance): skip and requestTimeout options (#302) - #366
Merged
Merged
Conversation
This was referenced Oct 15, 2026
`Options.skip` leaves rules out by id prefix, the complement of `only` and matched the same way (a plain string prefix; segment-aware matching for both is #367). It removes rules from what `tiers` and `only` select, and a rule it leaves out is absent from the report, as one `only` does not select is. `Options.requestTimeout` (a `Duration.Input`) bounds each request the runner sends: the client's whole `execute`, a `retry` included, and the reading of the body. Each redirect hop has a timeout of its own. A request it cuts short is interrupted and reads as a lost response, `status: 0` with a transport error naming the timeout, never as an unfollowed redirect. Owner decision on #302: it defaults to thirty seconds for `verify` and `verifyWith` alike. Any positive duration overrides it; `"Infinity"` (or `Duration.infinity`) turns it off. A duration that is not positive (zero, negative, `NaN`) is a defect: the run dies before it sends a request. Closes #302 Claude-Session: https://claude.ai/code/session_0187g2QpzaHqEzX8GJxiJBVc Co-Authored-By: Claude <noreply@anthropic.com>
thomasfosterau
force-pushed
the
issue-302-skip-and-timeout
branch
from
October 15, 2026 06:43
829a7d0 to
eb83353
Compare
…e durations (#302) A run of `verify` against handlers that never answer is still waiting a millisecond before thirty seconds and cuts each request short at thirty, so the default is pinned for `verify` as well as `verifyWith`; `verify`'s JSDoc says so, as `verifyWith`'s does. A `requestTimeout` that `Duration` cannot read (`"1e3 seconds"`, `".5 seconds"`), which the type admits, dies as a non-positive one does; the JSDoc, README, design note and changeset now name it, the `requestTimeoutOf` table and the run-level test cover it, and the message asks for "a positive duration `Duration` can read". The run-level "dies before it sends a request" tests now use a client that fails at once, so a run that read its options after sending would finish and fail on `sent`, not hang into the test timeout. Part of #302 Claude-Session: https://claude.ai/code/session_0187g2QpzaHqEzX8GJxiJBVc Co-Authored-By: Claude <noreply@anthropic.com>
This was referenced Oct 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the owner decisions recorded on #302: a skip list plus a timeout, with no per-rule budget options (2026-10-15 decision on the default and on prefix matching).
Options.skipis a list of rule-id prefixes to leave out, the complement ofonly.onlymatches one: a plainstartsWithon the rule id. So"spec/links/"covers the three link rules, and"spec/pagination/links-enumerate-collection"covers the pagination walk. Per the owner, plain prefixes stay in this PR. Segment-aware matching for bothonlyandskip(an exact id, or ids that continue it with/) is follow-up feat(Conformance): segment-aware rule-id prefixes foronlyandskip#367. It cites thespec/status/create-location/spec/status/create-location-matches-selfcollision and the accept-weight test prefix that would need updating.docs/conformance.md§1.5 links it.only.tiersandonlyselect first, and thenskipremoves rules from that selection (src/Conformance.ts,selected). A rule that bothonlyandskipname is left out.Report. It is absent, the same as a ruleonlydoes not select. ASkippedoutcome keeps its existing meaning: a rule that ran and could not decide.Options.requestTimeouttakes aDuration.Input. It is applied inrunner(src/internal/conformance/core.ts).verifyandverifyWithalike (owner decision).Core.requestTimeoutOfresolves the option, and the runner now takes the resolvedOption<Duration>explicitly, with no default parameter of its own.Duration.Inputoverrides the default.requestTimeout: "Infinity", orDuration.infinity, which is the same value."Infinity"is already aDuration.Input, so the option keeps a single type and needs nofalsesentinel. With the timeout off, the kit waits as long as the transport does, as it did before.NaN(whichDurationreads as zero),"-Infinity", or an inputDurationcannot read ("1e3 seconds",".5 seconds", which the type admits) all make the run die with aRangeErrorbefore any request is sent. The message reads`requestTimeout` must be a positive duration `Duration` can read, or "Infinity" for no timeout; got ….execute, a client's ownretryincluded, plus the reading of the body. Each redirect hop gets a timeout of its own. A request it cuts short is interrupted (overfetch, aborted) and reads as a lost response:status: 0, a transport error naming the timeout, andunfollowed: None. It is therefore never an unfollowed redirect, and the Conformance: follow same-origin redirects (follow-up to #328) #333 skip logic does not fire on it.requestTimeout: "Infinity", because its PATCH must still be waiting when the interrupt comes."Infinity"throughrequestTimeoutOf.verifyconformance tests pass unchanged under the default.Docs.
docs/conformance.md§1.5 now covers the default, the off spelling and the defect.skip,onlyand a read the timeout cuts short all narrow whatContext.observedhas seen before the write rules vet a create (write.ts: "only as complete as the reads before it").reachable,execute,retry,execute,cleanup), and Conformance: follow same-origin redirects (follow-up to #328) #333 is now a link.Options/verifyWithJSDoc (whose example is nowskip: ["spec/links/"]) and the changeset are updated.Closes #302
Verification
Run in order on head
9bb1a9a(rebased onorigin/trunk, which had not moved):pnpm install --frozen-lockfile: passvp run build: passvp check: pass (0 errors). This change adds no lint warning: per-rule warning counts on the four changed source and test files match829a7d0's, and the in-process default test usesRegistry.platformLayerrather than a cast.vp run check(sources, type-level tests, examples, emitted declarations): passvp run -r test:unit: pass (root 3397 tests; examples 43 / 45 / 55)vp run docgen(every JSDoc@examplecompiled and executed): passfallow audit --format json --quiet --explain --gate-marker agent: verdictpassfallow audit --base origin/trunk: pass ("No issues in 9 changed files"; 10 inherited findings excluded)pnpm audit --audit-level high: pass (exit 0; 1 moderate advisory, below the threshold)New tests for the default, the off spelling and the rejection. All run under
TestClockor with no clock at all.core.test.ts,requestTimeoutOf:"Infinity"andDuration.infinityresolve to no timeout.0,-1,NaN,"-5 seconds","-Infinity","1e3 seconds",".5 seconds"andDuration.zero.core.test.ts,runner (requestTimeout):HttpClient.retryruns inside the timeout. The first attempt hangs and the second would answer, yet under a 1s timeout the result isstatus: 0with 1 attempt and 1 interrupt."Infinity", the runner waits an hour.Conformance.test.ts,requestTimeout:30s.verifyagainst handlers that never answer (Effect.neverongetandlist) is still pending at 29.999s, cuts the first request short at 30s and the second at 60s, with notes naming30s."Infinity"orDuration.infinityis still waiting an hour later.0,-1,NaN,"-Infinity"or"1e3 seconds"dies before it sends a request. The client fails each request at once, so a run that sent one before reading its options finishes and fails onsent, rather than hanging into vitest's 5s timeout.Mutation checks. Each mutation was applied (to
requestTimeoutOf, or toexecutewhere noted), the core and conformance suites were run, and the mutation was reverted:requestTimeoutOfdefault)"Infinity"resolves to the default)verifyWith(execute: no timeout in process unless set)execute: validation moved after the requests)expected 2 to be +0(sent), not on the test timeoutThe earlier mutation checks still apply:
skipignored,onlyoverridingskip, no timeout applied, a per-probe timeout, the body read outside the timeout, a timeout classified as an unfollowed redirect, andrequestTimeoutnot passed torunner.Changeset
.changeset/conformance-skip-and-timeout.md: minor, because this is new public API onConformance.Options. It says in bold that the default changes for every run: a request unanswered after 30s is now a lost response where the run used to wait, and"Infinity"restores the old behaviour.Notes
onlyandskip#367, segment-aware prefixes foronlyandskip.Conformance.Optionsgainsskip?andrequestTimeout?.Core.requestTimeoutOfandCore.DEFAULT_REQUEST_TIMEOUTare new.runner's third parameter is now required.Core.ResolvedOptions.requestTimeoutis the resolvedOption<Duration>.verifyruns are now bounded too. A handler slower than 30s per request would read as lost. None of the repo's own suites comes close.cleanup, which is not a request the kit sends. A create it cuts short is treated as having created nothing, though the server may have created it anyway. Both points are documented.🤖 Generated with Claude Code
Generated by Claude Code