Skip to content

fix(release): shard server tests out of the quality job - #88

Merged
MTVaught merged 1 commit into
mainfrom
t3code/release-quality-shard
Sep 10, 2026
Merged

fix(release): shard server tests out of the quality job#88
MTVaught merged 1 commit into
mainfrom
t3code/release-quality-shard

Conversation

@MTVaught

Copy link
Copy Markdown
Owner

Problem

"Release quality checks" runs check, typecheck, and the whole test suite in one job with a 10-minute limit. Durations over the last eight release runs sat between 8m28s and 10m17s, and the last three all hit the limit while every test was passing. The timeout marks the whole run cancelled even when the release publishes, which is why several recent Release runs read as cancelled despite shipping.

The fork runs this job on GitHub-hosted ubuntu-24.04 rather than upstream's 8-vCPU Blacksmith runner, and the suite has grown since.

Fix

Move apps/server tests into a three-shard quality_server job and leave the remaining packages in quality, the same split ci.yml already uses. Server tests run one file at a time (fileParallelism: false), so spreading them across runners is what actually shortens the wall clock. From the Sep 10 log, quality should now finish in about 7 to 8 minutes and each shard in 3 to 4.

Gating is unchanged: the publish job did not depend on quality before and still does not.

Verification

Workflow parses as valid YAML and passes the formatter. The Release run triggered by merging this shows the real timings.

Made with Claude Fable 5.1 in Claude Code.

"Release quality checks" runs check, typecheck, and the full test suite
in one job with a 10-minute limit. The job has sat at that limit for
weeks and timed out on the last three releases while every test was
passing, which marks the whole run cancelled even when the release
publishes.

Move apps/server tests into a three-shard `quality_server` job, the
same split ci.yml uses, and leave the remaining packages in `quality`.
Server tests run one file at a time, so sharding them across runners
is what actually shortens the wall clock.

Made with Claude Fable 5.1 in Claude Code.
@github-actions github-actions Bot added size:M vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. labels Sep 10, 2026
@github-actions

Copy link
Copy Markdown

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 13.2 KiB 13.3 KiB +27 B (+0.2%) 15.1 KiB
Codex Thread snapshot wire 6.9 KiB 6.9 KiB −5 B (−0.1%) 7.3 KiB
Codex Live turn WebSocket wire 6.3 KiB 6.4 KiB +32 B (+0.5%) 7.8 KiB
Codex Live turn WebSocket decoded 55.5 KiB 55.5 KiB +44 B (+0.1%) 66.4 KiB
Codex Live turn messages 8 9 +1 (+12.5%) 21
Claude Total thread wire 13.3 KiB 13.5 KiB +215 B (+1.6%) 15.1 KiB
Claude Thread snapshot wire 6.9 KiB 6.9 KiB +1 B (+0.0%) 7.3 KiB
Claude Live turn WebSocket wire 6.4 KiB 6.6 KiB +214 B (+3.3%) 7.8 KiB
Claude Live turn WebSocket decoded 56.3 KiB 57.8 KiB +1.5 KiB (+2.7%) 66.4 KiB
Claude Live turn messages 8 10 +2 (+25.0%) 21

Baseline: a8040fb · PR result: bb744c5 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 109.5 KiB
  • Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@MTVaught
MTVaught merged commit ab0c39b into main Sep 10, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant