Make Windows a supported platform, and hand the agent the run - #3
Merged
Conversation
Two of the three guarantees the harness declared absent off POSIX were absent because nothing had implemented them, not because Windows cannot express them. claim_eval's lock reported unconditional success there, so two evals could run against the same pinned baseline worktree at once; it now takes a real one-byte lock through msvcrt, seeking to 0 first so the lock is the same lock on every call. The release order flips on Windows, which refuses to unlink an open file: unlock, close, then delete, instead of the deliberate unlink-before-close the POSIX side needs. The pid write moves off os.pwrite, which does not exist there at all. A timed-out benchmark had only its direct child killed, leaking grandchildren that keep burning CPU and corrupt every later measurement. runner now puts each child in a kill-on-close job object -- the Windows equivalent of the process group -- and _kill_tree terminates that, falling back to killing the child alone if the job could not be created. The job is closed on every path, so a leaked handle cannot accumulate across a long loop, and an eval killed outright takes its benchmark tree with it. The race this leaves is documented rather than hidden: a grandchild spawned between CreateProcess and AssignProcessToJobObject is outside the job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDNWUNHzZaAPxM6yhX7riC
The refusal existed because three guarantees were missing, not because the platform is unmeasurable; with them implemented, refusing to start is itself the wrong answer, and the override environment variable that unlocked it goes with it. `stop --force` no longer reports that it cannot help. It terminates the eval instead of signalling it, which is the honest shape of the thing there: no signal exists that a benchmark can act on mid-round, so the experiment is ended rather than asked to end, and it does not get to report what it abandoned. The kill takes the eval's job objects with it, so the benchmark tree dies too. Plain `stop` is unchanged everywhere. doctor stops calling the platform a FAIL that cannot be trusted and reports what is actually true: the guarantees hold, -force is immediate rather than a request, and doctor itself knows less here (no load average, no CPU governor) so it warns about less. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDNWUNHzZaAPxM6yhX7riC
Start here was a bash block for a human to translate; the Go and TypeScript siblings hand the agent one prompt to paste, and this one now matches -- setup, the KEEP/DISCARD contract, what the agent may never touch, and how the human stops the loop. The install commands and file names are this project's own (config.toml, conftest.py, pytest-benchmark benchmarks), not the Go original's. The POSIX-only warning at the top and its counterpart under Limitations are gone, replaced by what is actually true now, including the residual job-assignment race and the fact that -force is immediate there. The claim rests on CI: windows-latest joins the matrix in the same commit, because a README that says Windows works while nothing has ever run there would be exactly the guessed number this project refuses to print. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDNWUNHzZaAPxM6yhX7riC
The classifiers listed POSIX and macOS only, which is the same claim the README used to make in prose. A package that says it works on Windows in its README and not in its metadata is telling pip something different from what it tells the reader. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDNWUNHzZaAPxM6yhX7riC
Four production defects, none of them visible from a POSIX machine. The compile gate compiled nothing. compileall reports what it walks with the platform's own separator, so the "." this gate passes came back as ".\bad.py" -- and the exclude regex's `\.[^/]` dotfile branch read that backslash as "not a slash" and excluded the entire tree. The gate exited 0 on a syntax error; the pytest gate caught it a stage later as a collection FAIL where it should have been a compile CRASH. Both separators now, in every branch of the pattern. The claim lock locked the pid text. Windows locks are mandatory, not advisory: a locked range cannot be read by another handle either, so eval_running and every refused claim -- the paths whose whole job is to say which pid holds the run -- failed with a permission error instead of an answer. The lock moves to a fixed byte past EOF, which is just as exclusive and blocks nothing anyone reads. git output was decoded with the platform locale (cp1252 on a default Windows install), so a repository holding a café.py reported a caf?.py that matches nothing on disk to every gate that reads a diff. git speaks UTF-8; the calls now say so. gitx.root also normalizes git's POSIX separators to the platform's own form, since doctor prints it and the run's state directory is keyed on the repository path. profile's site paths render with one separator everywhere -- the elision that shortens a long path cuts on "/", and a native "pkg\mod.py" gave it no boundary to cut on. The remaining six were tests encoding POSIX assumptions: a grandchild script that put C:\Users into a non-raw string literal and died of a SyntaxError rather than of the timeout it was there to test, PYTHONPATH split on ":" instead of os.pathsep, an unguarded `import resource`, two interpreter paths written into TOML without escaping, and a -force test that asserted the POSIX mechanism by name. test_winjob's kill-on-close check now measures that the process died early rather than trusting an exit code a killed process does not have to set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDNWUNHzZaAPxM6yhX7riC
g4lb
added a commit
that referenced
this pull request
Sep 10, 2026
A real msvcrt claim lock, a kill-on-close job object per benchmark tree, and a `stop -force` that reaches the eval (immediately, rather than as a request) replace the three guarantees the harness used to declare absent off POSIX; `eval`'s refusal and its override environment variable go with them, and `doctor` reports the one real difference instead of a FAIL. windows-latest joins the CI matrix in the same change, which is what the claim rests on. It immediately found four production bugs no POSIX machine could see: a compile gate whose exclude regex swallowed the whole tree (so a syntax error exited 0), a mandatory file lock sitting on the pid text it needed to read back, git output decoded with the platform locale, and profile paths elided mid-component. README's "Start here" is now the paste-into-your-agent handoff prompt the Go and TypeScript siblings use.
g4lb
added a commit
that referenced
this pull request
Sep 11, 2026
A real msvcrt claim lock, a kill-on-close job object per benchmark tree, and a `stop -force` that reaches the eval (immediately, rather than as a request) replace the three guarantees the harness used to declare absent off POSIX; `eval`'s refusal and its override environment variable go with them, and `doctor` reports the one real difference instead of a FAIL. windows-latest joins the CI matrix in the same change, which is what the claim rests on. It immediately found four production bugs no POSIX machine could see: a compile gate whose exclude regex swallowed the whole tree (so a syntax error exited 0), a mandatory file lock sitting on the pid text it needed to read back, git output decoded with the platform locale, and profile paths elided mid-component. README's "Start here" is now the paste-into-your-agent handoff prompt the Go and TypeScript siblings use.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two things the README claimed that are no longer true, fixed in the code rather than the prose.
Windows
evalrefused to start off POSIX, anddoctorcalled the platform a FAIL naming three guarantees that were "silently absent". They were absent because nothing had implemented them, not because Windows cannot express them — the Go sibling has all three.runstop._try_lockreturned unconditional success off POSIX, so two evals could run against the same pinned baseline worktree at once. It now takes a one-bytemsvcrtlock, seeking to 0 first so it is the same lock every time. The release order flips there (unlock, close, delete) because Windows will not unlink an open file, and the pid write moves offos.pwrite, which does not exist there at all.winjob.py, ctypes);_kill_treeterminates that instead of killing the direct child and leaking grandchildren that keep burning CPU. It falls back to the old direct kill if the job could not be created, and the job is closed on every path.stop -forcereaches the eval. It terminates the process rather than signalling it — there is no signal a benchmark can act on mid-round — and the eval's job objects die with it, so the benchmark tree goes too. The honest difference is documented:-forcethere is immediate rather than a request, and the eval does not get to record what it abandoned. Plainstopis unchanged everywhere.doctornow reports that difference instead of a FAIL, and theAUTOR3SEARCH_PYTHON_ALLOW_UNSUPPORTED_PLATFORMescape hatch is gone with the refusal it escaped.One residual race is documented rather than hidden: a grandchild spawned between
CreateProcessandAssignProcessToJobObjectis outside the job.subprocessoffers no way to create a process directly into one, and Go'sos/exechas the same window.The claim rests on CI, not on assertion:
windows-latestjoins the matrix in this PR. If it does not go green, the code stays and the README claim narrows.Start here
Rewritten as the paste-into-your-agent handoff prompt the Go and TypeScript siblings use, instead of a bash block for a human to translate — setup, the KEEP/DISCARD/FAIL/CRASH contract, everything the agent must never touch, and how the human stops the loop. Names are checked against this project's own (
.autor3search/config.toml,conftest.py, pytest-benchmark).Testing
Windows branches are covered on POSIX through the module-level
_POSIXseams the project already uses (a fakemsvcrt, a fake job, a recordedos.kill), and for real by thewindows-latestjob — where the existing cross-process claim test and the timeout's grandchild test become the actual proof, joined bytests/test_winjob.py.🤖 Generated with Claude Code
https://claude.ai/code/session_01EDNWUNHzZaAPxM6yhX7riC