Repository navigation
Conversation
…l OOM kill Since pingdotgg#17662 every agent scope starts with OOMPolicy=stop. The Codex app-server is shared by every thread on a provider instance, and the commands its tools run live in the same scope. When the kernel OOM-killed one tool process, systemd stopped the whole scope, so the app-server died and every Codex thread on that instance failed. AgentScope.wrap now takes `shared`, and the Codex app-server spawn passes it. On systemd 253 or newer, shared scopes start with OOMPolicy=continue, so systemd logs the kill and the app-server keeps running. Claude and terminal scopes keep OOMPolicy=stop. The probe and the plain-scope fallback for older systemd are unchanged. With OOMPolicy=continue systemd never records Result=oom-kill, so such a scope can never explain one thread's failure. It is no longer tracked for the thread that opened it. Otherwise every later failed turn on that thread would poll the scope for about 2 seconds, waiting for a stop that never comes. Launching it still ends the thread's previous answer, and nothing reads its result, so it gets --collect like thread-less scopes. Older user managers cannot set OOMPolicy on a scope. There a shared scope starts as before and stays tracked for its thread without --collect, because systemd can still stop it on an OOM kill and record Result=oom-kill (systemd 252 does). Not changed: - The kernel still picks the victim. If it kills the app-server itself, every thread on the instance still loses it. - A systemd-oomd kill still ends the whole scope. - On systemd 253 or newer, a Codex turn that fails because its app-server was itself killed for memory is no longer labeled "Killed: out of memory.", since systemd records no OOM result under OOMPolicy=continue. Before, only the thread that opened the session got that label. - Recovering a session whose app-server died is pingdotgg#17046 and pingdotgg#18139. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
ApprovabilityVerdict: Approved at Macroscope's review found this PR approvable — This is a narrowly scoped fix for shared Codex app-server scopes being terminated when a tool is OOM-killed. The new policy is isolated to shared Codex launches, preserves existing behavior for other scopes and older systemd versions, and is covered by focused tests. You can add or adjust custom eligibility rules. Learn more. |
macroscopeapp
Bot
dismissed
their stale review
October 11, 2026 19:59
Dismissing prior approval to re-evaluate d25189d
2 of 3 tasks
The test now says it checks the shared app-server launch options and also asserts that the shared scope is launched with --collect. The probe comment now says which older systemd user managers keep the OOM label: 252 still stops a plain scope on an OOM kill and records it, while 251 and older do neither. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
only21mil
force-pushed
the
fix/17662-shared-codex-scope-oom-continue
branch
from
October 12, 2026 11:30
d25189d to
9796521
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
#17662 introduced a regression for the shared Codex app-server on systemd 252 or newer by putting it and its tools in a scope with
OOMPolicy=stopwhere supported. A kernel OOM kill of one tool now stops the whole scope, taking down the app-server used by every Codex thread on that provider instance. #16502 and #17045, both filed before #17662 merged, report threads staying broken after the shared app-server exits; #17046 and #18139 address that recovery.An October 11 comment on #17045 reports exit code 0 after an earlier kernel OOM kill in a T3 agent scope on an October 10 nightly, consistent with a scope stop but not proof of one. The commenter notes no OOM near the later failures and says OOM does not explain the repeated fresh-thread failures, which are the recovery bug.
Change
sharedtoAgentScope.wrapfor a process used by several threads.shared: truein the Codex app-server spawn helper, the only caller changed.OOMPolicy=continue, remove the launching thread's scope tracking and previous OOM answer, and add--collectbecause no thread reads the scope result. This avoids the roughly two-second poll after a tool OOM, since a continuing scope never recordsResult=oom-kill.apps/server/src/cloud/bootService.tsalready keeps the server unit onOOMPolicy=continuebecause a helper's kernel OOM kill would otherwise stop the whole unit.Behavior and limits
OOMPolicyon scopes. On 252 a tool's OOM kill still stops the shared scope, so this PR does not help there. On 251 and older a plain scope never stops for an OOM kill, so the problem does not arise.OpenCodeRuntime.tsis shared the same way and keepsstophere; it can takeshared: trueseparately. All other scope policies stay unchanged.Scope and approval
CONTRIBUTING permits: "A very small, focused fix for an obvious bug can be submitted without a prior issue or discussion."
There is no maintainer-triaged issue for this exact change; it fixes one defect, a tool OOM stopping unrelated Codex threads through the shared scope, and the exception does not imply maintainer approval.
Refs #17662, #17045, #16502, #17046, #18139.
Verification
Environment: Linux x64, systemd 259, Node 24.21.
Regression results at
3c38e194ec, with the same nine-test file against main's source and the branch. The branch image'sc3d03a7eb3has the same tree as3c38e194ec; only the author and committer differ:Main source: 2 failed | 7 passed (9), missing the shared policy and immediate answer.

Branch: 9 passed (9), including shared policy, immediate answer and older-systemd fallback.

With the policy swap alone, shared scopes still tracked as on main, 2 failed | 7 passed (9): the shared scope loses
--collectand the OOM answer waits again. This distinguishes the tracking fix from the policy change.These tests use fake systemd and do not create a real Codex OOM.
A scratch run used T3's Codex spawn helper and AgentScope service with a stand-in app-server on real systemd, a verified 64 MiB scope cap and no swap; the slice write was a logged no-op and the unit was renamed so local drop-ins did not apply.
2625327090--property=OOMPolicy=stop--property=OOMPolicy=continue --collect137, kernel OOM SIGKILLserver_alive=yes, scopeactive/running,oom_kill=1SIGTERM, systemd stopped the scope0oom-kill, scope failedsuccess; unit collected after the stand-in exitedoomKilled("e2e-thread")truefalseThe image's

ef46d65bafis this branch before its rebase ontobed923986f, with the same patch ID as9796521680.Focused checks at
9796521680: 161 Codex adapter tests and 1 shared test passed; lint, format, Knip and both typechecks exited 0, with two existing lint warnings and typecheck suggestions. Lint, format, Knip and the server typecheck exited 0 again on3c38e194ec.Not checked
Model and harness: Claude Opus 5.5 in Claude Code and GPT-6.1 Sol in Codex, run from T3 Code.
🤖 Generated with Claude Code