Bash Tool Fails with posix_spawnp Error After Extended Use

[hamadycisselp360]

Describe the bug

Running bash tools consistently fails after a period of time with the error: <exited with error: posix_spawnp failed.>

Affected version

0.0.358

Steps to reproduce the behavior

 1. Start multiple bash operations or long-running bash commands
 2. After some time (varies), bash tool execution fails
 3. Error appears to be related to process spawning

Expected behavior

Bash tools should execute reliably without timing out or failing due to process spawning issues.

Additional context

Error Details

 - **Error Type**: posix_spawnp failure
 - **Component**: Bash tool
 - **Frequency**: Intermittent, occurs after extended usage
 
 ## Environment
 - OS: macOS/Linux (POSIX-based)
 - Bash Tool Version: [Current]
 - Date Reported: 2025-11-27
 
 ## Possible Root Causes
 - File descriptor exhaustion
 - Process limit reached
 - Resource cleanup not happening properly
 - Signal handling issues
 
 ## Suggested Investigation
 - Check system resource limits (ulimit)
 - Verify proper cleanup of spawned processes
 - Review signal handling and process termination
 - Investigate file descriptor management
 
 ## Related Issues
 - None identified yet

[sj4nes]

I am seeing this error also, macOS Tahoe 26.1 and using Ghostty terminal:

copilot --version
0.0.365
Commit: 76d0881

When it happens starting a new terminal tab in Ghostty also kind of goes bonkers and fails to open a new tab.

[sj4nes]

I rebooted macOS completely and the issue hasn't reappeared so far 🤞

[hamadycisselp360]

I had already rebooted few days ago (Friday), I have started working with Copilot CLI since yesterday and hadn't seen the issue yet. I will monitor and eventually close the issue if it does not come back,

thanks

[sj4nes]

I knew I was jinxing myself. That error has returned after a couple more hours.

[sj4nes]

I switched from Ghostty to Kitty and copilot is able to spawn commands-- just for a bit. The spawnp error returned quickly after that, so my gut feeling is this is a macOS issue in general. I started a clean session again and watched PTY usage:

sjanes@MBP-LHYP9PW370 accession-console % while true; do sleep 3; lsof | grep -E "ttys|pty" | wc -l; done
     118
     123
     123
     123
     123
     123
     123
     138
     138
     138
     143
     143
     143
     143
     143
     143
     143
     148
     153
     163
     173
     183
     198
     198
     208
     218
     218
     218
     218
     218
     218
     218
     218

When it got to 218 in the system the spawnp error appeared. When I cancelled the agent's activity before it started thrashing on trying different things to execute (it had two spawnp errors in fast succession) the PTY level dropped to 68. So that was interesting. The PTY use didn't decrease until I ESC-cancelled the agent's activity. In the same session I told the agent to resume with the more complicated pipeline it first attempted and it was able to execute that without the error.

[mozumder]

Seeing the same issue running copilot CLI with batch jobs. Mac OS X Tahoe running Visual Studio Code terminal running zsh running Copilot CLI running subagent prompt.

[tobiashochguertel]

Important

I have a personal stake in obtaining the spent premium points. This would be cordial.

I hope this helps the developers and project stakeholders to find and resolve the issue:

---

I can give as feedback that it looks like that copilot CLI is just opening new shell sessions and that we reach the limit with that behavior quickly. I explained to Copilot that he should investigate why we get this spawn issue and what he should do when he runs into that again and try to analyze and document it then…

What happened is this:

Here is the start of my prompt:

 > Can you continue with the work, not all tasks from the original prompt are yet done. If you get again the issues with the terminal, can you try to close a bahs session and check if the issue perist, and if not can you then debug if the shell / bash issue is based on how many processes or so are open on my macos? if the issue is related by `ulimit -a`, tty, pty amount, etc.? That happens oftener and it is something what is related to the amount of processes or forks we or you open.

It looks like when copilot CLI runs, it doesn't clean up its forked processes?

When I exit the copilot session, I can then open a new tab in iterm or Terminal...

 > I closed the Copilot session and now you should be able to continue to work and you might can analyse the terminal / spawn
   issue, so that I can report it to Github. I have the feeling that when you open many bash-sessions / tools that they are not
   properly cleaned up as long your parent process is running?

 ✔ List shells
   └ <no active sessions>

 ● Good, sessions are clean now. Let me first check GitHub forks status and then continue with creating the documentation:

 ✔ Check existing forks in GitHub account
    $ gh repo list tobiashochguertel --fork --json nameWithOwner,name 2>/dev/null | head -50
    └ 2 lines...

 ✔ List directory usage-lib-src
   └ 37 files found

 ✔ Check mise repo exists
    $ ls -la /Users/tobiashochgurtel/mise 2>/dev/null | head -20
    └ 21 lines...

 ● Both forks exist (tobiashochguertel/usage and tobiashochguertel/mise). Now let me verify the usage-lib-src repository is your
   fork and create the documentation structure:

 ✔ Check usage-lib-src remotes
    $ cd /Users/tobiashochgurtel/usage-lib-src && git remote -v
    └ 5 lines...

 ✔ Check mise remotes
    $ cd /Users/tobiashochgurtel/mise && git remote -v
    └ 7 lines...

 ● Both repositories are set up with forks. Now let me examine the code to verify the terminal width issue and document everything
   properly:

 ✔ Read usage-lib-src/lib/src/docs/layout.rs
   └ 255 lines read

 ✔ Check existing branches in usage repo
    $ cd /Users/tobiashochgurtel/usage-lib-src && git branch -a | head -20
    └ 16 lines...

 ✔ Check existing branches in mise repo
    $ cd /Users/tobiashochgurtel/mise && git branch -a | head -20
    └ 21 lines...

 ● Excellent! I can see that:

     - The usage-lib-src repo already has the fix branch fix/terminal-width-wrapping
     - The code has been modified with the proper terminal width detection

   Now let me check the Cargo.toml to verify the terminal_size dependency was added, then create the documentation:

 ✔ Check terminal_size dependency
    $ cd /Users/tobiashochgurtel/usage-lib-src && grep -n "terminal_size" lib/Cargo.toml
    └ 2 lines...

 ✔ Create documentation directory
    $ mkdir -p /Users/tobiashochgurtel/docs/research/mise-improvements
    └ 1 line...

 ● The dependency is added. Now let me research the task description issue in the mise codebase:

 ✔ Grep "description" in *.rs (/Users/tobiashochgurtel/mise/src/cli/tasks)
   └ 3 files found

 ✔ Grep "task.*help|help.*task" in *.rs (/Users/tobiashochgurtel/mise/src)
   └ 8 files found

 ✔ Grep "SpecUsage|usage::Spec" in *.rs (/Users/tobiashochgurtel/mise/src)
   └ 4 files found

 ✔ Read mise/src/task/mod.rs:1-100
   └ 105 lines read

 ✔ Read mise/src/task/task_script_parser.rs:1-100
   └ 105 lines read

 ✔ Grep "to_usage_spec|SpecUsage" in *.rs (/Users/tobiashochgurtel/mise/src)
   └ No matches found

 ✔ Grep "usage::Spec" in *.rs (/Users/tobiashochgurtel/mise/src)
   └ 261 lines found

 ✔ List shells
   └ Active shell sessions:

 ● You're correct. Looking at the list_bash output, I can see 8 bash sessions (sessionId 0-7) that are still "active" even though
   they've all completed (status: completed).

   Key observations for your bug report:

     - Sessions persist after completion - All sessions show status: completed (exit code: 0) but remain in the active sessions
   list
     - No automatic cleanup - Sessions from ~54 seconds ago (0, 1) through ~28 seconds ago (6, 7) are all still tracked
     - Resource accumulation - Each session has a PID, suggesting actual processes or resources being held
     - Unread output retained - Most sessions still have unread output: yes, meaning buffers aren't released

   Bug hypothesis: The Copilot CLI's bash session manager:

     - Creates new sessions for parallel tool calls
     - Marks them "completed" but doesn't terminate/cleanup
     - Accumulates sessions until... (the crash you experienced?)

   For your GitHub issue, I'd suggest:

     ## Issue: Bash sessions not cleaned up after completion

     **Observed behavior:**
     - `list_bash` shows 8+ sessions with `status: completed` but still "active"
     - Each has a PID and retains unread output buffers
     - Sessions accumulate during a session lifetime

     **Expected behavior:**
     - Completed sync sessions should be cleaned up automatically
     - Resources (PIDs, buffers) should be released

     **Impact:**
     - Potential resource exhaustion during long sessions
     - May contribute to PTY/terminal instability

   Want me to check if I can manually stop these sessions, or would you prefer I leave them as evidence for debugging?

---

 > Can you try to stop the sessions with your builtin tools or functions and if that not works can you try to close them manually? And can you
   debug the issue somehow further and document it for Github? You might want to stop the sessions then after the debugging?

 ✔ List shells
   ╭──────────────────────╮
   │ <no active sessions> │
   ╰──────────────────────╯

 ● Good - no active sessions currently. Let me check system resources and then proceed with the work:

 ✔ Check system resources for session debugging
    $ echo "=== System Resource Check ===" && echo "PTY devices:" && ls /dev/pty* 2>/dev/null | wc -l && echo "TTY devices:" && ls /dev/tty*
    2>/dev/null | wc -l && echo "" && echo "=== ulimit -a ===" && ulimit -a && echo "" && echo "=== Process count for current user ===" && ps aux |
    grep -c "$(whoami)" && echo "" && echo "=== Open file descriptors (approximate) ===" && lsof 2>/dev/null | wc -l
   ╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
   │ === System Resource Check ===                                                                                                                 │
   │ PTY devices:                                                                                                                                  │
   │      128                                                                                                                                      │
   │ TTY devices:                                                                                                                                  │
   │      531                                                                                                                                      │
   │                                                                                                                                               │
   │ === ulimit -a ===                                                                                                                             │
   │ core file size              (blocks, -c) 0                                                                                                    │
   │ data seg size               (kbytes, -d) unlimited                                                                                            │
   │ file size                   (blocks, -f) unlimited                                                                                            │
   │ max locked memory           (kbytes, -l) unlimited                                                                                            │
   │ max memory size             (kbytes, -m) unlimited                                                                                            │
   │ open files                          (-n) 1048575                                                                                              │
   │ pipe size                (512 bytes, -p) 1                                                                                                    │
   │ stack size                  (kbytes, -s) 8176                                                                                                 │
   │ cpu time                   (seconds, -t) unlimited                                                                                            │
   │ max user processes                  (-u) 10666                                                                                                │
   │ virtual memory              (kbytes, -v) unlimited                                                                                            │
   │                                                                                                                                               │
   │ === Process count for current user ===                                                                                                        │
   │ 638                                                                                                                                           │
   │                                                                                                                                               │
   │ === Open file descriptors (approximate) ===                                                                                                   │
   │    29394                                                                                                                                      │
   │ <exited with exit code 0>                                                                                                                     │
   ╰─────────────────────────────

● Done! Here's the summary:

   Session Cleanup Results

   Found & Stopped: 6 stale sessions (IDs 8-13)

     - All were "completed" but still persisted in memory
     - Most had "unread output: yes"

   Key Findings

     - Completed sessions don't auto-cleanup - They persist until explicitly stopped
     - stop_bash works - Successfully terminates even finished sessions
     - No sessions remaining - Verified with list_bash

   Documentation Created

   📄 /Users/tobiashochgurtel/docs/mise-improvements/99-copilot-cli-session-debugging.md

   Contains:

     - Issue symptoms and debugging procedures
     - System resource checks
     - Session management tools reference
     - Findings from this debug session
     - GitHub issue template ready to file

   Would you like me to view the complete documentation or help file a GitHub issue?

I attached the created documentation: 99-copilot-cli-session-debugging.md

99-copilot-cli-session-debugging.md

[lukaszsamson]

Workaround: kill the session and copilot --resume

[crmitchelmore]

This happens all the time and is very tedious to manage especially when running multiple sessions

[bbloch-veepee]

Workaround: kill the session and copilot --resume

Thanks 👍

[djdefi]

Additional Data: FD/PTY Handle Leak Analysis

We've been investigating this issue and have concrete data on the root cause.

Evidence

Comparing a fresh session vs a long-running session:

Session	Age	Total FDs	PTY Handles
Fresh	~12 hours	120	16
Long-running	~29 hours	937	31

Analysis

Each bash tool call allocates a PTY that isn't properly released. The FD accumulation rate is approximately 32 FDs/hour.

# Commands used to gather data
lsof -p <copilot_pid> | wc -l           # Total FDs
lsof -p <copilot_pid> | grep -c ttys    # PTY handles

Hypothesis

The copilot binary spawns PTY devices for bash sessions but doesn't properly close them after the command completes. Over time:

1. FDs accumulate
2. PTY handles accumulate
3. Eventually posix_spawn fails with ENOENT when trying to allocate a new PTY

Workaround

Restart copilot CLI sessions periodically (we restart daily).

Environment

• macOS (darwin-arm64)
• Copilot CLI 0.0.389
• Node.js via homebrew

cc @github/copilot-cli

[djdefi]

Definitive Evidence: PTY File Descriptor Leak

I've reproduced and confirmed the root cause with concrete evidence:

Orphaned PTY Analysis

Long-running session (PID 2772, ~29 hours):

• Orphaned PTYs: 29
• Total PTY FDs held: 35
• Active TTYs with processes: 1 (the parent terminal)

Fresh session (PID 9410, ~5 minutes):

• Total PTY FDs: 12
• All appear to be active bash sessions

Evidence of Leak

Using lsof and ps, I verified that the long-running copilot process holds FDs to ~30 PTY devices (/dev/ttysXXX) that have no process attached:

/dev/ttys024, /dev/ttys217, /dev/ttys233, /dev/ttys151, /dev/ttys004,
/dev/ttys007, /dev/ttys011, /dev/ttys157, /dev/ttys161, /dev/ttys221,
... (29 total orphaned)

Root Cause

The bash tool creates PTY sessions via node-pty, but when a bash command completes:

1. The child shell process exits ✅
2. The PTY master FD is NOT closed in the parent (copilot) process ❌

This causes FD accumulation (~32 FDs/hour), eventually hitting the system PTY limit.

Suggested Fix

Ensure PTY cleanup code explicitly closes the master FD when:

• A bash command exits
• A session is stopped
• Session timeout occurs

[MRayermannMSFT]

@djdefi I'm taking a look into this right now. Do you know if there's a reliable way to accelerate the FD leaking? I'd love to get a test we can add for this. Would also help with fix investigation.