Repository navigation
Segfault in call_soon_threadsafe() when another thread closes the loop (free-threaded Python) #788
Description
- uvloop version: 0.23.0 (also current
master, 9719743) - Python version: 3.14.8t and 3.14.10t, GIL disabled
- Platform: macOS arm64 and Linux x86_64
- Can you reproduce the bug with
PYTHONASYNCIODEBUGin env?: not tried, the repro does not depend on debug mode - Does uvloop behave differently from vanilla asyncio? How?: yes, asyncio does not crash with the same script
What happens
On a free-threaded build, call_soon_threadsafe() can segfault when another thread closes the loop at the same time.
We first saw it in a production-like load test of a web app: about 1 crash in 400 runs.
A thread finished its work and posted the result to a loop that was closing.
Why
call_soon_threadsafe() (loop.pyx) does two steps:
_append_ready_handle()checks_closedand appends to_ready.- It calls
self.handler_async.send().
_close() sets _closed = 1, closes the handles, and at the end sets self.handler_async = None.
Nothing makes these two methods exclusive.
A poster can pass the _closed check, then the closing thread runs _close(), and then the poster calls send() on None.
handler_async is a typed cdef attribute, so Cython does not check for None, and the process crashes.
The same race also makes close() itself fail with RuntimeError: deque mutated during iteration or RuntimeError: new callbacks were queued during loop closing.
With the GIL, the whole method runs as one step, so the race cannot happen.
loop.pyx declares freethreading_compatible=True since #693.
This is a different race than #720 (_ready_len, fixed in 0.23.0 by #721).
Reproduce
Save this as uvloop_close_race.py (stdlib and uvloop only):
"""uvloop: call_soon_threadsafe() races with close() on free-threaded Python.
Owner threads run a loop for a moment, then close it. Poster threads call
call_soon_threadsafe() on those loops at the same time. A poster may only get
RuntimeError('Event loop is closed'). It must never crash the process.
python3.14t uvloop_close_race.py [seconds] # free-threaded build: segfault
python3.14 uvloop_close_race.py [seconds] # GIL build: OK
Needs only the standard library and uvloop.
"""
import faulthandler
import sys
import threading
import time
import uvloop
SECONDS = float(sys.argv[1]) if len(sys.argv) > 1 else 20.0
OWNERS = 4
POSTERS = 16
faulthandler.enable()
stop = threading.Event()
slots = [None] * OWNERS
counts = {"loops": 0, "posts": 0, "closed_errors": 0, "close_failures": 0}
def owner(i):
while not stop.is_set():
loop = uvloop.new_event_loop()
slots[i] = loop
loop.call_later(0.0005, loop.stop)
loop.run_forever()
try:
loop.close()
except RuntimeError as exc: # close() itself must not fail either
counts["close_failures"] += 1
print(f"close() failed: {exc!r}", flush=True)
counts["loops"] += 1
def poster(i):
n = errors = 0
while not stop.is_set():
loop = slots[n % OWNERS]
n += 1
if loop is None:
continue
try:
loop.call_soon_threadsafe(int)
except RuntimeError: # 'Event loop is closed' is the expected error
errors += 1
counts["posts"] += n
counts["closed_errors"] += errors
gil = sys._is_gil_enabled() if hasattr(sys, "_is_gil_enabled") else True
print(f"Python {sys.version.split()[0]}, GIL enabled: {gil}, uvloop {uvloop.__version__}", flush=True)
threads = [threading.Thread(target=owner, args=(i,)) for i in range(OWNERS)]
threads += [threading.Thread(target=poster, args=(i,)) for i in range(POSTERS)]
for t in threads:
t.start()
time.sleep(SECONDS)
stop.set()
for t in threads:
t.join()
print(f"OK: no crash in {SECONDS:g} s, {counts}", flush=True)Free-threaded build:
$ python3.14t uvloop_close_race.py 20
Python 3.14.8, GIL enabled: False, uvloop 0.23.0
close() failed: RuntimeError('deque mutated during iteration')
Fatal Python error: Segmentation fault
<Cannot show all threads while the GIL is disabled>
Stack (most recent call first):
File "uvloop_close_race.py", line 52 in poster
...
It crashes in under a second in most runs on an M-series Mac.
GIL build:
$ python3.14 uvloop_close_race.py 20
Python 3.14.8, GIL enabled: True, uvloop 0.23.0
OK: no crash in 20 s, {'loops': 144, 'posts': 25865146, 'closed_errors': 8939073, 'close_failures': 0}
With a debug build, addr2line maps the crash address to Loop.call_soon_threadsafe, the self.handler_async.send() line.
In lldb, self.handler_async is None at that point.
I have a fix and will open a PR.
Disclosure: the analysis, the repro, and the fix were made with Claude Code. The crash itself comes from our own production-like load tests.
Activity