Repository navigation
3.12 regression: "can't create new thread at interpreter shutdown" from non-daemon threads or atexit handlers #113964
Description
Activity
Can you please provide a full, self-contained reproducer? The provided snippet does not appear to be complete.
Reacted by ShantanuNot easily. The use case is complex, but the error is real, and was not there in 3.10. Maybe you can help me define a case I can get a reproducer for, or help me debug why this is happening now. The process entry point uses importlib.import_module() to load 'external code', starts a thread...
class WorkerThread(threading.Thread): def __init__(self, worker): threading.Thread.__init__(self) self.setDaemon(False) self.stopped = False self.worker = worker self.worker_thread = WorkerThread(self) self.worker_thread.start()
...and starts the timer (which is another thread I guess)...
self.timer = None self.refresh_timer()
...and then, I don't know... exits, but doesn't because the WorkerThread is not a daemon. The WorkerThread uses the the passed in 'worker' to handle all the business logic. Periodically, as in often, the refresh_timer() method will be called from the WorkerThread instance, via the self.worker reference.
The initial timer setup is fine in the 'main' thread, but after that thread finishes (but doe not exit due to the non-daemon thread), the refresh_timer() is called again and fails with "can't create new thread at interpreter shutdown". Maybe this is related to work on multiple interpreters in 3.12?
Explaining this this way has helped in any case, and maybe I can get these classes setup. I think the module load, which is hard to replicate, may have nothing to do with this and I can try with a main thread,, a worker thread, and a timer thread. However, if the answer is obvious from my use case don't hesitate to save me some work and tell me what's up.
Voila...
import threading import time def main(): worker = Worker(); class Worker: def __init__(self): self.worker_thread = WorkerThread(self) self.worker_thread.start() self.timer = None self.refresh_timer() print("worker init complete") def refresh_timer(self): if self.timer is not None: self.timer.cancel() self.timer = threading.Timer(30 * 60, self.stop_cycle) self.timer.start() def stop_cycle(self): print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!") class WorkerThread(threading.Thread): def __init__(self, worker): threading.Thread.__init__(self) self.setDaemon(False) self.stopped = False self.worker = worker def run(self): while True: print("running...") self.worker.refresh_timer() time.sleep(3) main()
C:\Users\david\Documents\testme.py:30: DeprecationWarning: setDaemon() is deprecated, set the daemon attribute instead self.setDaemon(False) running... worker init complete running... Exception in thread Thread-1: Traceback (most recent call last): File "C:\Program Files\Python312\Lib\threading.py", line 1073, in _bootstrap_inner self.run() File "C:\Users\david\Documents\testme.py", line 37, in run self.worker.refresh_timer() File "C:\Users\david\Documents\testme.py", line 21, in refresh_timer self.timer.start() File "C:\Program Files\Python312\Lib\threading.py", line 992, in start _start_new_thread(self._bootstrap, ()) RuntimeError: can't create new thread at interpreter shutdown
If I keep the main entry point running, as with timer.sleep(a_long_time), then I don't get the thread creation error. However, that would force me to throw in some waiting loop on a lock to keep the main thread alive until the non-daemon thread terminates, being very careful to trap any error and release the main thread (try/catch/finally). But none of that makes any difference to me. The whole point of designating the second thread as non-daemon is so the main thread does not exit until all the non-daemon threads terminate, so the system is responsible for all that.
My guess is that whatever throws "can't create new thread at interpreter shutdown" is checking some attribute of the main thread, but failing to check for any running non-daemon threads. Or, whatever code makes up the main thread is responding that yes, it has terminated, despite the fact that non-demon child threads are still running. So I think this is a python bug.
On a first glance this behaviour does seem correct:
main()creates a new non-daemon thread that does the actual work, and then returns. That mains the main thread immediately tries to exit the script and is therefore finalising the interpreter, waiting for theWorkerThreadto stop before actually doing most of the cleanup and exiting the interpreter.If you call
worker.worker_thread.join()inmain()the script will continue to run as long as the worker thread is.I'm not an expert on the finer details of interpreter shutdown though.
Reacted by Zachary Ware and Felix SchefferThe problem is that I want the function to return. I'll agree that a join is better than an event or a barrier, but anything like those will prevent the init function return, which is ugly, But more than that, it's wrong. The first problem with the argument that the main thread should exit the interpreter, but not 'finish' until the non-daemon thread exits, is that this is a significant change in behavior. This works correctly in 3.10 and throws this error in 3.12. That alone is dangerous. The second problem is that it makes no sense. A thread which is 'finalising' in the interpreter has exited as far as any client usage, and the docs on daemon threads say...
A thread can be flagged as a “daemon thread”. The significance of this flag is that the entire Python program exits when only daemon threads are left.
So the 'program' can not exit due to the non-daemon thread, but the main thread kind of can? Even if that wasn't vague enough, if the program has not exited, why can't the live thread create a timer, or start another thread? The program has not exited. There is code running in the interpreter. But it's now blocked from creating a new thread?
The docs should be amended to say...
A thread can be flagged as a “daemon thread”. The significance of this flag is that the entire Python program exits when only daemon threads are left. However, a program running only child non-daemon threads will not be able to create new threads, such as with timer.start().
Does that really make sense?
by the way, def _shutdown() in threading .py, contains this code...
# Join all non-deamon threads while True: with _shutdown_locks_lock: locks = list(_shutdown_locks) _shutdown_locks.clear() if not locks: break for lock in locks: # mimic Thread.join() lock.acquire() lock.release() # new threads can be spawned while we were waiting for the other # threads to complete
Note the comments. I really think the problem is the check in timer.start() which s not properly deciding to check for active non-daemon threads. I'm trying to find that code. Any pointers to it would be helpful.
There should IMHO at least be documentation about this behaviour. I'm therefore reopening the issue.
I've done some research.
Py_FinalizeExfirst sets thefinalizingflag in the interpreter state, the waits for threads to exit:Lines 1823 to 1842 in ac92527
Starting a new thread ends up in
_threading.start_joinable_thead, which in the end callsdo_start_new_threadwhich raises an exception when thefinalizingflag in the interpreter state is set:cpython/Modules/_threadmodule.c
Lines 1249 to 1265 in ac92527
This is unfortunately deep as Timer sub-classes Thread, and the error gets thrown in do_start_new_thread() of _threadmodule.c...
if (interp->finalizing) { PyErr_SetString(PyExc_RuntimeError, "can't create new thread at interpreter shutdown"); return -1; }
So the interpreter is responding that it is finalizing, so I think my contention is either that...
- The interpreter should be blocked from entering the finalizing state until all non-daemon threads have exited. I think that makes sense if there's a single interpreter because there's code still running. My guess is this bug is tied up with the work on allowing multiple interpreters, which makes sense version wise, and the main thread's interpreter is finalizing and there are now other interpreters for the non-daemon threads, but the thread start only checks the main interpreter?
or
- That both the main thread and all non-daemon thread interpreters need to be checked before throwing that error?
There is a specific test for this error being thrown in test_threading.py, but no test that this error is not thrown if there's a non-daemon thread still running.
And yes, what you saw above. Thank you.
21 remaining items
def exit_func(): thread = threading.Thread(target=foo) thread.start() atexit.register(exit_func)
The current PR is focused on spawning a thread during finalization from a main ended garbage collection-ish context. The
atexitcontext may need its own PR as is opens new questions that need specific answers and a bit of different implementation.We need additional changes to make that work safely. It raises the question of "when do we call atexit handlers?"... if they allow code to launch a thread during an atexit handler... it'll run concurrently and must not re-trigger the same atexit handlers to be called when it is the last thread standing and itself exits. do atexit handlers executing prevent the registration of new atexit handlers? people could setup creative loops registering new ones from such a thread... newly registered ones weren't in the first list to be called, they could remain and get called. ... pondering ...
recording a use case from the linked duped issue #115219 directly for reference: TL;DR - Subprocess on windows uses a thread thus cannot be used in atexit context in 3.12 there as
matplotlibdoes:In Matplotlib we are holding onto a latex process and triggering this via a weakref finalize, but a minimal reproducer is:
from subprocess import Popen, PIPE import atexit # anything that holds stdin and stdout should work proc = Popen(['powershell'], stdin=PIPE, stdout=PIPE) def cleanup(): print('hi bob') proc.kill() proc.communicate() atexit.register(cleanup)
- changed the title
[-]3.12 regression: "can't create new thread at interpreter shutdown" from non-daemon threads[/-][+]3.12 regression: "can't create new thread at interpreter shutdown" from non-daemon threads or atexit handlers[/+]on Mar 19, 2024 Just to have another reproducer, we have code that sometimes starts a new thread and sometimes gets called from GC during shutdown (actually from C objects that are destructed). This lead to processes hanging during shutdown.
Code:
from threading import Thread class CleanupOnDelete: def __del__(self): print("Starting thread due to deletion") thread = Thread(target=lambda: None) thread.start() print("Thread started") thread.join() print("Thread joined") foo = CleanupOnDelete()
This used to work with Python 3.7, lead to interpreter hangs in intermediate versions and not raises an exception:
$ docker run -it --rm -v "$PWD":/src -w /src python:3.7 python exit_gc_hang.py Starting thread due to deletion Thread started Thread joined $ docker run -it --rm -v "$PWD":/src -w /src python:3.8 python exit_gc_hang.py Starting thread due to deletion <hangs...> $ docker run -it --rm -v "$PWD":/src -w /src python:3.10 python exit_gc_hang.py Starting thread due to deletion <hangs...> $ docker run -it --rm -v "$PWD":/src -w /src python:3.11 python exit_gc_hang.py Starting thread due to deletion <hangs...> $ docker run -it --rm -v "$PWD":/src -w /src python:3.12 python exit_gc_hang.py Starting thread due to deletion Exception ignored in: <function CleanupOnDelete.__del__ at 0x7f83834f6200> Traceback (most recent call last): File "/src/exit_gc_hang.py", line 7, in __del__ File "/usr/local/lib/python3.12/threading.py", line 992, in start RuntimeError: can't create new thread at interpreter shutdown $ docker run -it --rm -v "$PWD":/src -w /src python:3.13-rc-bookworm python exit_gc_hang.py Starting thread due to deletion Exception ignored in: <function CleanupOnDelete.__del__ at 0x7fd02488d760> Traceback (most recent call last): File "/src/exit_gc_hang.py", line 7, in __del__ File "/usr/local/lib/python3.13/threading.py", line 971, in start PythonFinalizationError: can't create new thread at interpreter shutdown
We can actually live with getting an exception, but it is difficult to deal with processes stuck in shutdown.
Currently, we are guarding the thread creation with a checkif main_thread().is_alive().You can try threading.Thread.join() function to block the main thread
Bug report
Bug description:
I have a single threading.Timer held on a class as self.timer...
As recently as 3.10 this code worked. As of 3.12 it errors with "can't create new thread at interpreter shutdown". The code has not changed, and the interpreter is not shutting down at the time this error is thrown, as in the process is live and code is executing. There are no code changes from where this is working in 3.10 to where it is not in 3.12.1 (upgraded from 3.12 to 3.12.1 and tsted again to see if this had been fixed).
CPython versions tested on:
3.12
Operating systems tested on:
Windows
Linked PRs