Repository navigation
Multithreaded subinterpreters can be running during finalization #126016
Description
Activity
Using .join() will not trigger an assertion error. For the reason, please see the PR submitted by Eric and me.
>>> def silly(): ... interp = _interpreters.create() ... _interpreters.run_string(interp, "import time; time.sleep(100)") ... >>> if __name__ == "__main__": ... Thread(target=silly).start() ... >>> >>> Thread(target=silly).start() >>> Thread(target=silly).start() >>> quit ^CTraceback (most recent call last): File "/root/cpython/Lib/threading.py", line 1540, in _shutdown _thread_shutdown() KeyboardInterrupt: python: Python/pylifecycle.c:2463: finalize_subinterpreters: Assertion `!_PyInterpreterState_IsRunningMain(interp)' failed.>>> def silly(): ... interp = _interpreters.create() ... _interpreters.run_string(interp, "import time; time.sleep(100)") ... >>> t = Thread(target=silly) >>> t.start() >>> t.join() ^CTraceback (most recent call last): File "<python-input-14>", line 1, in <module> t.join() ~~~~~~^^ File "/root/cpython/Lib/threading.py", line 1092, in join self._handle.join(timeout) ~~~~~~~~~~~~~~~~~^^^^^^^^^ KeyboardInterruptYes, I know. The issue is that CTRL+C'ing causes the thread to not finish joining, meaning that the thread state is still marked as running. I'm working on a fix to properly clean up thread states before the interpreter shuts down using pending calls, but daemon threads will probably have to get addressed elsewhere (there's a data race, I think).
If you want, you can insert a signal catcher to catch
SIGINTand do error handling on it. And in the error handling iterate over the list of threads and handle themReacted by RUANG (James Roy)That would result in a thread safety nightmare if the thread was still running :)
Reacted by RUANG (James Roy)#126026 addresses non-daemon threads. It's a bit hacky, though, because I have to manually mess with the status of the thread state.
I think #128640 will fix this as well.
Closing this for now. Ideally we'll get to a point where we can add the assertion back, but that can be tracked elsewhere.
Metadata
Metadata
Assignees
Labels
Projects
- Status
Crash report
What happened?
(Found while investigating a fix for #125920.)
Subinterpreters that are running in a different thread can still be running upon the finalization of the main interpreter, depending on what the thread was doing. Take the following script:
On 3.13 and 3.14, killing the process with CTRL+C results in an assertion failure:
However, if the thread is daemon, then the interpreter fully segfaults:
I'm going to investigate possible fixes.
cc @ericsnowcurrently
CPython versions tested on:
3.13, CPython main branch
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
No response
Linked PRs
PyThreadState_Clear#139158