Repository navigation
Split up _testcapimodule.c #93649
Description
Modules/_testcapimodule.c is a nearly-8000-line behemoth with no clear structure or organization. It is getting hard to maintain.
It also doesn't work well with testing (a) feature macros that affect Python.h and (b) module initialization, so we have additional C-API testing modules: _testmultiphase, _testimportmultiple, _testinternalcapi.
_testbuffer is already split out, but there are many other aspects of the API that would use a similar dedicated test suite.
We should split and combine these, ideally without polluting the namespace of top-level modules.
Activity
arhadthedev commentedon Jun 9, 2022
Can hypothetical
Lib/_testcapimodule/*.cexport their objects astest.capi.*?encukou commentedon Jun 10, 2022
As far as I know they can, as long as
test.capiitself is not an extension module. It would need some extra support in both make and Windows builds, though.pythongh-93649: Split vectorcall testing from _testcapimodule.c
encukou commentedon Jul 4, 2022
PR: #94549
vstinner commentedon Jul 5, 2022
I tried to group de 299 attributes of the _testcapi extension module. I'm not
sure about each attribute, I only looked quickly at the name, sometimes at the
implementation. But it should give a coarse idea of which "groups" make sense
or not.
Python/getargs.c and Python/modsupport.c (parse and build arguments) (51):
datetime, pytime.c (33):
Types, metaclasses, type inheritance (wide category, not sure if it makes sense) (30):
Function calls and spawn threads to call functions (25):
Unicode, codecs (12):
Memory allocators (17):
Exceptions, "error handling", C "errno" variable and signals (13):
C types limit and size (20):
Float, PyLongObject and PyNumber C API (16):
Docstring (10):
Garbage collector (3):
Tracemalloc (3):
os module helpers (1):
Subinterpreters (1):
C API (57):
PyObject and PyVarObject C API (9):
PyObject_Bytes(), PyObject_Repr(), PyObject_Str() (3):
PyTypeObject C API (2):
C API macros (3):
PyCodeObject C API (2):
PyFrameObject C API (5):
PyDictObject C API (3):
PyThreadState C API (2):
PyListObject C API (1):
PySequence and PyMapping C API, PyObject_GetItem() (6):
C API "feature" macros like HAVE_FORK (1):
HAMT C API (1):
PyBuffer and memoryview C API (5):
PyCapsule C API (1):
PyThread_tss C API (1):
PyStructSeq C API (2):
PyEval_SetTrace C API (1):
PyMarshal C API (6):
Py_AddPendingCall C API (1):
Misc C API (2):
Misc (7):
__doc____file____loader____name____package____spec__vstinner commentedon Jul 5, 2022
It might be interesting to split the large _testcapi module into multiple modules. But I like the idea of starting by splitting the long C file into multiple C files.
gh-93649: Split vectorcall testing from _testcapimodule.c (GH-94549)
tiran commentedon Jul 8, 2022
You do not need to include
_testcapi/vectorcall.cinMODULE__TESTCAPI_DEPS. It is already listed as dependency for_testcapiin the Setup file.pythongh-93649: Fix dependencies of _testcapi
gh-93649: Fix dependencies of _testcapi (#94695)
encukou commentedon Jul 11, 2022
Thank you for the fix! I'll 🔨 test-with-buildbots next time I touch the buildsystem.
(I wonder why this issue didn't get the broken buildbot notification, though...)
tiran commentedon Jul 11, 2022
We think that there was a problem in the buildbot hook, python/buildmaster-config#333
gh-93649: Fix linkage of _PyTestCapi_Init_Vectorcall (#94702)
68 remaining items
pythongh-93649: Move _testcapi tests to specific files
pythongh-93649: Move _testcapi tests to specific files
gh-93649: Move _testcapi tests to specific files (#129544)
pythongh-93649: Move PyFrame C API tests to test_capi (python#129512)
pythongh-93649: Add Modules/_testcapi/type.c file (python#129516)
pythongh-93649: Add Modules/_testcapi/function.c file (python#129521)
pythongh-93649: Move _testcapi tests to specific files (python#129544)
leleliu008 commentedon Feb 20, 2025
@tiran
It appears not to be the same as what you said. https://github.com/python/cpython/blob/main/Modules/Setup#L285
I encounter this:
skirpichev commentedon Feb 20, 2025
You respond to a comment, dated 2022. Maybe something was changed? See Modules/Setup.stdlib.in.
We can't guess how you do this. Even CPython version, which you are using. E.g.:
Anyway, if you encounter a build failure - you should open a new issue. This issue is a wrong place for that.
leleliu008 commentedon Feb 20, 2025
@skirpichev
I run
sed -n -E 's/^#([a-z_\*].*)$/\1/p' Modules/Setup > Modules/Setup.localI built the python-3.13.2
It seem that
Modules/Setupis not keep sync withModules/Setup.stdlib.inskirpichev commentedon Feb 20, 2025
Could you, please, show us where on https://docs.python.org/3/ or https://devguide.python.org/ you have found such instruction?
leleliu008 commentedon Feb 20, 2025
@skirpichev It comes from https://github.com/python/cpython/blob/main/Modules/Setup#L83
skirpichev commentedon Feb 20, 2025
@leleliu008, if you think you have found a bug in CPython, please open an issue.
But first you may want do start a thread in https://discuss.python.org/c/help/7 and describe in details what you are trying to do and why, and what's happened. Please follow to the documentation, if you are trying to build CPython.
leleliu008 commentedon Feb 20, 2025
@skirpichev I'm not sure if this is a bug. It might be due to a misunderstanding of the
Modules/Setup.*mechanism on my part. I couldn't find any detailed documents about the relationship betweenModules/Setup.*, I only found https://devguide.python.org/developer-workflow/extension-modules/#configuring-the-cpython-project , it just motionsModules/Setup.bootstrap.inandModules/Setup.stdlib.in, it doesn't motion others. I guessModules/Setup.localwould override others. I resolved my problem via runninggsed -i "/^_testcapi/d" Modules/Setup.local