Skip to content

PEP 632: Remove the distutils package (Python 3.12) #92584

Closed
Closed
PEP 632: Remove the distutils package (Python 3.12)#92584
Labels
3.12only security fixestype-bugAn unexpected behavior, bug, or error

Description

@vstinner
vstinner
opened on May 9, 2022

The distutils module was deprecated in Python 3.10 and is scheduled for removal in Python 3.12.

Right now, Python still uses distutils for different purpose:

  • Build C extensions which can not be built by Makefile yet. Python 3.11 can build way more C extensions with Makefile than Python 3.10.
  • peg_generator uses distutils to build C extensions.
  • test_cppext uses distutils to check that the Python C API can be used in C++.
  • Tools/c-analyzer/ uses distutils to get a C preprocessor.

Until Python can be built and used without distutils, I propose to first rename the distutils package to _distutils: rename th Lib/distutils/ directory to Lib/_distutils/.

Right now, test_venv fails without distutils, because pip fails on importing the distutils module in pip/_internal/locations/_distutils.py: https://github.com/pypa/pip/blob/cb24fb4052ca8ab8009866b0de61980c81a7e13c/src/pip/_internal/locations/_distutils.py#L9-L12

I created the PR #92585 which renames distutils to _distutils.

Activity

  1. vstinner
    added
    type-bugAn unexpected behavior, bug, or error
    on May 9, 2022
  2. vstinner
    changed the title [-]Remove the distutils package to _distutils[/-] [+]Rename the distutils package to _distutils[/+] on May 9, 2022

  3. vstinner commented on May 9, 2022

    @vstinner
    vstinner
    on May 9, 2022
    MemberAuthor

    Right now, test_venv fails without distutils, because pip fails on importing the distutils module in pip/_internal/locations/_distutils.py

    I reported this issue to pip: pypa/pip#11103

  4. vstinner commented on May 9, 2022

    @vstinner
    vstinner
    on May 9, 2022
    MemberAuthor

    The distutils package was deprecated by the issue #85454.

  5. vstinner
    changed the title [-]Rename the distutils package to _distutils[/-] [+]PEP 632: Rename the distutils package to _distutils[/+] on May 10, 2022

  6. vstinner commented on May 10, 2022

    @vstinner
    vstinner
    on May 10, 2022
    MemberAuthor

    PEP 632 "Deprecate distutils module": https://peps.python.org/pep-0632/

  7. FFY00 commented on May 10, 2022

    @FFY00
    FFY00
    on May 10, 2022
    Member

    That seems like a good plan. Thanks.

  8. jaraco commented on May 10, 2022

    @jaraco
    jaraco
    on May 10, 2022
    Member

    My fear with renaming it is that it lingers indefinitely, and diverging in subtle ways from pypa/distutils. Why not address the dependencies and fully remove the functionality?

    Regardless, I agree renaming is preferable to not removing the module at all.

  9. vstinner commented on May 10, 2022

    @vstinner
    vstinner
    on May 10, 2022
    MemberAuthor

    Why not address the dependencies and fully remove the functionality?

    Someone has to modify Python to no longer depend on distutils. The issue description lists code using it. It doesn't seem trivial.

  10. FFY00 commented on May 10, 2022

    @FFY00
    FFY00
    on May 10, 2022
    Member

    Would it be reasonable to keep it around in the repo while we don't drop the dependency but not ship it as part of Python installations?

  11. vstinner commented on May 10, 2022

    @vstinner
    vstinner
    on May 10, 2022
    MemberAuthor

    Would it be reasonable to keep it around in the repo while we don't drop the dependency but not ship it as part of Python installations?

    With my PR, the Lib/_distutils/ directory is not installed by make install.

  12. jkloth commented on May 10, 2022

    @jkloth
    jkloth
    on May 10, 2022
    Contributor

    Do note that we have buildbots that run from an installed Python. The tests would need to also be modified to fail gracefully
    (or skip) when _distutils is not available.

  13. vstinner commented on May 10, 2022

    @vstinner
    vstinner
    on May 10, 2022
    MemberAuthor

    Do note that we have buildbots that run from an installed Python. The tests would need to also be modified to fail gracefully (or skip) when _distutils is not available.

    I propose skipping tests relying on _distutils if _distutils is not available.

  14. vstinner commented on May 10, 2022

    @vstinner
    vstinner
    on May 10, 2022
    MemberAuthor

    I wrote two PRs to avoid the deprecated distutils module in two tests:

  15. vstinner
    added a commit that references this issue on May 10, 2022

    gh-92584: test_decimal uses shutil.which() (#92640)

    dfdebda

  16. 75 remaining items

  17. ambv
    added a commit that references this issue on Aug 22, 2023

    GH-92584: Remove distutils from the newtypes tutorial includes (#108024)

    e97b7be
  18. miss-islington
    added a commit that references this issue on Aug 22, 2023

    pythonGH-92584: Remove distutils from the newtypes tutorial includes (p…

    1be1eb8
  19. Yhg1s
    added a commit that references this issue on Aug 22, 2023

    [3.12] GH-92584: Remove distutils from the newtypes tutorial includes (…

    200af42
  20. ppigazzini
    added 6 commits that reference this issue on Aug 28, 2023

    Replace deprecated `distutils` package

    e2561df

    Replace deprecated `distutils` package

    86e63d4

    Replace deprecated `distutils` package

    0259598

    Replace deprecated `distutils` package

    622c1c7

    Replace deprecated `distutils` package

    5410bb0

    Replace deprecated `distutils` package

    87317f7
  21. Yhg1s
    added a commit that references this issue on Oct 2, 2023

    GH-92584: Move installation schemes overview to sysconfig docs (#108018)

    f16e81f
  22. miss-islington
    added a commit that references this issue on Oct 2, 2023

    pythonGH-92584: Move installation schemes overview to sysconfig docs (p…

    1d46452
  23. AA-Turner
    added a commit that references this issue on Oct 2, 2023

    [3.11] pythonGH-92584: Move installation schemes overview to sysconfi…

    d36c5cf
  24. Yhg1s
    added a commit that references this issue on Oct 2, 2023

    [3.12] GH-92584: Move installation schemes overview to sysconfig docs (…

    48afd1c
  25. AA-Turner
    added a commit that references this issue on Jan 11, 2024

    [3.11] GH-92584: Move installation schemes overview to sysconfig docs (…

    71288ed
  26. Glyphack
    added a commit that references this issue on Sep 2, 2024

    pythonGH-92584: Move installation schemes overview to sysconfig docs (p…

    b4e318f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.12only security fixestype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions