Documentation doesn't generate with docutils >= 0.22

Documentation

The html doc is not building with the new docutils >= 0.22. It's not a problem yet, because the current Sphinx used (sphinx~=8.2.0) doesn't install the latest one, but it's something that should be fixed for the future.

The problem can be reproduced:

$ cd Doc
$ python3 -m venv env
$ source env/bin/activate
$ pip install -r requirements.txt
$ pip install -U docutils
$ sphinx-build --pdb -b html . build/html
[...]
      File "env/lib64/python3.13/site-packages/docutils/parsers/rst/states.py", line 1494, in parse_enumerator
        ordinal = int(self.enum.converters[sequence](text))
    TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'

The actual problem is directly related to this line in docutils, done as part as this commit.

They wrap the converters with int so the monkey patch done in pyspecific.py doesn't work anymore.

# monkey-patch reST parser to disable alphabetic and roman enumerated lists
from docutils.parsers.rst.states import Body
Body.enum.converters['loweralpha'] = \
    Body.enum.converters['upperalpha'] = \
    Body.enum.converters['lowerroman'] = \
    Body.enum.converters['upperroman'] = lambda x: None

Linked PRs

pythongh-139257: Support docutils >= 0.22

pythongh-139257: Add news entry

I don't think we should actually worry about a docutils version that is not installed by the environment itself. I would rather pin the docutils version in constraints.txt instead of re-patching conf.py directly. It might not be the only issue we'll have if we upgrade to newest Sphinx.

Is there a reason why the docutils version is actually being updated after installing the requirements? maybe we should pin the docutils version anyway so that it's compatible with our conf.py (having version-guarded ifs always complicates maintenance).

I don't think we should actually worry about a docutils version that is not installed by the environment itself. I would rather pin the docutils version in constraints.txt instead of re-patching conf.py directly. It might not be the only issue we'll have if we upgrade to newest Sphinx.

Is there a reason why the docutils version is actually being updated after installing the requirements? maybe we should pin the docutils version anyway so that it's compatible with our conf.py (having version-guarded ifs always complicates maintenance).

This problem appears in openSUSE where we are trying to update docutils, and we can patch downstream to keep it working. But as I said, this is not a problem right now, but will be a problem in the future, so I tried to work on a fix for it.

Ah I see in the logs

[ 7s] [169/192] cumulate python313-docutils-0.22.2-1.2

So the venv you're using for building the docs on openSUSE is actually using the latest docutils and the latest Sphinx (for some reasons) instead of those given in Doc/requirements.txt. How easy is it to actually use those instead? I would prefer pinning the versions in Doc/requirements.txt and Doc/constraints.txt instead as other users might benefit from this locally. If this is not easy, I can indeed accept the docutils patch (other distros might have the same issue)

That is the main issue. We build cpython in openSUSE with the current packages, everything installed from the rpm repository so there's no venv during build. That's why if we want to update docutils we need to patch cpython to work correctly with the new version.

For documentation usually the Sphinx version is the problem and we are more conservative when updating Sphinx, but in this case, it's docutils and this specific monkey-patch that doesn't work with the current version.

The patch is not very complex and it's compatible with the previous versions, so I though that it could be interesting to report and fix upstream.

@AA-Turner

Mmh. Are you aware of other distributors that could have a similar issue? I don't know much about their process. If all other major distributors also don't use our venv, then I think we could allow this patch (or we can also change upstream docutils, cc @AA-Turner).

Considering the issue, I think I'd be fine with the version-guarded patch.

We don't support building the documentation with exotic configurations/environments. When we update to a newer version of Sphinx, we'll address this. Thank you for raising the issue, though.

A

Mmh. Are you aware of other distributors that could have a similar issue? I don't know much about their process. If all other major distributors also don't use our venv, then I think we could allow this patch (or we can also change upstream docutils, cc @AA-Turner).

I think this will be the problem for all Linux distributions (and others like *BSD, Homebrew on Mac, etc.), only some of them have so far old docutils (e.g., Fedora is still on 0.21.2, Debian will hits this in experiemental).

@AA-Turner

@AA-Turner I think we could make the change here otherwise it could complicate redistribution. We're already somehow monkey-patching docutils so maybe it doesn't hurt to monkey-patch it better (the fix looks fine, and instead of an ImportError, we could use docutils' version guards instead).

Update python39 to version 3.9.23 / rev 80 via SR 1308251

Update python312 to version 3.12.11 / rev 38 via SR 1308661

Coming here from Debian which is hitting the same issue:

Are you aware of other distributors that could have a similar issue?

I would expect all Linux distributions to run into this. We build self-contained systems. This typically requires things to work with the latest versions of each other. We can hold something at an older version for a while while we figure out an issue, but pinning anything to an ancient version is not possible.

pythongh-139257: docutils 0.22 support

@AA-Turner

I see. @AA-Turner The fix on our side is not really hard and I think this would help getting the latest python on mainstream distros. Would you be willing to reconsider your position here?

How is anyone maintaining documentation supposed to know what needs escaping? These escapes are unnatural.

What checks do we have in place that prevent docs from being checked in that conflict with unsupported-by-us future versions of docutils? Unless we have linting and CI checks that make these very-awkward escaping rules discoverable in our docs maintenance I expect such a PR will be temporary at best and regress in the future.

That doesn't mean we should not simply apply the PR for practical purposes - but nobody can expect our doc builds to stay working for the long term on arbitrary not-what-we-use docutils version without a CI test matrix in place.

How is anyone maintaining documentation supposed to know what needs escaping?

My PR #139257 removed the monkeypatch, because things were escaped, but it could instead raise an exception, as I proposed on it.

What checks do we have in place that prevent docs from being checked in that conflict with unsupported-by-us future versions of docutils?

From my PoV, I don't think that's something you really need to worry about. If things break loudly (like this) people will come and fix them. If they break silently, maybe you won't hear for a while, but you'll have the same silence problem when you make the docutils upgrade yourself.

Python could, of course, have a CI job that builds the docs with upstream unpinned versions of things, that'd probably be nice. But I'm not a Python Core developer at this point, so I'm not going to propose that you take on the burden of maintaining that.

Late to the party, but coming from Fedora with the same issue - also confirming that pinning is not really an option for us. I managed to fix or backport the fixes to all packages broken with docutils 0.22 in Fedora, the ecosystem seems to be pretty prepared for it. For me, it's just the Python docs left.

pythonGH-139257 Use an explicitly named link

Since this issue has been addressed in #121970, I'm going to close this it. We went for an alternative change that doesn't monkeypatch docutils.

We also fixed this in #142057.

Navigation Menu

Repository navigation

Documentation doesn't generate with docutils >= 0.22 #139257

Documentation

Linked PRs

3 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Documentation doesn't generate with docutils >= 0.22 #139257

Description

Documentation

Linked PRs

Activity

picnixz commented on Sep 23, 2025

danigm commented on Sep 23, 2025

picnixz commented on Sep 23, 2025

danigm commented on Sep 23, 2025

picnixz commented on Sep 23, 2025

AA-Turner commented on Sep 23, 2025

mcepl commented on Sep 24, 2025

picnixz commented on Sep 24, 2025

3 remaining items

stefanor commented on Oct 13, 2025

picnixz commented on Oct 13, 2025

gpshead commented on Oct 13, 2025

stefanor commented on Oct 14, 2025

befeleme commented on Nov 24, 2025

picnixz commented on Nov 29, 2025

picnixz commented on Nov 29, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions