Skip to content

datetime.fromisoformat() (C) drops the sub-second part of a UTC offset, breaking round-trips #152079

Description

@tonghuaroot

Bug report

Bug description:

The C implementation of datetime.fromisoformat() silently drops the sub-second part of a
UTC offset whenever the offset's whole-second part is zero, returning timezone.utc. This
breaks round-trips: a value the module itself produces via isoformat() is not parsed
back faithfully.

>>> from datetime import datetime, timezone, timedelta
>>> dt = datetime(2020, 6, 15, 12, 34, 56, tzinfo=timezone(timedelta(microseconds=1)))
>>> s = dt.isoformat()
>>> s
'2020-06-15T12:34:56+00:00:00.000001'
>>> datetime.fromisoformat(s).utcoffset()      # C accelerator
datetime.timedelta(0)                          # the 1-microsecond offset is gone

The pure-Python implementation is correct:

>>> import _pydatetime
>>> _pydatetime.datetime.fromisoformat(s).utcoffset()
datetime.timedelta(microseconds=1)

The negative case '...-00:00:00.000001' is dropped the same way.

This is not a non-standard format: isoformat() is documented to emit
+HH:MM[:SS[.ffffff]] and fromisoformat() is documented to accept it ("Time zone
offsets may have fractional seconds"), so this is documented round-trippable data being
silently dropped.

Root cause

Modules/_datetimemodule.c, tzinfo_from_isoformat_results() short-circuits to UTC on the
whole-second offset alone, ignoring the parsed sub-second component:

// Create a timezone from offset in seconds (0 returns UTC)
if (tzoffset == 0) {
    return Py_NewRef(CONST_UTC(NO_STATE));
}

When tzoffset == 0 but tz_useconds != 0, the sub-second part is discarded. The
pure-Python implementation checks all offset components before collapsing to UTC.

Suggested fix

Only short-circuit to UTC when both the whole-second and sub-second parts are zero:

if (tzoffset == 0 && tz_useconds == 0) {
    return Py_NewRef(CONST_UTC(NO_STATE));
}

The fall-through path already builds the correct timezone(timedelta(...)). A plain
+00:00 offset still returns timezone.utc. I confirmed against a full C-vs-pure-Python
differential that the only inputs whose behaviour changes are exactly these zero-whole-
second sub-second offsets, with no other divergence introduced. I have a patch and a
round-trip regression test (running under both implementations) ready.

This is a sibling to #152060, a separate fromisoformat() defect in the pure-Python
implementation.

CPython versions tested on:

3.16 (main, built from source)

Operating systems tested on:

macOS

Linked PRs

Activity

  1. StanFromIreland commented on Jun 24, 2026

    @StanFromIreland
    Member

    This is a sibling to #152060, a separate fromisoformat() defect in the pure-Python
    implementation.

    Well, now we know we're not just bad at writing Python, we also bad at writing C ;-)


    Out of curiosity, how are you finding all these issues? Is it fuzzing with hypothesis or something like that, or LLM analysis?

  2. added a commit that references this issue on Jun 25, 2026
  3. tonghuaroot commented on Jun 25, 2026

    @tonghuaroot
    ContributorAuthor

    Ha — honestly the bugs are almost always in the seam between the two implementations rather than in either one alone, so I'll let the C off the hook ;-)

    It's basically differential testing: _datetimemodule.c and _pydatetime are meant to behave identically, so I feed both the same edge cases (offset / fraction / separator boundaries, and round-trips of strings the module emits itself) and wherever they disagree, one side has a bug. Two independent implementations of the same spec is the gift that keeps giving.

    And yeah — AI-assisted (Claude Code) for surfacing and triaging the candidate divergences, all checked against a debug build before I file. Not pure fuzzing, though "C ≡ pure-Python" is basically the property you'd hand to hypothesis. Happy to share the harness if it'd be useful.

  4. StanFromIreland commented on Jun 25, 2026

    @StanFromIreland
    Member

    Please share! We've considered doing similar things before in https://github.com/python/cpython/pull/124550/changes.

  5. added 3 commits that reference this issue on Jun 25, 2026
  6. tonghuaroot commented on Jun 25, 2026

    @tonghuaroot
    ContributorAuthor

    Happy to! Here's the harness, cleaned up as a standalone gist:

    https://gist.github.com/tonghuaroot/b2a17a17c5c30ae7b8b4667c570c45b1

    It's the C-vs-pure-Python differential — the same idea as the strftime grammar
    in #124550, but pointed at the literal ISO-8601 surface fromisoformat
    consumes. It builds the corpus from a small ISO-8601 component grammar
    (OneFrom / EachFrom / Optional) and exhaustively enumerates the bounded
    product — date / time / fraction / offset forms, separators, surrounding
    whitespace, non-ASCII digits, plus a handful of NUL / surrogate / very-long
    inputs — then feeds every string to all three constructors on both modules and
    groups whatever disagrees. No third-party dependency; just run it on a normal
    build (_datetime vs _pydatetime).

    On a current build it re-finds the two already reported — #152079 (the
    sub-second-offset drop) and #152157 (empty fraction before a tz) — and turns up
    a few more I hadn't seen filed:

    1. pure-Python date.fromisoformat silently mis-parses some malformed
      basic-format dates
      — returns a wrong date instead of raising:
      date.fromisoformat('2020+12') → date(2020, 1, 2) and
      date.fromisoformat('2020061') → date(2020, 6, 1). The C path rejects
      both; it's the fixed-width int(dtstr[...]) slices in _parse_isoformat_date
      accepting a +/space or a too-short tail.
    2. C accepts a space before the tz designator when there's no fraction:
      time.fromisoformat('12:34:56 Z') parses on C, raises on pure-Python —
      looks like the no-fraction sibling of the datetime: pure Python implementation of fromisoformat() handles times with trailing spaces inconsistently with the C extension #130959 fix.
    3. C truncates at an embedded NUL in a bare time:
      time.fromisoformat('12:34:56\x00') → time(12, 34, 56) on C, ValueError
      on pure-Python (narrow — trailing NUL after a complete value).

    I went ahead and opened the first one as #152204, with a fix in #152205, since a
    silent wrong value felt worth nailing down. The other two I'm happy to file as
    well — or if you'd rather fold them into your own fixes / take them however
    suits, that's totally fine, whatever's least work.

    And on the test side: I also have the parity check written up as a differential
    unittest (a whole-corpus expectedFailure plus a per-divergence method that
    flips into a regression guard once each is fixed). If it'd be useful, I'd be
    glad to shape it into the differential class for
    Lib/test/test_datetime_property.py in #124550 — just say how you'd like it
    laid out and I'll match the surrounding style.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    extension-modulesC modules in the Modules dirtype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions