Repository navigation
inspect.getsourcelines() is buggy on 3.12 #104866
Copy link
Copy link
Closed
Labels
3.12only security fixesonly security fixes3.13only security fixesonly security fixestype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Description
Activity
added on May 24, 2023
type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
3.12only security fixesonly security fixes
3.13only security fixesonly security fixes
added on May 24, 2023
stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory
and removed stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory
This is a bug in the implementation of the new tokenize module. The tokenizer checks whether something is in a parenthesized block in order to emit a NL token instead of a NEWLINE token. It also sets and unsets a flag about a comment-only line, in order to do the same. When both are true (a comment inside a parenthesized block), it nevers resets the comment flag. Here's a minimal example:
cpython on main [$?] via C v14.0.3-clang via 🐍 pyenv 3.11.3
❯ cat hello.py
[
# hello
1
]
cpython on main [$?] via C v14.0.3-clang via 🐍 pyenv 3.11.3
❯ ./python.exe -m tokenize hello.py
0,0-0,0: ENCODING 'utf-8'
1,0-1,1: OP '['
1,1-1,2: NL '\n'
2,4-2,11: COMMENT '# hello'
2,11-2,12: NL '\n'
3,4-3,5: NUMBER '1'
3,5-3,6: NL '\n'
4,0-4,1: OP ']'
4,1-4,2: NL '\n' # This should be a NEWLINE instead
5,0-5,0: ENDMARKER '' I'm opening a PR shortly.
added a commit that references this issue on May 24, 2023
added a commit that references this issue on May 24, 2023
Metadata
Metadata
Assignees
Labels
3.12only security fixesonly security fixes3.13only security fixesonly security fixestype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
On 3.12, the last two lines (
a, )) are not returned as part of the source lines. When I tried to minify by removing the first assert_is_value() call, it instead returned the print() line as part of the function.This looks related to the tokenizer, cc @pablogsal @lysnikolaou.
Linked PRs