Repository navigation
Documentation is unclear how "y*" and "y#" format units vary #74810
Description
| BPO | 30625 |
|---|---|
| Nosy | @skrah, @vadmium, @indygreg |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2017-06-10.18:16:12.023>
labels = ['type-bug', '3.7', 'docs']
title = 'Documentation is unclear how "y*" and "y#" format units vary'
updated_at = <Date 2017-06-17.03:18:44.489>
user = 'https://github.com/indygreg'bugs.python.org fields:
activity = <Date 2017-06-17.03:18:44.489>
actor = 'martin.panter'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2017-06-10.18:16:12.023>
creator = 'indygreg'
dependencies = []
files = []
hgrepos = []
issue_num = 30625
keywords = []
message_count = 5.0
messages = ['295649', '296010', '296116', '296182', '296229']
nosy_count = 4.0
nosy_names = ['skrah', 'docs@python', 'martin.panter', 'indygreg']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30625'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']
Activity
indygreg commentedon Jun 10, 2017
indygreg/python-zstandard#26 is a bug report against python-zstandard that a C API using the "y#" format unit is unable to accept a memoryview instance constructed from a bytes literal. It fails with "TypeError: argument 1 must be read-only bytes-like object, not memoryview".
I understand why the "y*" format unit and the buffer protocol are a superior API. In hindsight, I should have used the buffer protocol from the beginning in python-zstandard. However, this decision was primarily influenced because the docs aren't clear about the apparent limitations of "y#" compared to "y*" and I believed "y#" would get me what I wanted (pointer to read-only memory) without having to be burdened by the complexity of the buffer protocol in my C code.
So, docs issue #1 is that the limitations of "y#" compared to "y*" aren't immediately obvious when reading their respective sections in https://docs.python.org/3/c-api/arg.html#strings-and-buffers. To their credit, the docs for "y*" do have a bold recommendation to use it. But what's missing is the critical "why." There is also a paragraph above the format unit list explaining "bytes-like object" but this is detached from the definitions for "y#" and "y*" so it is easy to pass over.
Issue #2 (which may be an implementation bug) is why "y#" isn't accepting a memoryview constructed from bytes. The docs for "y#" say it accepts a "read-only bytes-like object," which is defined at https://docs.python.org/3/glossary.html#term-bytes-like-object. And the paragraphs there explicitly state that a memoryview of an immutable bytes is in fact a "read-only bytes-like object." So I have no clue why "y#" is refusing such an object.
I'll gladly author a documentation fix. However, I'm not sure what to say because I don't understand why "y#" isn't working in this memoryview(bytes) case and whether that issue applies to other types.
[-]Documentation is unclear how "y*" and "y#" formit units vary[/-][+]Documentation is unclear how "y*" and "y#" format units vary[/+]skrah commentedon Jun 14, 2017
Out of curiosity:
Is the 3.2 documentation clearer?
https://docs.python.org/3.2/c-api/arg.html#strings-and-buffers
Lately we have a lot of churn in the docs, not necessarily written by subject experts.
indygreg commentedon Jun 15, 2017
IMO I don't find the 3.2 docs more useful. Specifically, the behavior for memoryview is still unclear.
skrah commentedon Jun 16, 2017
Okay thanks, it's good to hear what others think about the docs.
"y#" is refusing memoryview(bytes) because "y#" only allows objects without a releasebufferproc and memoryview itself always has one.
I wonder if we could simply use the cleanup() solution also for "y#", but I have to look closer at this.
vadmium commentedon Jun 17, 2017
bpo-24009 proposes deprecating y# (among other units).
IMO the documentation and error message aren’t specific enough regarding the reference to “bytes-like”.
methane commentedon Sep 30, 2026
#98710 documents the bf_releasebuffer restriction, including why memoryview(bytes) is rejected by borrowed-buffer formats. Can this documentation issue be closed?