Repository navigation
Description
Checklist
- The bug is reproducible against the latest release and/or
master. - There are no similar issues or pull requests to fix it yet.
Describe the bug
I alluded to this in #1463. That issue is about the type of exception being raised; this bug is about the source of the exception being raised.
In this code block:
async with httpx_client.stream("GET", url) as response:
async for blob in response.aiter_raw():
pass
# ... Now, imagine `aiter_raw()` raises an exception...
# ... and imagine `response.aclose()` raises an exception within `__aclose__()`.
The exception from aiter_raw() is caught by __aclose__(); the exception from response.aclose() is the one I see.
To reproduce
See #1463 for example code that causes a "Connection reset" error within the aiter_raw() block.
Expected behavior
I expect to see a stack trace with aiter_raw() in it, because aiter_raw() caused an exception. (Sure, response.aclose() also throws an exception; but as a user, do I really care?)
Actual behavior
The stack trace is for the error in response.aclose(). aiter_raw() doesn't appear in it.
Debugging material
See #1463
Environment
- OS: Linux
- Python version: 3.8.7
- HTTPX version: 0.16.1
- Async environment: asyncio
- HTTP proxy: no
- Custom certificates: no
Additional context
As a workaround, this code block will raise the correct exception:
request = client.build_request("GET", f"http://{host}:{port}")
response = await client.send(request, stream=True)
try:
async for blob in response.aiter_raw():
pass
except Exception as err:
try:
await response.aclose()
finally:
raise err
... this uses the inside knowledge that aiter_raw() calls aclose() to handle these three cases:
- Exception while streaming and exception from
aclose(): ignore the exception fromaclose()and raise the one fromaiter_raw(). - Exception while streaming and success from
aclose(): raise the exception fromaiter_raw(). - Streaming succeeds but exception in
aiter_raw()'s call toaclose(): we'll callaclose()again in our exception handler; it will no-op (because it's a second invocation); and we'll raise the exception from the original call toaclose().
I've been using Python for decades, and I feel very uncomfortable nesting exception handlers like this.
Activity
florimondmanca commentedon Feb 12, 2021
@adamhooper I'm not sure yet I fully understood the problem here.
Here's a sample script to try and simulate the situation I think you're referring to:
The output from this is:
Sync:
Async:
We already have #1465 to fix the leaking
httpcore.CloseError.But from as far as I can see, we do see errors from
iter_raw()as well as errors from.close(). They're included as causes. Right? What else would you expect to see?adamhooper commentedon Feb 12, 2021
The error comes when you want to catch the error.
I'd expect to catch
httpx.ReadErrorin this case, nothttpx.CloseError.florimondmanca commentedon Feb 12, 2021
Hmm I see. Well unfortunately Python only follows a "single-exception-at-a-time" design, so you'd need to catch errors as close to where they happen as possible.
So what you could do as a workaround is…
Which outputs the following, and doesn't look too bad, does it?
adamhooper commentedon Feb 12, 2021
Sure -- there's also the workaround I mentioned above.
When should users use the approach laid out in the documentation, and when should users use one of these workarounds that raises the exception users want?
euri10 commentedon Feb 12, 2021
if this lands could be helpful
florimondmanca commentedon Feb 16, 2021
@tomchristie Heya :-) Any thoughts on this one, or #1465?
lovelydinosaur commentedon Apr 14, 2021
Yup, we really ought to be handling this neatly.
We could for instance do something like this...
Although it's not immediately clear to me if that's what we ought to be doing, ...or... if we should be ensuring something similar, but inside
httpcore.stale commentedon Feb 20, 2022
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
lovelydinosaur commentedon Feb 21, 2022
Probably still valid thanks @Stale - needs looking into.
stale commentedon Mar 25, 2022
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
stale commentedon Oct 15, 2022
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
lovelydinosaur commentedon Dec 8, 2022
I believe this was closed by encode/httpcore#310