Skip to content

Handle asynchronous cancellations for HTTPConnection instances #785

Description

@TheTechromancer

The gist of the race condition is that when a web request is cancelled within a specific time window, it will degrade the state of the connection pool, leaving it unable to issue further requests. It also seems to create a deadlock.

It can be reproduced with the following code:

import trio
import httpcore


TIMEOUT = {"read": 5, "write": 5, "connect": 5, "pool": 5}


async def send_request(client, seen_response):
    response = await client.request("GET", "http://127.0.0.1:8000", extensions={"timeout": TIMEOUT})

    # Print the first response that we see and set the "seen_response" flag.
    if not seen_response.is_set():
        print(response)
        seen_response.set()


async def main():
    async with httpcore.AsyncConnectionPool() as client:
        while 1:
            async with trio.open_nursery() as nursery:
                # This event is used to signal the first response we recieve.
                seen_response = trio.Event()

                # Kick off one hundred HTTP requests.
                for idx in range(100):
                    nursery.start_soon(send_request, client, seen_response)
                
                # As soon as we see a response we can cancel out of the nursery,
                # rather than allowing all tasks to complete.
                await seen_response.wait()
                nursery.cancel_scope.cancel()

trio.run(main)

This should produce a httpcore.PoolTimeout error on the second or third iteration, after which the loop enters into a deadlock.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions