Repository navigation
RetryPolicy resends an exhausted generator body as empty after a 503 #49587
Description
- Package Name: azure-core
- Package Version: 1.41.0
- Operating System: Linux (Pop!_OS 24.04)
- Python Version: 3.12.3
Describe the bug
When a request body is a generator or async generator (HttpRequest(content=<generator>)), RetryPolicy and AsyncRetryPolicy retry a transient failure by sending the same generator again. The first attempt has already consumed it, so the retry goes out with an empty body. If the retry succeeds, pipeline.run returns 200 and the caller has no sign that an empty or truncated body was stored.
To Reproduce
A fake transport answers 503 then 200 and records the bytes it reads from the body on each attempt.
import asyncio
from azure.core.pipeline import AsyncPipeline, Pipeline
from azure.core.pipeline.policies import AsyncRetryPolicy, RetryPolicy
from azure.core.pipeline.transport import AsyncHttpTransport, HttpTransport
from azure.core.rest import HttpRequest
from azure.core.rest._http_response_impl import HttpResponseImpl
from azure.core.rest._http_response_impl_async import AsyncHttpResponseImpl
class _Internal:
def __init__(self, status):
self.status_code = status
self.reason = "OK" if status == 200 else "Service Unavailable"
class FakeTransport(HttpTransport):
def __init__(self, statuses):
self.statuses = list(statuses)
self.bodies = []
def send(self, request, **kwargs):
self.bodies.append(b"".join(request.content))
status = self.statuses.pop(0)
return HttpResponseImpl(
request=request, internal_response=_Internal(status), block_size=1024,
status_code=status, reason="", content_type=None, headers={}, stream_download_generator=None,
)
def __enter__(self):
return self
def __exit__(self, *args):
pass
def open(self):
pass
def close(self):
pass
class FakeAsyncTransport(AsyncHttpTransport):
def __init__(self, statuses):
self.statuses = list(statuses)
self.bodies = []
async def send(self, request, **kwargs):
self.bodies.append(b"".join([chunk async for chunk in request.content]))
status = self.statuses.pop(0)
return AsyncHttpResponseImpl(
request=request, internal_response=_Internal(status), block_size=1024,
status_code=status, reason="", content_type=None, headers={}, stream_download_generator=None,
)
async def __aenter__(self):
return self
async def __aexit__(self, *args):
pass
async def open(self):
pass
async def close(self):
pass
def sync_case():
def body():
yield b"part1"
yield b"part2"
transport = FakeTransport([503, 200])
pipeline = Pipeline(transport, policies=[RetryPolicy(retry_backoff_factor=0)])
response = pipeline.run(HttpRequest("PUT", "https://example.invalid/blob", content=body())).http_response
print(f"sync : bodies seen by transport = {transport.bodies}, final status = {response.status_code}")
async def async_case():
async def body():
yield b"part1"
yield b"part2"
transport = FakeAsyncTransport([503, 200])
pipeline = AsyncPipeline(transport, policies=[AsyncRetryPolicy(retry_backoff_factor=0)])
result = await pipeline.run(HttpRequest("PUT", "https://example.invalid/blob", content=body()))
print(f"async: bodies seen by transport = {transport.bodies}, final status = {result.http_response.status_code}")
sync_case()
asyncio.run(async_case())Output:
sync : bodies seen by transport = [b'part1part2', b''], final status = 200
async: bodies seen by transport = [b'part1part2', b''], final status = 200
Expected behavior
A body that cannot be rewound is not retried, so the 503 comes back to the caller, the same as for a non-seekable file-like body. The second attempt should never go out with an empty body and report success.
Screenshots
N/A
Additional context
Cause, at main (c00962557a): RetryPolicyBase.increment in sdk/core/azure-core/azure/core/pipeline/policies/_retry.py only checks whether the body can be rewound when it has a read attribute (line 325). A generator or async generator, which set_content_body accepts (rest/_helpers.py line 146), skips that check and reaches return True at line 346. _configure_positions records no position for such a body either. AsyncRetryPolicy calls the same increment (_retry_async.py lines 180 and 196), so the async path behaves the same.
Proposed fix: in increment, return False when the body is an iterator or async iterator that cannot be rewound, that is anything with __iter__ or __aiter__ that is not bytes, str or seekable. Add sync and async tests with generator content and one 503.
I'm happy to open a PR with the fix.
Activity
github-actions commentedon Oct 17, 2026
Thank you for your feedback. Tagging and routing to the team member(s) best able to assist.
github-actions commentedon Oct 17, 2026
🎯 Agentic Issue Triage
Summary:
RetryPolicy/AsyncRetryPolicyin azure-core retry a request whose body is a generator/async generator by resending the same (now-exhausted) iterator, so a retried request after a transient failure (e.g. 503) goes out with an empty body; if that retry returns 200, the caller sees success despite an effectively empty/truncated upload.📋 Issue Details
Package:
azure-core1.41.0Affected API:
RetryPolicyBase.increment/_configure_positionsinsdk/core/azure-core/azure/core/pipeline/policies/_retry.py; consumed identically byRetryPolicy(sync) andAsyncRetryPolicy(_retry_async.py)Scenarios:
HttpRequest(content=<generator>)PUT request hits a transient 503, policy retries, generator body has already been fully consumed on attempt 1, so attempt 2 sends an empty bodyAsyncRetryPolicyreadbut not seekable), which are correctly not retried todayRoot ask: Make
incrementtreat non-seekable iterator/async-iterator bodies (generators) the same as non-seekable file-like bodies — i.e., returnFalse(don't retry) instead of silently resending an empty body. Reporter offered to submit a PR.🔎 Debugging / Reproduction Notes
Issue includes a complete, self-contained repro script using a fake sync/async transport that records bytes read per attempt, demonstrating
bodies seen by transport = [b'part1part2', b'']with a final reported status of 200 for both sync and async pipelines.Reporter's root cause analysis (at
main, commitc00962557a):RetryPolicyBase.increment(_retry.py~line 325) only guards against resending when the body has areadattribute and is not seekable; generators/async-generators bypass that guard entirely and fall through toreturn True(retry allowed) around line 346_configure_positionsrecords no rewind position for iterator-based bodies eitherset_content_body(rest/_helpers.py~line 146) explicitly accepts generators/async generators as validcontent=bodies, so this is a supported, not accidental, use caseAsyncRetryPolicyshares the sameincrementlogic (_retry_async.pylines ~180, ~196), so the sync and async paths need the same fixSuggested investigation steps:
read-based bodies (file-like objects) to use as the template for the fixincrementto detect iterator/async-iterator bodies (__iter__/__aiter__) that are notbytes/strand are not confirmed seekable, returningFalseto stop retry attempts🏷️ Label Confidence
Client— azure-core is a data-plane core library; the bug is in pipeline retry policy behavior, not a management-plane concernAzure.Core— issue explicitly names theazure-corepackage and cites specific file paths undersdk/core/azure-core/azure/core/pipeline/policies/👥 Owner Routing
# ServiceLabel: %Azure.Core(line 19) — exact match for the predicted service label; scanned bottom-to-top, this was the only entry found forAzure.Core@kashifkhan@kashifkhan(sole AzureSdkOwner); addedneeds-team-attentionsince this is customer-reported; "Service Attention" not added because an AzureSdkOwner is present# ServiceLabel: %Azure.Coreentries existed elsewhere in the file to compare againstkashifkhan commentedon Oct 19, 2026
Hi digit50, thank you for the issue and PR. Can you point to an example when this happened while using an SDK scenario please?
github-actions commentedon Oct 19, 2026
Hi digit50. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.
github-actions commentedon Oct 26, 2026
Hi digit50, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!
digit50 commentedon Oct 27, 2026
Hi Kashif Khan (@kashifkhan), thanks for looking. I don't have a production incident to point to. I found this while reading the azure-core retry policy and reproduced it with the script in the issue, which uses a fake transport so it runs without an Azure account.
The scenario is any client built on azure-core that sends a request whose body is a generator or async generator, since
HttpRequest(content=...)documents those as supported bodies. Any transient 5xx on the first attempt makes the retry go out with an empty body and the caller sees whatever the service returns for that, with no error from the SDK. Non-seekable file-like bodies are already excluded from retry, so the PR (#49592) just extends that rule to iterators. Happy to close this if you consider generator bodies out of scope for retries.