Skip to content

Fix parallel multipart download recording part 1 too late - #7504

Open
S-Saranya1 wants to merge 2 commits into
masterfrom
somepal/fix-parallel-multipart-download-null-response
Open

S-Saranya1 wants to merge 2 commits into
masterfrom
somepal/fix-parallel-multipart-download-null-response

Conversation

@S-Saranya1

@S-Saranya1 S-Saranya1 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Motivation and Context

With a multipart-enabled S3AsyncClient, a parallel multipart download could complete with a null response. When part 1 finished, the remaining parts were started before part 1's response was saved. If they all finished first, the download returned null, and S3TransferManager.downloadFile then reported a failure even though every part downloaded. A pause in the same window could also produce a resume token missing part 1.

Modifications

  • In ParallelMultipartDownloaderSubscriber, part 1's response and resume context are now recorded before totalPartsFuture completes, which is what starts the remaining parts.
  • The part-1 callback now checks once for a single-part object and returns early. Single-part objects behave as before.

Testing

  • Added ParallelMultipartDownloaderSubscriberTest. It completes the remaining parts inside part 1's callback, so the race happens on every run.
  • The multipart tests run with 2 parts/2 in flight, 4/4 and 4/2. A single-part test confirms that behavior is unchanged.
  • The existing S3 multipart download tests and the Transfer Manager unit tests pass.

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

@S-Saranya1
S-Saranya1 requested a review from a team as a code owner November 3, 2026 00:08
getObjectResponse = res;
MultipartDownloadUtils.multipartDownloadResumeContext(getObjectRequest)
.ifPresent(ctx -> {
ctx.addCompletedPart(1);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could pausing at this point capture part 1 as completed before response and totalParts are available? On resume, would that skip part 1 while leaving totalPartsFuture unresolved, preventing the remaining requests from starting?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants