Merged
Conversation
github-project-automation
bot
moved this to Waiting Review/Merge from Staff
in Ray's Project
cdrini
reviewed
Jul 23, 2025
Collaborator
cdrini
left a comment
cdrini
left a comment
There was a problem hiding this comment.
Nice this looks like a good approach! A few fixes/comments.
| // This block catches network errors (e.g., DNS, connection refused) and the server errors we threw above. | ||
| } | ||
| const backoff = Math.pow(2, attempt) * initialDelay; | ||
| const jitter = Math.random() * backoff; |
Collaborator
There was a problem hiding this comment.
Would it make more sense to have the jitter be based only on the initialDelay without the exponential? That should avoid it having a range up to 2x the backoff.
| // TODO: this line is just for testing locally, remove before merging | ||
| CONFIGS.OL_BASE_BOOKS = 'https://openlibrary.org' | ||
|
|
||
| export async function fetchWithRetry(resource, options = {}, maxRetries = 5, initialDelay = 5000) { |
Collaborator
There was a problem hiding this comment.
Can you experiment with 1500 or 2000 as the default? Does that work?
Suggested change
| export async function fetchWithRetry(resource, options = {}, maxRetries = 5, initialDelay = 5000) { | |
| export async function fetchWithRetry(resource, options = {}, maxRetries = 5, initialDelay = 1500) { |
Collaborator
Author
|
@cdrini this one has incorporated all your feedback and is ready for another review. Seems to work quite well with the little check on the last 429 seen |
github-project-automation
bot
moved this from Waiting Review/Merge from Staff
to Done
in Ray's Project
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #10875
Just a draft at the moment but it works very well.
Testing with this url
http://localhost:8080/works/merge?ol_base=openlibrary.org&records=OL54120W,OL53924W,OL8193488W,OL6037022W,OL73243W,OL192489W,OL313321W,OL15298516W,OL151880W,OL61982W,OL15150515W,OL15673999W,OL16086453W,OL15706726W,OL20600W
Technical
Testing
Screenshot
Stakeholders