Skip to content

fix(producer): distinguish project files in cache hashes - #5249

Closed
user-github-me wants to merge 1 commit into
heygen-com:mainfrom
user-github-me:fix/producer-project-hash-boundaries
Closed

user-github-me wants to merge 1 commit into
heygen-com:mainfrom
user-github-me:fix/producer-project-hash-boundaries

Conversation

@user-github-me

Copy link
Copy Markdown
Contributor

Project cache keys currently concatenate each file path and its bytes without delimiting the end of the contents. A tree containing a = "bc\0def" hashes identically to a tree containing empty a plus bc = "def". Both cloud adapters then skip uploading the changed tree and reuse the wrong project.

Include the byte length before each file's contents in the shared hash input. Keep deterministic traversal, the 16-character ID format, and the existing top-level skip rules. Cover the collision in the hash helper and both adapters.

Validation:

  • 1,529 producer unit tests pass, with one existing skipped test; 144 AWS tests and 102 GCP tests pass.
  • Two hash collisions and both adapter cache regressions fail on main; unchanged-tree, binary/Unicode, empty-file, rename, and skip-rule controls pass.
  • Producer build, all three package typechecks, root lint, changed-file formatting, test reachability, and commit hooks pass.
  • Node 22 with the built producer export and local cloud transports: both adapters produce the same two distinct IDs, upload each tree once, skip the repeat upload, and preserve the correct contents in all four extracted archives.

Default project IDs change on upgrade, causing one fresh upload per project. Existing handles and explicit siteId overrides continue to work. No live cloud deployment was used.\n

@jrusso1020

Copy link
Copy Markdown
Collaborator

Thanks for this. The encoding is technically ambiguous, but a collision needs a file that ends in a NUL byte followed by exactly the next file's path and contents. That doesn't happen with real project trees, and the cache sits in the user's own bucket. Since the change also invalidates every existing project ID on upgrade, we'd rather not take it right now. — Rames

@jrusso1020 jrusso1020 closed this Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants