Skip to content

fix(idempotency): keep the payload hash when completing records in the Redis layer - #8656

Merged
leandrodamascena merged 2 commits into
aws-powertools:developfrom
thebadalov:fix/idempotency-redis-payload-validation-retry
Nov 4, 2026
Merged

leandrodamascena merged 2 commits into
aws-powertools:developfrom
thebadalov:fix/idempotency-redis-payload-validation-retry

Conversation

@thebadalov

Copy link
Copy Markdown
Contributor

Issue number: closes #8655

Summary

Changes

CachePersistenceLayer._update_record now writes validation_key_attr when payload validation is enabled, the same way _put_in_progress_record and DynamoDBPersistenceLayer._update_record already do. Before this, the COMPLETED record replaced the in-progress one without the payload hash.

Added test_idempotent_function_and_lambda_handler_redis_validation_same_payload, which checks that the completed record keeps the hash and that a retry with the same payload returns the stored response without running the handler again. It fails on the current develop and passes with the fix.

User experience

With payload_validation_jmespath set on the Redis/Valkey layer:

  • Before: a retry with the same payload raised IdempotencyValidationError: Payload does not match stored record for this event key.
  • After: the same-payload retry returns the stored response, and a retry with a different payload still raises IdempotencyValidationError.

Records written without payload validation are unchanged.

Testing

  • tests/functional/idempotency and tests/unit/idempotency: 203 passed
  • make format-check lint-check and mypy on the idempotency package: clean
  • Checked against a local Redis server: same payload returns the stored response, different payload raises, TTL is kept, no validation field is written when validation is off

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…e Redis layer

CachePersistenceLayer._update_record overwrote the record with a new SET
that only had data, status and expiration, dropping the validation hash
written by _put_in_progress_record. With payload_validation_jmespath set,
a retry with the same payload then raised IdempotencyValidationError
instead of returning the stored response.

Write the validation key in _update_record when payload validation is
enabled, as the DynamoDB layer already does.
@thebadalov
thebadalov requested a review from a team as a code owner October 24, 2026 14:18
@thebadalov
thebadalov requested a review from svozza October 24, 2026 14:18
@powertools-for-aws-oss-automation powertools-for-aws-oss-automation Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Oct 24, 2026
@boring-cyborg boring-cyborg Bot added the tests label Oct 24, 2026
@boring-cyborg

boring-cyborg Bot commented Oct 24, 2026

Copy link
Copy Markdown

Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

@blacksoxx blacksoxx left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff against develop; I did not run the suite here, so these are reading notes.

The asymmetry is exactly as described: _put_in_progress_record writes validation_key_attr when payload_validation_enabled, and the DynamoDB layer's _update_record appends #validation_key = :validation_key under the same condition, so the Redis _update_record was the one place that dropped the hash when the record moved to COMPLETED. With the four lines the completed record carries the hash again, which is what _get_record reads back into payload_hash for the next validation. Records written without payload validation stay as they are.

The functional test pins both halves, the stored hash and the second call not re-running the handler. Nothing blocking from my side.

@leandrodamascena leandrodamascena left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @thebadalov, confirmed against a real Valkey server: on develop the completed record loses the hash and a same-payload retry raises, with your change the retry returns the stored response and a different payload still raises. Records without validation are unchanged, and old records written before the fix keep behaving as they did until they expire, so nothing breaks on upgrade.

I'm approving now. This goes out in 3.40.0, so it will sit here for a while before I merge it. No rebase needed in the meantime, and you'll be credited in the release notes.

@sonarqubecloud

sonarqubecloud Bot commented Nov 4, 2026

Copy link
Copy Markdown

@codecov

codecov Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.01%. Comparing base (1ba1118) to head (054efac).
⚠️ Report is 1 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #8656   +/-   ##
========================================
  Coverage    97.00%   97.01%           
========================================
  Files          321      321           
  Lines        16421    16423    +2     
  Branches      1483     1484    +1     
========================================
+ Hits         15930    15932    +2     
  Misses         350      350           
  Partials       141      141           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@leandrodamascena
leandrodamascena merged commit 07f2a7a into aws-powertools:develop Nov 4, 2026
14 checks passed
@powertools-for-aws-oss-automation

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request and thank you for helping improve everyone's experience!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Denotes a PR that changes 30-99 lines, ignoring generated files. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: CachePersistenceLayer raises IdempotencyValidationError on retries with the same payload when payload_validation_jmespath is set

3 participants