Skip to content

fix(parser): fall back to Python validation for non-JSON string payloads - #8699

Merged
leandrodamascena merged 3 commits into
aws-powertools:developfrom
azmi2409:fix-8695-cloudwatch-non-json-messages
Nov 4, 2026
Merged

leandrodamascena merged 3 commits into
aws-powertools:developfrom
azmi2409:fix-8695-cloudwatch-non-json-messages

Conversation

@azmi2409

Copy link
Copy Markdown
Contributor

Issue number: closes #8695

Summary

Changes

  • In _parse_and_validate_event, when validate_json rejects a string payload with a ValidationError, fall back to validate_python on the raw value instead of failing immediately. If Python validation also fails, the original JSON error is re-raised, so genuinely invalid payloads still surface the same error as before.
  • This fixes the shared string-parsing path, so it applies to every envelope that parses string payloads, not only CloudWatch Logs.
  • Added three regression tests in test_cloudwatch.py: plain-text Lambda runtime messages (START/END/REPORT plus an INFO line) are returned as-is with model=str, mixed JSON and plain-text messages are returned as-is with model=str, and plain text still raises ValidationError for an object model.

User experience

Before: parsing a real CloudWatch Logs subscription event with CloudWatchLogsEnvelope and model=str raised a ValidationError as soon as any log event was a plain-text Lambda runtime message (START, END, REPORT), even though the user asked for strings.

After: those plain-text messages are returned as-is alongside JSON messages, JSON messages for object models still parse as before, and plain text for an object model still raises a ValidationError.


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@leandrodamascena leandrodamascena left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @azmi2409, confirmed with real Lambda runtime lines from one of our functions: START, END and REPORT now come back as strings with model=str, a JSON line next to them still parses, and a plain line against an object model still fails with the same json_invalid as before. Since this touches the shared string path I ran the other envelopes and parse() directly too. Two things change beyond the issue, both for the better: model=str with a JSON object payload now returns the string instead of failing, and bare ISO dates parse for model=datetime. I'll mention them in the release notes.

I'm approving now. This goes out in 3.40.0, so it will sit here for a while before I merge it. No rebase needed in the meantime, and you'll be credited in the release notes.

@leandrodamascena leandrodamascena left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving after a small refactor on top: the nested fallback pushed parser/functions.py over the module complexity rank xenon enforces in CI, so I moved the string handling into _parse_string_payload. Same behaviour, 222 parser tests pass.

@sonarqubecloud

sonarqubecloud Bot commented Nov 4, 2026

Copy link
Copy Markdown

@codecov

codecov Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.61538% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.01%. Comparing base (8b5ec90) to head (5eb80f2).

Files with missing lines Patch % Lines
...ws_lambda_powertools/utilities/parser/functions.py 84.61% 2 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #8699   +/-   ##
========================================
  Coverage    97.01%   97.01%           
========================================
  Files          321      321           
  Lines        16424    16431    +7     
  Branches      1484     1484           
========================================
+ Hits         15933    15940    +7     
  Misses         350      350           
  Partials       141      141           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@leandrodamascena
leandrodamascena merged commit 083460d into aws-powertools:develop Nov 4, 2026
13 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Denotes a PR that changes 30-99 lines, ignoring generated files. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: CloudWatchLogsEnvelope raises ValidationError on non-JSON logEvents[*].message (e.g. Lambda START/END/REPORT)

2 participants