Skip to content

feat: Add Lambda-Runtime-Invocation-Id support for cross-wiring protection - #214

Open
vip-amzn wants to merge 1 commit into
mainfrom
feat/invocation-id-cross-wiring
Open

feat: Add Lambda-Runtime-Invocation-Id support for cross-wiring protection#214
vip-amzn wants to merge 1 commit into
mainfrom
feat/invocation-id-cross-wiring

Conversation

@vip-amzn

@vip-amzn vip-amzn commented Jul 16, 2026

Copy link
Copy Markdown

Summary

Add Lambda-Runtime-Invocation-Id header support for cross-wiring protection.

The RIC now echoes the invocation ID received from RAPID on /next back on /response and /error, enabling RAPID to detect and reject stale responses from timed-out invocations.

Problem

On Lambda Managed Instances (LMI) and On-Demand (OD), when an invoke times out, the runtime process continues running in the background. If a new invoke arrives with the same requestId, RAPID accepts it. The still-running old invocation eventually posts its response, and RAPID matches it to the new invoke — delivering the wrong response (cross-wiring).

Solution

RAPID sends a unique per-invoke nonce via Lambda-Runtime-Invocation-Id header on /next. The runtime echoes it back on /response and /error. RAPID validates the match before accepting the response.

Changes

  • runtime_client.cpp: Parse Lambda-Runtime-Invocation-Id from /next response, pass to post_success/post_failure via z format (accepts Nonenullptr)
  • lambda_runtime_client.py: Thread invocation_id through to native client
  • bootstrap.py: Pass invocation_id from event request to response/error posting
  • deps/patches/aws-lambda-cpp-add-invocation-id.patch: Patch aws-lambda-cpp to support invocation_id in get_next/do_post
  • deps/patches/aws-lambda-cpp-musl-no-execinfo.patch: Extracted Alpine/musl fix from d3e9632 into a proper patch file so update_deps.sh can reproduce the tarball from scratch (previously baked directly into the tarball without a patch — see d3e9632)
  • Unit tests: 6 new invocation-id specific tests + fix existing assertions for the new parameter

Backward Compatibility

Fully backward compatible in both directions:

  • If RAPID doesn't send the header → RIC doesn't see it → doesn't echo → no behavior change
  • If RIC doesn't echo it (old version) → RAPID skips validation → no behavior change

Testing

  • 95 unit tests pass (includes 6 new invocation-id tests)
  • E2E tested with updated RIE binary (validates invocation-id echo) and stock RIE (backward compat verified)
  • Alpine 3.19 Docker build verified locally (musl/execinfo conditional compilation works)

Comment thread tests/test_bootstrap.py
Comment on lines 909 to +931
@@ -927,6 +928,7 @@ def test_binary_with_application_json(self):
invoke_id,
error_result,
xray_fault,
invocation_id,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably working by chance because nothing is passed at line 909 and we are defaulting to none.
In general I think we should have a stack of test checking that the value is wired from the bootstrap to the client.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added test_invocation_id_wired_to_post_invocation_result and test_invocation_id_wired_to_post_invocation_error. There we pass an actual invocation_id through handle_event_request and assert it reaches the client call.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks this was the most important change I wanted.

runtime_client.post_invocation_result(invoke_id, response_data)

mock_runtime_client.post_invocation_result.assert_called_once_with(
invoke_id, response_data.encode("utf-8"), "application/json"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we default to a value instead of None for the these tests.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing tests validate other behavior and explicitly test the None default path (backward compat). The positive invocation_id path is covered by dedicated tests (test_post_invocation_result_with_invocation_id, test_post_invocation_error_with_invocation_id).

Let me know if you still prefer to add the default value, don't want to mix the concern of the tests here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was more of a nit. Rethinking it it's fine.

@vip-amzn
vip-amzn force-pushed the feat/invocation-id-cross-wiring branch from c14cbf5 to 1fea904 Compare July 28, 2026 10:18
…ction

Echo the invocation ID received from RAPID on /next back on /response
and /error, enabling RAPID to detect and reject stale responses from
timed-out invocations. Fully backward compatible — header only sent
when received.
@vip-amzn
vip-amzn force-pushed the feat/invocation-id-cross-wiring branch from 1fea904 to a5775f7 Compare July 28, 2026 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants