fix(ai-bedrock): forward Converse cache token counts to TokenUsage

[devholic]

Bedrock Converse drops prompt-cache read and write counts from usage responses. Cached requests can therefore appear to use only a few input tokens. This change maps both counters to the existing promptTokensDetails fields.

🎯 Changes

• Add buildConverseUsage() to normalize Bedrock Converse usage.
• Use the helper in chatStream, structuredOutput, and structuredOutputStream.
• Preserve reported zero values. Omit promptTokensDetails only when Bedrock omits both cache fields.
• Document the usage shape and add a patch changeset for @tanstack/ai-bedrock.

The adapter cannot create cachePoint blocks yet. That request-side feature belongs in a separate PR.

✅ Checklist

• I have followed the steps in the Contributing guide.
• I have tested code changes locally with pnpm run test:pr, or these tests do not apply to this pull request.
• I fully understand the code in this pull request, including any code generated with AI assistance.
• Docs: I updated docs/ for this change, or this change is not user-facing.
• Changeset: I added a changeset (pnpm changeset), or this PR does not change a published package.

🚀 Release Impact

• This change affects published code, and I have generated a changeset.
• This change is docs/CI/dev-only (no release).

Root cause

Issue. onUsage and RUN_FINISHED.usage omit the cache counters when Bedrock returns them. Callers therefore see only the uncached input count.

Cause. Three Converse response paths copy only inputTokens, outputTokens, and totalTokens from metadata.usage. Bedrock reports cache reads and writes as separate fields.

During prompt caching, inputTokens contains only uncached input. The full input count is:

inputTokens + cacheReadInputTokens + cacheWriteInputTokens

Fix. buildConverseUsage() maps the two cache fields to promptTokensDetails.cachedTokens and promptTokensDetails.cacheWriteTokens. All three response paths now use this function.

The function preserves zero values. Bedrock can report cacheWriteInputTokens: 0 for a cache hit, which differs from an omitted field.

Possible alternatives

• Use providerUsageDetails. The adapter could store both fields under their Bedrock names. This would not populate AG-UI cachedInputTokens or the OpenTelemetry cache attributes.
• Map each response path separately. Each path could add the fields to its existing usage object. This would duplicate the same mapping in three places.

Testing

Commands run.

• pnpm test:pr: passed.
• pnpm nx run @tanstack/ai-bedrock:test:lib: 10 test files and 95 tests passed.
• pnpm --filter @tanstack/ai-e2e test:e2e: not run.

The E2E suite cannot replay the Bedrock Converse binary event stream. This limitation is documented under "Bedrock Converse coverage gap" in testing/e2e/README.md.

Independent repro.

I wrote a temporary Vitest test for the three affected paths and ran it twice: once against packages/ai-bedrock/src extracted from main at 416a4e34, once against this branch.

Command, from the repository root:

pnpm --filter @tanstack/ai-bedrock exec vitest run tests/_repro/cache-token-repro.test.ts

main at 416a4e34:

× preserves cache read and write counts on the finish event
× preserves cache counts for non-streaming structured output
× preserves cache counts for streaming structured output
AssertionError: expected { promptTokens: 3, …(2) } to match object { promptTokensDetails: { …(2) } }

Test Files  1 failed (1)
Tests       3 failed (3)

This branch at 7c9364a3:

Test Files  1 passed (1)
Tests       3 passed (3)

A real request produced these values:

cache write:
{ inputTokens: 3, outputTokens: 4, totalTokens: 8416,
  cacheReadInputTokens: 0, cacheWriteInputTokens: 8409 }

cache hit:
{ inputTokens: 3, outputTokens: 4, totalTokens: 8416,
  cacheReadInputTokens: 8409, cacheWriteInputTokens: 0 }

Manual test.

This test requires AWS credentials and a Claude model that supports prompt caching.

1. On main, subclass BedrockConverseTextAdapter.
2. Append a cachePoint to system in buildInput.
3. Call chat() twice with a system prompt longer than 4,096 tokens.
4. Read onUsage after each call.
5. Repeat the test on this branch.

On main, promptTokensDetails is absent. On this branch, the second call includes cachedTokens and cacheWriteTokens.

How this PR makes testing easy.

tests/converse/stream-processor.test.ts covers the normal streaming path. tests/converse/adapter.test.ts covers both structured-output paths without AWS credentials.

Risk / rollback

Risk is low. This change populates an existing optional field only when Bedrock returns cache counters. Request construction does not change.

Revert this PR to restore the previous behavior.

Public API change

Callers can now read Bedrock cache usage from the existing TokenUsage fields.

Before

onUsage(_ctx, usage) {
  usage.promptTokens // 3
  usage.promptTokensDetails // undefined
}

After

onUsage(_ctx, usage) {
  usage.promptTokens // 3
  usage.promptTokensDetails
  // { cachedTokens: 8409, cacheWriteTokens: 0 }
}

Summary by CodeRabbit

• New Features

  • Bedrock Converse now reports prompt-cache token usage, including cached and cache-write token counts, across chat and structured output responses.
  • Cache token fields are included when available and omitted when Bedrock does not provide them.

• Documentation

  • Added guidance on Bedrock token usage fields and calculating total input tokens.

• Tests

  • Added coverage for cache token reporting in streaming and non-streaming responses.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 57c7a657-451b-4002-ba68-ffe67d060cef

📥 Commits

Reviewing files that changed from the base of the PR and between 416a4e3 and 7c9364a.

📒 Files selected for processing (8)

• .changeset/bedrock-converse-cache-usage.md
• docs/adapters/bedrock.md
• docs/config.json
• packages/ai-bedrock/src/adapters/converse-text.ts
• packages/ai-bedrock/src/converse/stream-processor.ts
• packages/ai-bedrock/src/converse/usage.ts
• packages/ai-bedrock/tests/converse/adapter.test.ts
• packages/ai-bedrock/tests/converse/stream-processor.test.ts

Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.

---

📝 Walkthrough

Walkthrough

Bedrock Converse usage mapping now forwards cache-read and cache-write token counts through structured output and streaming APIs. The change adds shared normalization, tests, documentation, a changeset, and a documentation timestamp update.

Changes

Bedrock Converse usage forwarding

Layer / File(s)	Summary
Converse usage normalization packages/ai-bedrock/src/converse/usage.ts	Adds buildConverseUsage, which maps standard token fields and conditionally includes cachedTokens and cacheWriteTokens.
Usage routing and validation packages/ai-bedrock/src/adapters/converse-text.ts, packages/ai-bedrock/src/converse/stream-processor.ts, packages/ai-bedrock/tests/converse/*, docs/adapters/bedrock.md, .changeset/bedrock-converse-cache-usage.md, docs/config.json	Routes structured and streaming usage through the shared mapper. Tests verify cache token forwarding. Documentation and the changeset describe the usage fields.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 7c936

This change adds Bedrock prompt-cache token counts to the existing usage data without changing request handling or deployment behavior. The localized update is merge-ready after normal checks, with no actionable merge-blocking risk remaining.

Suggested reviewers: alemtuzlak, tombeckenham

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely identifies the main change: forwarding Bedrock Converse cache token counts to TokenUsage.
Description check	✅ Passed	The description is complete and directly related to the change. It explains the cause, fix, affected paths, testing, documentation, changeset, release impact, risks, rollback, and API behavior. All re…
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 5 files. (3 skipped: 3 …
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description is complete and directly related to the change. It explains the cause, fix, affected paths, testing, documentation, changeset, release impact, risks, rollback, and API behavior. All required template sections are present and appropriately completed.

Full details: Docstring Coverage

Explanation

Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 5 files. (3 skipped: 3 unsupported.)

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[nx-cloud]

View your CI Pipeline Execution ↗ for commit 7c9364a

Command	Status	Duration	Result
nx affected --targets=test:sherif,test:knip,tes...	✅ Succeeded	1m 34s	View ↗

---

☁️ Nx Cloud last updated this comment at 2026-09-02 18:24:43 UTC

[nx-cloud]

View your CI Pipeline Execution ↗ for commit 7c9364a

Command	Status	Duration	Result
nx run-many --targets=build --exclude=examples/...	✅ Succeeded	7s	View ↗

---

☁️ Nx Cloud last updated this comment at 2026-09-02 17:51:14 UTC