feat(ai): add a flush option to emitCustomEvent

[oztune]

Changes

@tanstack/ai-compaction emits compaction:started when a summarize begins and
compaction:ended seconds later when it finishes - but no client can observe the gap
between them. Both arrive in the same batch, at the end, alongside the first model
chunks. So the event whose entire job is to signal "compaction in progress" never
arrives while compaction is in progress, and nothing downstream can react to it in
time.

The cause is durability batching, not compaction. isDurabilityFlushBoundary does
not include CUSTOM, so a compaction:started emitted in beforeModel sits in the
batch until an unrelated flush: the batch hitting DEFAULT_DURABILITY_BATCH (32), or
a RUN_FINISHED / TOOL_CALL_END. RUN_STARTED is a boundary but cannot help -
drainMiddlewareCustomQueue emits the synthetic RUN_STARTED before the first queued
custom event and hasPublicRunStarted suppresses the adapter's, so a pre-model
CUSTOM lands after the only RUN_STARTED with nothing behind it. This hits any
CUSTOM emitted before the first model output; compaction is just the case that ships
in-tree. The only workaround is durability: { batch: 1 }, which kills batching for
the whole stream.

The fix. Opt a single event out of batching:

ctx.emitCustomEvent('compaction:started', value, { flush: true })

isDurabilityFlushBoundary returns true for a CUSTOM chunk carrying the flag, so it
flushes on its own; unflagged CUSTOM chunks batch as before. Per-event rather than a
blanket CUSTOM boundary, since high-volume events (process.stdout) should keep
batching. On both ChatMiddlewareContext and ToolExecutionContext.
@tanstack/ai-compaction sets it on compaction:started / :state / :ended, so the
gap between start and end is observable.

Carrying the flag (two calls I'd like your read on).

• Rides in metadata.tanstack.flush via withTanstackMetadata - the only chunk field
  that survives normalizeStreamChunk's spec-key rebuild, so the only channel from
  emitCustomEvent to the producer. Round-trips on the wire like other tanstack
  metadata; the client ignores it. Can strip it in toWireChunk / before append if
  you'd rather it not persist.
• Typed on TanStackRunMetadata so tanstackMetadata(chunk)?.flush resolves. It is a
  delivery hint, not run metadata - happy to give it a dedicated type.

Tests

• stream-to-response-durability: hand-built stream into toServerSentEventsResponse,
  spy on append - a flagged CUSTOM flushes alone, an unflagged one batches with the
  chunks after it.
• Compaction e2e: real withCompaction through chat(), asserting compaction:started
  is appended in an earlier batch than any TEXT_MESSAGE_CONTENT.
• Tool e2e: a tool emitting three flagged events mid-run, asserting three separate
  append batches.

Docs

• docs/protocol/custom-events.md: documented the flag.
• docs/advanced/compaction.md: it claimed compaction:started shows up live during a
  slow summarize - only true with the flush; corrected.

Checklist

• Followed the Contributing guide.
• Tested locally: test:lib / test:types / test:oxlint for @tanstack/ai and
  @tanstack/ai-compaction (nx, deps built first).
• I understand all the code here.
• Updated docs.
• Changeset added (@tanstack/ai minor, @tanstack/ai-compaction patch).

Release impact

• Affects published code; changeset added.

Summary by CodeRabbit

• New Features

  • Custom events can now request immediate delivery with a per-event flush option.
  • High-volume events continue to be batched for efficiency.
  • Tool and middleware events support the same delivery control.

• Bug Fixes

  • Compaction progress events now appear immediately while summarization is running, instead of arriving together after processing completes.

• Documentation

  • Added guidance on using immediate flushing for low-volume progress updates while retaining batching for frequent output.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 5adf1e5c-92ab-4c9a-9046-1c2aa210cf75

📥 Commits

Reviewing files that changed from the base of the PR and between 53e2ec0 and 656c716.

📒 Files selected for processing (11)

• .changeset/emit-custom-event-flush.md
• docs/advanced/compaction.md
• docs/protocol/custom-events.md
• packages/ai-compaction/src/index.test.ts
• packages/ai-compaction/src/index.ts
• packages/ai/src/activities/chat/index.ts
• packages/ai/src/activities/chat/middleware/types.ts
• packages/ai/src/activities/chat/tools/tool-calls.ts
• packages/ai/src/stream-to-response.ts
• packages/ai/src/types.ts
• packages/ai/tests/stream-to-response-durability.test.ts

Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.

---

📝 Walkthrough

Walkthrough

The change adds a per-event { flush: true } option to custom events. Marked events become durability boundaries. Compaction lifecycle events use the option, with tests and documentation covering immediate delivery and batching behavior.

Changes

Custom event flush flow

Layer / File(s)	Summary
Flush contract and event propagation packages/ai/src/types.ts, packages/ai/src/activities/chat/...	Adds EmitCustomEventOptions, extends middleware and tool context APIs, and propagates the option to custom event chunks.
Durability boundary and validation packages/ai/src/stream-to-response.ts, packages/ai/tests/stream-to-response-durability.test.ts	Treats marked CUSTOM chunks as flush boundaries. Tests verify separate batches for marked events and batching for unmarked events.
Compaction integration and documentation packages/ai-compaction/src/*, docs/..., .changeset/emit-custom-event-flush.md	Marks compaction lifecycle events with { flush: true }. Integration coverage verifies compaction:started precedes model text in durability. Documentation describes the option and compaction behavior.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to 656c7

Selected custom progress events flush immediately while ordinary custom events remain batched, with coverage for middleware, tools, durability, and compaction behavior.

Sequence Diagram(s)

sequenceDiagram
  participant Compaction as Compaction middleware
  participant Chat as Chat execution
  participant Durability as Durability layer
  participant Client as Client
  Compaction->>Chat: emit compaction:started with { flush: true }
  Chat->>Durability: create CUSTOM chunk with flush metadata
  Durability->>Client: persist and deliver compaction:started
  Compaction->>Chat: emit compaction:state and compaction:ended
  Chat->>Durability: flush each marked lifecycle event
  Durability->>Client: deliver lifecycle events before grouped model output

Loading

Suggested reviewers: jherr, alemtuzlak, tombeckenham

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 55.56% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 9 functions across 8 files. (3 skipped: 3…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description explains the problem, fix, tests, documentation, and release impact. It includes the required checklist content, although the section headings differ slightly from the template.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding a flush option to emitCustomEvent.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 55.56% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 9 functions across 8 files. (3 skipped: 3 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}