fix(server): Muse turns stopped by the subscription quota are Limited, not failed

[ntindle]

Problem

When a Meta subscription window runs out, Meta refuses the next request with a 429, and Muse fails the turn with API error 429: Subscription quota exhausted… (rate_limit_error). The Muse adapter reported every failed turn as provider_error (or transport_error). So T3 showed a red "Provider error" and marked the thread Failed, and Resume at reset, Snooze until reset and auto-resume never applied. Codex, Claude and Grok already report the same kind of stop as usage_limit. Fixes #18236.

Change

In packages/provider-muse:

• The adapter keeps the newest usage/changed report its host sends. That is Meta's subscription windows after each response; it is account-wide, so the notification names no session. It is listed in INFORMATIONAL_NOTIFICATIONS, so an unreadable report is skipped instead of ending the session.
• When a turn fails with Muse's quota refusal, the failure is usage_limit, titled "Usage limit reached". A refusal counts when Muse types it as a quota stop (error.quota) or when its text has 429 with "quota" or "usage limit"; the real refusal below arrived untyped, so the text check stays. Its resetAt is the latest reset still ahead among the typed quota.resetsAtMs, the time the refusal names (Your usage window resets at 2026-10-13T10:45:52Z.), and the windows the last report showed used up, because every exhausted window has to reset before the turn can run. With none of those, the thread is Limited without a reset time.
• A plain 429 rate limit (API error 429: Rate limit exceeded.) stays provider_error.

No user guide changes: Inspect agent work already describes Limited, Resume at reset and Snooze until reset without naming providers.

Scope and approval

Fixes #18236, which Julius confirmed as a bug on main (Grok replying on Julius's behalf) and asked for this change: "A focused PR that classifies the quota refusal as usage_limit (and sets resetAt when a prior usage/changed report has it) is welcome — include before/after of the Limited thread state per CONTRIBUTING.md." (comment). Showing Muse in Usage → Limits is separate (#18235) and not part of this PR.

Verification

Tests (Windows desktop, Node 24; rerun on macOS after the reset-time follow-up):

• vp test run in packages/provider-muse: 96 passed on macOS. That includes the new usageLimits.test.ts and five new adapter tests:

  • A quota refusal after a used-up report is usage_limit, with the session window's reset; a later Rate limit exceeded 429 stays provider_error.
  • Without a used-up report it is usage_limit with no reset time.
  • A refusal that names its reset (the real text below) is usage_limit at that time, with no report.
  • A stop Muse types as error.quota is usage_limit at its resetsAtMs, whatever the text says.
  • Unreadable reports (malformed, or a reset no Date can hold) are skipped without closing the host.

  These adapter tests fail on main's adapter.

• The Muse replay fixtures in OrchestratorReplayFixtures.integration.test.ts pass. tsc --noEmit is clean for provider-muse and apps/server, and vp lint and vp fmt --check pass.

Before and after (2026-10-11): a dev server (vp run dev with its own home directory) on Linux ran main ed0f7b908 for before and this branch for after. Muse 1.4.4 was signed in to Meta, with a config folder of its own, and answered real turns. A local proxy between Muse and api.meta.ai (Muse's endpoint_transport) passed traffic through. For after, it rewrote one report's 5-hour window to 100% used, keeping its reset time. In both runs it then answered the next turn with a 429 in the shape CLIProxyAPI's Meta executor tests use (code: rate_limit_exceeded, with resets_at); a real refusal is in the last section. Captured in T3's preview browser at 1280×800 and cropped.

	Before (main)	After (this PR)
The refused turn
The thread's status

"Retry after 4:03 PM" is the rewritten report's 5-hour reset (21:03 UTC). The refusal is listed twice in both columns; main does the same, and this PR doesn't change it.

iOS (2026-10-11): the dev client, built from main ed0f7b908 on an iPhone 17 simulator (iOS 27.0), paired with the same two servers. The app is the same in both columns; only the server changed. The iOS runs are separate threads with the same prompts, so their titles differ. (Xcode 27.0 needed a local edit to the compiler guard in T3LayoutMetricsView.swift to build main; it is not part of this PR.)

	Before (main)	After (this PR)
The refused turn
The thread list

A real quota stop (2026-10-13): macOS, Muse 1.4.6, this branch's dev server, with an account whose Meta subscription window had actually run out. Meta refused every request with a 429. Muse made ten attempts, 60 seconds apart, without sending turn/retryScheduled, so the thread showed Working for about nine minutes. Then it ended the turn with error.kind: "modelError", retryable: true, no error.quota, and this message (the "All credentials … cooling down" wrapper comes from a credential proxy in front of Meta):

API error 429: All credentials for model muse-wonder-1.0-contributor are cooling down (last error: rate_limit_exceeded: Subscription quota exhausted. Your usage window resets at 2026-10-13T10:45:52Z.) (after 10 provider attempts)

The thread ended Limited. Its host was new, so no usage report had arrived, and the first version showed "Reset time unavailable". With a16c2d4 the reset comes from the refusal itself (Retry after 5:45 AM local), and with auto-snooze on, the thread snoozes until then.

Before a16c2d4	After a16c2d4

Not checked: Android, the desktop app (same web UI), and Muse 1.4.7 (out 2026-10-13), which may type this refusal as error.quota.

Created with Claude Opus 5.5 in Claude Code.

🤖 Generated with Claude Code

[macroscopeapp]

Approvability

Verdict: Approved at a16c2d4

Macroscope's review found this PR approvable — This is a focused Muse adapter bug fix that maps subscription-quota stops to the existing Limited/usage-limit flow and derives reset times from validated provider data. The change is contained to the provider protocol and adapter, preserves ordinary failure handling, and adds targeted coverage for the new cases.

^{You can add or adjust custom eligibility rules. Learn more.}

Dismissing prior approval to re-evaluate 4102621

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Path: .coderabbit.config.ts
• Review profile: CHILL
• Plan: Advanced
• Run ID: d3d0fda7-bd73-4d9c-b601-833027efac9f

📥 Commits

Reviewing files that changed from the base of the PR and between a5663de and a16c2d4.

📒 Files selected for processing (5)

• packages/provider-muse/src/server/adapter.test.ts
• packages/provider-muse/src/server/adapter.ts
• packages/provider-muse/src/server/protocol.ts
• packages/provider-muse/src/server/usageLimits.test.ts
• packages/provider-muse/src/server/usageLimits.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

📝 Walkthrough

The Muse adapter records subscription usage reports, identifies quota refusals, and derives reset times from exhausted usage windows or reset details in the refusal.

Changes: Muse quota failure handling

Layer / File(s)	Summary
Usage report contract and reset calculation packages/provider-muse/src/server/protocol.ts, packages/provider-muse/src/server/usageLimits.ts, packages/provider-muse/src/server/usageLimits.test.ts	The protocol schema validates subscription usage reports and optional quota reset metadata. Helpers identify usage-limit failures and return the latest qualifying future reset time. Tests cover recognition and reset-time selection.
Adapter notification and failure handling packages/provider-muse/src/server/adapter.ts, packages/provider-muse/src/server/adapter.test.ts	The adapter stores host-wide usage reports and uses them to classify failed turns. Usage-limit failures receive a reset time when one is available and use the “Usage limit reached” title. Tests cover quota and ordinary rate-limit failures, malformed reports, and session state.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Low

Suggested reviewers: juliusmarminge

Merge Risk: ⚪ Minimal · up to a16c2

The quota-handling change is mergeable after normal checks; cross-host notification delivery remains unconfirmed.

Security architecture risk: 🔵 Low · up to a16c2

The change is bounded to Muse quota reporting and preserves existing turn and host-failure controls. No new security finding was established, but upstream report ordering, account consistency, and downstream automatic recovery were not fully verified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — Control of the receiving host's quota metadata or refusal text can influence that session's failure classification and reset timestamp. The inspected changed branch emits failure events rather than selecting credentials, granting permissions, or issuing tool commands. Downstream recovery execution remains only partially assessed.

Trust Boundaries and Controls

• observed — Typed quota metadata passes completion-schema decoding; text classification requires both 429 and quota or usage-limit wording. Reset candidates must be finite, later than completion time, and within the JavaScript date bound. These controls constrain failure reporting but do not authenticate account identity.

Resilience and Maintainability Implications

• inferred — The cache accepts reports in arrival order without comparing observedAtMs. Delayed or incomplete delivery could therefore affect recovery timing, although producer behavior was not established and typed or named refusal resets provide fallback evidence. The existing automatic-resume preference defaults to false.

Hardening Proposals

• proposed — Document the producer's ordering, fanout, and account-lifetime guarantees. If chronological delivery is not guaranteed, use observedAtMs to prevent older reports from replacing newer state. This is a contract-hardening proposal, not a verified vulnerability.

Pre-merge checks · 4

Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check		The changes satisfy the coding requirements in #18236. adapter.ts accepts host-wide usage/changed reports as informational, stores the newest report, and maps recognized Muse quota refusals to `us…
Out of Scope Changes check		The changes stay within #18236. The protocol schema, adapter logic, usage-limit helpers, and focused tests support Muse quota classification and reset behavior. No unrelated Usage → Limits implementat…
Title check		The title clearly and concisely describes the main change: classifying Muse subscription-quota failures as Limited instead of failed.
Description check		The description includes all required sections, explains the problem and change, references issue approval, and provides detailed test results, manual verification, screenshots, and known gaps.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[ntindle]

Still waiting on a real Meta quota stop to confirm this end to end. It's hard to set one up on demand: the account has to actually use up a window.

The refusal in the screenshots came from a local proxy answering with the 429 body that CLIProxyAPI's Meta executor tests use (code: rate_limit_exceeded, with resets_at). Muse turns it into API error 429: Subscription quota exhausted…, and this PR matches that text (429 plus "quota" or "usage limit"). So a real stop is what confirms Meta's wording matches.

Dismissing prior approval to re-evaluate a16c2d4

[ntindle]

Confirmed with a real quota stop on 2026-10-13 (macOS, Muse 1.4.6): the thread ends Limited, not Failed.

The real refusal names its reset (…Subscription quota exhausted. Your usage window resets at 2026-10-13T10:45:52Z.), so a16c2d4 takes resetAt from it, and from Muse's typed error.quota when that is present. That closes a gap the simulated run hid: a fresh host has no usage/changed report yet, so the first version showed "Reset time unavailable".

Before a16c2d4	After a16c2d4

The exact refusal, and Muse's nine minutes of retries before it, are in the updated description.

Created with Claude Opus 5.5 in Claude Code.