fix: agent reliability — no restart on channel-add, visible dead-letter notice#1468
Merged
Conversation
Adding a running local agent to a channel used stop+start to pick up the new subscription, killing any in-flight turn. The harness has handled membership additions dynamically since buzz-acp learned live member-added subscriptions, so the restart branch is dead weight — remove it and the associated "restarted" result plumbing/copy. Co-authored-by: Brain <21994759fc7a6fa6b965551d35cfd7897d262f2495467f2d78694ddcfa6a5c7e@sprout-oss.stage.blox.sqprod.co> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Silent turn deaths left channels waiting on replies that would never
come. EventQueue::requeue now returns the exhausted batch instead of
discarding it, and handle_prompt_result posts a short kind:9 notice
("couldn't process the last request after multiple retries") into the
triggering thread. Transient failures still retry with backoff and stay
quiet — only final dead-letter is surfaced.
Also raises the buzz-agent LLM call timeout default from 120s to 240s
so slow-but-successful LLM calls stop being classified as failures.
Co-authored-by: Brain <21994759fc7a6fa6b965551d35cfd7897d262f2495467f2d78694ddcfa6a5c7e@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: Wes <wesbillman@users.noreply.github.com>
wpfleger96
approved these changes
Jul 2, 2026
wpfleger96
added a commit
that referenced
this pull request
Jul 2, 2026
…n-metrics * origin/main: feat: per-community workspace icon set by admins, served via NIP-11 (#1463) perf(relay): batch outbound websocket data frames (#1464) Make reaction ingest atomic (#1458) Serialize fan-out EVENT frames once (#1459) fix: agent reliability — no restart on channel-add, visible dead-letter notice (#1468) Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com> # Conflicts: # crates/buzz-relay/src/handlers/event.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two focused fixes for the dominant agent-reliability failure modes discussed in #agent-reliability: turns silently dying when a running agent is added to a channel, and dead turns leaving channels waiting forever with no visible signal.
1. Don't restart running local agents on channel-add (
desktop)Adding a running local agent to a channel did
stopManagedAgent→startManagedAgentto pick up the new subscription — killing any in-flight turn. That branch dates to #86, before buzz-acp learned dynamic subscriptions; the harness now subscribes to new channels live via member-added notifications, and the desktop doesn't setBUZZ_ACP_CHANNELS, so nothing blocks the dynamic path.The fix is mostly deletion: remove the restart branch in
attachManagedAgentToChannelplus therestartedresult plumbing and stale UI copy.2. Visible notice when a turn is dead-lettered (
buzz-acp,buzz-agent)When retries were exhausted, the queue dead-lettered the batch with only an ERROR log — the channel never learned the turn died. Now:
EventQueue::requeuereturns the exhausted batch instead of discarding it.handle_prompt_resultposts a short kind:9 into the triggering thread: "Also raises the buzz-agent LLM call timeout default 120s → 240s so slow-but-successful LLM calls stop being classified as failures and burning retries.
Testing
cargo test: buzz-acp 405 passed, buzz-agent 122 passed (includes a new dead-letter queue test)cargo clippyclean; root + desktop-tauri rustfmt cleanpnpm test1494 passed; typecheck cleanCo-authored-by: Brain 21994759fc7a6fa6b965551d35cfd7897d262f2495467f2d78694ddcfa6a5c7e@sprout-oss.stage.blox.sqprod.co