feat(agent,acp): wire provider total_tokens through NIP-AM publish chain - #3593
Merged
Conversation
Parse OpenAI-reported total_tokens (Chat Completions + Responses API) in buzz-agent/llm.rs. Accumulate per-session cumulative total with an explicit tri-state (Unseen | Exact(n) | Unknown) in buzz-agent/lib.rs: exact only when every usage-bearing response in the session supplied a provider total; unknown after the first missing total, until a new session resets. Anthropic stays None (no genuine total; NIP-AM forbids summing categories). Extend buzz-acp/usage.rs UsageUpdatePayload with an optional accumulatedTotalTokens field (goose compat preserved). Track previous/current snapshots in SessionState; derive a turn delta only when both snapshots are present and monotonic -- absence, decrease, or no baseline leaves only the total delta null. Missing totals never flip deltaReliable or discard reliable input/output/cache deltas. Map turn and cumulative totals into the two TokenCounts fields in buzz-acp/pool.rs (replacing hardcoded None). Required tests added: Chat/Responses total_tokens parsing; multiple provider rounds all-exact; mixed present/missing totals within one turn; permanent session poisoning + new-session reset; first update without a baseline; cumulative decrease; omitted optional field from goose-shaped JSON; exact turn/cumulative publisher JSON mapping. 20 new tests, all passing. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Run cargo fmt --all to resolve 5 style hunks in pool.rs, usage.rs, lib.rs, and types.rs. Drop the redundant Some()/unwrap() wrapping in the null-totals-never-derived test (clippy::unwrap_used on a known-Some value); bind TokenCounts directly instead. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…honest publisher tests Overflow: replace saturating_add with checked_add in TurnTotalState::fold() and the turn→session merge. Overflow now produces Unknown rather than Exact(u64::MAX), preserving the exact-only invariant for both boundaries. Centralize: extract TurnTotalState::merge_session() so the per-response fold and the turn→session boundary share one checked-add implementation. Replace the open-coded match in lib.rs with a call to merge_session(). Session-boundary tests: add two integration tests in fake_llm.rs that drive the Session.accumulated_total_state match through the wire harness: - session_total_poisoned_by_missing_total_and_stays_poisoned: verifies Exact→Unknown transition and permanent poisoning, with i/o unaffected. - new_session_resets_total_accumulation: verifies accumulated_total_state is Unseen on session/new, not inherited from a prior session. Publisher tests: extract build_turn_metric_counts() as a pub(crate) pure function called by publish_agent_turn_metric(). Replace the two tests that duplicated the production logic with tests that call the real builder and assert on serialized JSON field names (camelCase) and values. Reverting the production builder fields to None now breaks these tests. Minor: add an unexpected total_tokens field to the Anthropic parse test fixture so the test would catch future accidental parsing. Add a shape- assumption comment to agent.rs explaining the input-or-output usage gate. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…antics The acc.checked_add(n) → Exact|Unknown match was duplicated in fold() and merge_session(). Extract a private checked_exact_sum(acc, n) helper and have both arms delegate to it. Overflow semantics now have exactly one definition; a future change needs to be made in one place only. Update comments in fold() and merge_session() to reference the helper rather than claiming each independently implements the contract. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
joahg
added a commit
to joahg/buzz-dev-mode
that referenced
this pull request
Jul 29, 2026
…-style * origin/main: feat: add first-class OpenRouter provider support (block#1975) feat(agent,acp): wire provider total_tokens through NIP-AM publish chain (block#3593) Co-authored-by: Amp <amp@ampcode.com> Signed-off-by: Joah Gerstenberg <joah@squareup.com> Amp-Thread-ID: https://ampcode.com/threads/T-019fabca-7cdc-777d-9ccd-ed829c1e60f0 # Conflicts: # desktop/src-tauri/src/commands/agent_models.rs
tlongwell-block
pushed a commit
that referenced
this pull request
Jul 30, 2026
* origin/main: (29 commits) feat(replica): portable heartbeat-token fence with snapshot-local reader routing (#3268) fix(git): channel binding tooling + author remediation for unbound repos (#3626) feat: configure S3 URL addressing style (#3400) feat: add first-class OpenRouter provider support (#1975) feat(agent,acp): wire provider total_tokens through NIP-AM publish chain (#3593) chore(release): release Buzz Desktop version 0.5.2 (#3624) docs: add Linux rendering troubleshooting guide (#3573) fix(desktop): discover bun-installed agent CLIs in ~/.bun/bin (#3343) feat(tracing): correlate trace IDs in relay logs (#3608) chore(ci): bump Linux AppImage build container to ubuntu:24.04 (#3602) feat(cli): mirror Desktop mention delivery (#3330) fix(desktop): deduplicate relay outage notification (#3579) fix(desktop): reconcile thread arrivals at bottom (#3585) fix(mobile): keep TLS on relays joined by invite (#3139) Improve emoji autocomplete matching (#3571) Fix shared agent avatar import profiles (#3578) Fix inline raster avatars in agent catalog (#3581) revert(acp): remove dead GOOSE_ACP_SCHEDULER_DISABLED env injection (#3576) feat(agent): make Gemini and MLflow-route models usable through databricks_v2 (#3569) fix(cli): mask credential env values in --help output (#3570) ... Co-authored-by: Tyler Longwell <tlongwell@block.xyz> Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
wpfleger96
added a commit
that referenced
this pull request
Jul 30, 2026
…chive * origin/main: (22 commits) feat(catalog): resolve publisher display name in catalog detail pane (#3640) feat(mesh): upgrade embedded mesh to v0.74 and harden shared compute (split 1/2 of #3467) (#3741) docs(nips): specify kind:30621 multi-repo projects (NIP-MP) (#3163) Refine agent sharing dialog (#3699) desktop: enable getUserMedia in the Linux WebKitGTK webview (#3607) fix: align responsive agent views (#3688) Add macOS agent menu-bar menu (#3565) Fix pending message feedback (#3543) fix(desktop): remove remaining Projects panel fills (#3742) feat(mobile): desktop-parity emoji and thread experience (#3485) desktop: restore direct community member adds (#3634) fix(desktop): explain open agent access (#2561) fix(cli): resolve agents from owner records (#3178) fix(desktop): remove Projects overview card fills (#3416) feat(replica): portable heartbeat-token fence with snapshot-local reader routing (#3268) fix(git): channel binding tooling + author remediation for unbound repos (#3626) feat: configure S3 URL addressing style (#3400) feat: add first-class OpenRouter provider support (#1975) feat(agent,acp): wire provider total_tokens through NIP-AM publish chain (#3593) chore(release): release Buzz Desktop version 0.5.2 (#3624) ... Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96
pushed a commit
that referenced
this pull request
Jul 30, 2026
…g-pipeline * origin/main: (25 commits) Refine agent sharing dialog (#3699) desktop: enable getUserMedia in the Linux WebKitGTK webview (#3607) fix: align responsive agent views (#3688) Add macOS agent menu-bar menu (#3565) Fix pending message feedback (#3543) fix(desktop): remove remaining Projects panel fills (#3742) feat(mobile): desktop-parity emoji and thread experience (#3485) desktop: restore direct community member adds (#3634) fix(desktop): explain open agent access (#2561) fix(cli): resolve agents from owner records (#3178) fix(desktop): remove Projects overview card fills (#3416) feat(replica): portable heartbeat-token fence with snapshot-local reader routing (#3268) fix(git): channel binding tooling + author remediation for unbound repos (#3626) feat: configure S3 URL addressing style (#3400) feat: add first-class OpenRouter provider support (#1975) feat(agent,acp): wire provider total_tokens through NIP-AM publish chain (#3593) chore(release): release Buzz Desktop version 0.5.2 (#3624) docs: add Linux rendering troubleshooting guide (#3573) fix(desktop): discover bun-installed agent CLIs in ~/.bun/bin (#3343) feat(tracing): correlate trace IDs in relay logs (#3608) ... Signed-off-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
michaelneale
added a commit
that referenced
this pull request
Jul 31, 2026
Nine conflicts. The deletions resolve trivially -- goose owns those files now, so mcp.rs, llm.rs, handoff.rs and tests/fake_llm.rs stay deleted. The four content conflicts (agent.rs, config.rs, lib.rs, types.rs) all resolve to this branch's rewrite, but two of main's changes are wire-facing and buzz-acp now depends on them, so they are ported onto the goose loop rather than discarded: * #3463 `accumulatedCachedInputTokens` -- buzz-acp/src/usage.rs reads it for pricing. Sourced from goose's `cache_read_input_tokens` + `cache_write_input_tokens`, which goose documents as subsets of `input_tokens` (`token_usage.rs:72-78`), so it stays a subset here too. * #3593 `accumulatedTotalTokens` -- emitted only when exactly known. One turn without a provider total poisons the session cumulative to `None` and the field is omitted, because buzz-acp must not read a missing total as zero. Cargo.lock was regenerated rather than resolved by hand: main moved mesh-llm to v0.74.0, which requires rmcp ^1.8, and the stale lock still pinned 1.7.0. 89 tests green, fmt + clippy -D warnings clean workspace-wide; workspace, sprig and desktop/src-tauri all build. Signed-off-by: Michael Neale <michael.neale@gmail.com>
calvadev
pushed a commit
to shopstr-eng/buzz
that referenced
this pull request
Aug 3, 2026
…ain (block#3593) ## What Wires genuine provider-reported `total_tokens` through the full buzz-agent → buzz-acp publish chain so kind-44200 events carry real per-turn and cumulative totals for OpenAI-backed models, while preserving all existing behaviour for Anthropic and external harnesses (goose, claude-code). ## Why Live prod data showed 0 of 1,934 archived reports carry `totalTokens`. Both hardcoded `total_tokens: None` in `pool.rs` and the absent field in `buzz-agent`'s parser are root causes. This is the backend half of a two-track fix; the display-fallback half lands in [block#2035](block#2035). ## Changes **`crates/buzz-agent/src/types.rs`** - Added `total_tokens: Option<u64>` to `LlmResponse` with an explicit doc comment that NIP-AM forbids deriving it. - Added `TurnTotalState` enum (`Unseen | Exact(u64) | Unknown`) with `fold()` and `exact_value()` — the tri-state accumulator that distinguishes not-yet-observed from permanently poisoned. **`crates/buzz-agent/src/llm.rs`** - `parse_responses` and `parse_openai`: read `usage.total_tokens` from OpenAI Chat Completions (including Databricks routes) and the Responses API via `sum_usage`. - Anthropic: explicit `total_tokens: None` — no genuine total available; NIP-AM forbids summing categories. **`crates/buzz-agent/src/agent.rs`** - Added `turn_total_state: &'a mut TurnTotalState` to `RunCtx`. - Fold `response.total_tokens` into the accumulator after each usage-bearing response; non-usage-bearing responses (keepalive/stream frames) do not poison. **`crates/buzz-agent/src/lib.rs`** - Added `accumulated_total_state: TurnTotalState` to `Session` (default `Unseen`). - Per-turn state passed to `RunCtx`, folded into session cumulative after each turn. - Emits `accumulatedTotalTokens` in `usage_update` only when cumulative is `Exact(n)`. **`crates/buzz-acp/src/usage.rs`** - Added `accumulated_total_tokens: Option<u64>` (serde default) to `UsageUpdatePayload` — optional for goose compat. - Added `last_total: Option<u64>` to `SessionState`. - Added `turn_total_tokens` and `cumulative_total_tokens` to `TurnUsage` (field-local — never affect `delta_reliable`). - Derive turn-total delta only when prev and current are both `Some` and monotonic; absence, decrease, or no baseline leaves only the total delta null without touching input/output reliability. **`crates/buzz-acp/src/pool.rs`** - Replaced both hardcoded `total_tokens: None` in `publish_agent_turn_metric` with `usage.turn_total_tokens` and `usage.cumulative_total_tokens`. ## Tests 20 new tests across the four touched files: | File | Tests | |------|-------| | `types.rs` | `TurnTotalState` fold, accumulation, exact_value, default (7 tests) | | `llm.rs` | Chat present/absent, Responses present/absent, Anthropic always-None (5 tests) | | `usage.rs` | First turn no baseline, second-turn delta, cumulative decrease (field-local), current absent, goose-shaped deserialization, baseline absent (6 tests) | | `pool.rs` | Exact turn+cumulative mapping, null totals never derived (2 tests) | `cargo test -p buzz-acp -p buzz-agent` — all passing, 0 failures. ## Scope Boundary: `crates/buzz-agent/**` + `crates/buzz-acp/**` only. Desktop unchanged. `costUsd` explicitly out of scope. Related: [block#2035](block#2035) --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1g8493u0xfsjrvflg4n08ezd7vec99mnwzlv0qgwpr9d7gvjwhuzqx59rhw <41ea58f1e64c243627e8acde7c89be667052ee6e17d8f021c1195be4324ebf04@buzz.block.builderlab.xyz>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Wires genuine provider-reported
total_tokensthrough the full buzz-agent → buzz-acp publish chain so kind-44200 events carry real per-turn and cumulative totals for OpenAI-backed models, while preserving all existing behaviour for Anthropic and external harnesses (goose, claude-code).Why
Live prod data showed 0 of 1,934 archived reports carry
totalTokens. Both hardcodedtotal_tokens: Noneinpool.rsand the absent field inbuzz-agent's parser are root causes. This is the backend half of a two-track fix; the display-fallback half lands in #2035.Changes
crates/buzz-agent/src/types.rstotal_tokens: Option<u64>toLlmResponsewith an explicit doc comment that NIP-AM forbids deriving it.TurnTotalStateenum (Unseen | Exact(u64) | Unknown) withfold()andexact_value()— the tri-state accumulator that distinguishes not-yet-observed from permanently poisoned.crates/buzz-agent/src/llm.rsparse_responsesandparse_openai: readusage.total_tokensfrom OpenAI Chat Completions (including Databricks routes) and the Responses API viasum_usage.total_tokens: None— no genuine total available; NIP-AM forbids summing categories.crates/buzz-agent/src/agent.rsturn_total_state: &'a mut TurnTotalStatetoRunCtx.response.total_tokensinto the accumulator after each usage-bearing response; non-usage-bearing responses (keepalive/stream frames) do not poison.crates/buzz-agent/src/lib.rsaccumulated_total_state: TurnTotalStatetoSession(defaultUnseen).RunCtx, folded into session cumulative after each turn.accumulatedTotalTokensinusage_updateonly when cumulative isExact(n).crates/buzz-acp/src/usage.rsaccumulated_total_tokens: Option<u64>(serde default) toUsageUpdatePayload— optional for goose compat.last_total: Option<u64>toSessionState.turn_total_tokensandcumulative_total_tokenstoTurnUsage(field-local — never affectdelta_reliable).Someand monotonic; absence, decrease, or no baseline leaves only the total delta null without touching input/output reliability.crates/buzz-acp/src/pool.rstotal_tokens: Noneinpublish_agent_turn_metricwithusage.turn_total_tokensandusage.cumulative_total_tokens.Tests
20 new tests across the four touched files:
types.rsTurnTotalStatefold, accumulation, exact_value, default (7 tests)llm.rsusage.rspool.rscargo test -p buzz-acp -p buzz-agent— all passing, 0 failures.Scope
Boundary:
crates/buzz-agent/**+crates/buzz-acp/**only. Desktop unchanged.costUsdexplicitly out of scope.Related: #2035