Part of epic #8286 (Phase 3 — AI observability). ORB-side counterpart to #10198.
Problem
PostHog holds no token data at all for the ORB's AI calls. Verified on project 527730:
$ai_input_tokens and $ai_output_tokens do not appear in the property list for $ai_generation at all (read-data-schema, event_properties).
posthog.ai_events.input_tokens / output_tokens / total_tokens are NULL for every model over the last 2 days — including claude-sonnet-5, which accounts for 2,321 calls and $498.43 of real spend in the same window.
Cost lands correctly ($ai_total_cost_usd is populated), so the event itself is arriving and being parsed — it is specifically the token fields that carry nothing.
Why this is not simply "the provider reports no tokens"
capturePostHogAiGeneration (src/selfhost/posthog.ts) always sets both properties:
$ai_input_tokens: Number.isFinite(event.inputTokens) ? event.inputTokens : 0,
$ai_output_tokens: Number.isFinite(event.outputTokens) ? event.outputTokens : 0,
So a value is always sent — 0 when the provider reported nothing. Two candidate mechanisms, not yet distinguished:
usage.inputTokens / usage.outputTokens are genuinely undefined for the CLI providers, so 0 is sent for every call and PostHog treats 0 as no-data. extractCliUsage/mergeUsage can populate them — recordCliUsageMetrics guards its Prometheus counters on usage.inputTokens !== undefined — so whether they are populated in practice for claude-code needs checking against real CLI stdout.
- The values are populated but dropped somewhere between capture and ingestion.
Worth checking (1) first: if the ORB is sending 0 for every call, that is the same failure shape as #10198 on the miner side — a hardcoded/degenerate 0 where a real figure exists — and the fix is the same in spirit.
Impact
PostHog's LLM cost views break down spend by token usage. With no tokens, the ORB shows dollar cost but no tokens/call, no input-vs-output ratio, and no cost-per-token — and $ai_input_cost_usd/$ai_output_cost_usd, which PostHog derives from tokens, cannot be computed.
This also makes the "AI spend by model" dashboard panel unable to carry token columns; they were removed from it rather than left reading null.
Deliverables
Part of epic #8286 (Phase 3 — AI observability). ORB-side counterpart to #10198.
Problem
PostHog holds no token data at all for the ORB's AI calls. Verified on project 527730:
$ai_input_tokensand$ai_output_tokensdo not appear in the property list for$ai_generationat all (read-data-schema,event_properties).posthog.ai_events.input_tokens/output_tokens/total_tokensareNULLfor every model over the last 2 days — includingclaude-sonnet-5, which accounts for 2,321 calls and $498.43 of real spend in the same window.Cost lands correctly (
$ai_total_cost_usdis populated), so the event itself is arriving and being parsed — it is specifically the token fields that carry nothing.Why this is not simply "the provider reports no tokens"
capturePostHogAiGeneration(src/selfhost/posthog.ts) always sets both properties:So a value is always sent —
0when the provider reported nothing. Two candidate mechanisms, not yet distinguished:usage.inputTokens/usage.outputTokensare genuinelyundefinedfor the CLI providers, so0is sent for every call and PostHog treats0as no-data.extractCliUsage/mergeUsagecan populate them —recordCliUsageMetricsguards its Prometheus counters onusage.inputTokens !== undefined— so whether they are populated in practice forclaude-codeneeds checking against real CLI stdout.Worth checking (1) first: if the ORB is sending
0for every call, that is the same failure shape as #10198 on the miner side — a hardcoded/degenerate 0 where a real figure exists — and the fix is the same in spirit.Impact
PostHog's LLM cost views break down spend by token usage. With no tokens, the ORB shows dollar cost but no tokens/call, no input-vs-output ratio, and no cost-per-token — and
$ai_input_cost_usd/$ai_output_cost_usd, which PostHog derives from tokens, cannot be computed.This also makes the "AI spend by model" dashboard panel unable to carry token columns; they were removed from it rather than left reading null.
Deliverables
claude-codeandollamacall and inspect theAiUsageactually handed tocapturePostHogAiGeneration.0— a fabricated 0 is indistinguishable from a real 0 in an aggregate, which is the same reasoning applied in ai(observability): the miner reports 0 input/output tokens because both drivers sum the split away #10198.