Repository navigation
Telemetry drops the inference span of a sub-agent that inherits a non-Gemini model #7892
Description
🔴 Required Information
Describe the Bug:
A sub-agent with no model of its own, under a non-Gemini root (for example LiteLlm("openai/gpt-4o")), loses its generate_content span and its inference metrics when the google-genai instrumentor is active.
tracing._agent_model_name resolves an agent's model as agent.model if agent.model != "" else agent._default_model. It ignores the model the agent inherits from its nearest LlmAgent ancestor, which is what LlmAgent.canonical_model (and so llm_request.model) actually runs. The child is therefore judged by the process default (gemini-3.5-flash). _should_emit_native_telemetry then defers the call to the google-genai instrumentor, which never sees a LiteLLM call. The inference token metrics are gated on the same name.
The opposite case fails too. A sub-agent that inherits a Gemini model while set_default_model names a non-Gemini model is recorded by ADK instead of the instrumentor, so its span comes from a different recorder than its Gemini root's. (An earlier version of this issue said "recorded twice". ADK's maybe_without_genai_instrumentation peels the instrumentor off a call that ADK records, so the span isn't duplicated.)
Steps to Reproduce:
- Turn telemetry on with
GoogleGenAiSdkInstrumentor().instrument()andOTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental. - Run
root = LlmAgent(name="root", model=LiteLlm("openai/gpt-4o"), sub_agents=[LlmAgent(name="child", instruction="answer")]), and have root transfer to child.
Expected Behavior:
Two model calls produce two generate_content openai/gpt-4o spans (root and child), with duration and token metrics for both.
Observed Behavior:
child.canonical_model.model = openai/gpt-4o
tracing._agent_model_name(child) = gemini-3.5-flash
model calls made: 2
SPAN 'generate_content openai/gpt-4o' | agent = root <- child's span is missing
gen_ai.client.inference.operation.input_tokens count = 1
Environment Details:
- ADK Library Version (pip show google-adk): 2.13.0 (also on
mainat e9e8f61) - Desktop OS: macOS
- Python Version (python -V): 3.11.16
Model Information:
- Are you using LiteLLM: Yes
- Which model is being used:
openai/gpt-4othrough LiteLLM (a fakeBaseLlmin the repro)
🟡 Optional Information
Minimal Reproduction Code:
from google.adk.agents import LlmAgent
from google.adk.models.lite_llm import LiteLlm
from google.adk.telemetry import tracing
child = LlmAgent(name="child", instruction="answer") # inherits the model
root = LlmAgent(name="root", model=LiteLlm("openai/gpt-4o"), sub_agents=[child])
print(child.canonical_model.model) # openai/gpt-4o
print(tracing._agent_model_name(child)) # gemini-3.5-flashHow often has this issue occurred?:
- Always (100%), for a sub-agent that inherits a non-Gemini model
Activity
surajksharma07 commentedon Nov 2, 2026
@adityasingh2400 Reproduced both directions on 2.13.0 and current main: the child under an openai/gpt-4o root loses its generate_content span, and a child inheriting Gemini while the default model is non-Gemini gets recorded by ADK as well as the instrumentor.
Walking up parent_agent for the model name in _agent_model_name, like #7893 does, fixed both cases for me with the default model still as the fallback. The telemetry tests I could run locally showed no new failures.
Could you check that on your side and add a test for the duplicate case too? It might also be worth trying llm_request.model first at those call sites since that’s the model actually sent. Please test it well with the instrumentor on before #7893 goes for review.
adityasingh2400 commentedon Nov 3, 2026
Thanks, @surajksharma07. I added
test_generate_content_span_with_inherited_gemini_modelto #7893 (40a79f0). It fails on main and passes with the fix.I also ran the Gemini case end to end with the real instrumentor (0.7b2), answering Gemini's HTTP call with canned responses. That showed I'd overstated it here: the child isn't recorded twice. When ADK records a call itself,
maybe_without_genai_instrumentationtakes the instrumentor off it, so on main the child's span came from ADK while its Gemini root's came from the instrumentor. With the fix, both come from the instrumentor. I've corrected the issue text and the PR description.On
llm_request.model:basic.pysets it fromagent.canonical_model.model, so with the ancestor walk it matches_agent_model_namefor every request the flow builds. The two differ only if abefore_model_callbackrenames the model. Switching would also bypass_is_gemini_agent, whichtest_functional.py,test_node_functional.pyandtest_instrumentation.pypatch, so I've left it out of this PR. I can follow up if a maintainer wants it.In
tests/unittests/telemetry, the same 13 schema-recording cases fail locally with and without this change; everything else passes.