[LiteLLM] role='tool_responses' for Gemma-4 models breaks tool calling on LM Studio (strict OpenAI role validation)

[armandokeller]

Describe the Bug

_content_to_message_param in google/adk/models/lite_llm.py forces
role="tool_responses" for the tool-result message whenever the model
name matches _is_gemma4_model (regex gemma-?4), regardless of which
serving backend is actually handling the request:

tool_role = "tool_responses" if _is_gemma4_model(model) else "tool"

This was introduced for Ollama (#5650) and broadened to also match
hyphenated gemma-4 names to cover vLLM/llama.cpp (#6334). Both of
those fixes assume every llama.cpp-family server passes the role
value straight through to the model's chat template without validating
it against the standard OpenAI role enum.

LM Studio does not — its OpenAI-compatible endpoint validates
messages[].role strictly and rejects tool_responses outright, even
though LM Studio also serves GGUF models via a llama.cpp-based runtime.
The very backend #6334 was broadened to support ("vLLM, llama.cpp")
turns out not to be uniform: LM Studio's own compatibility layer sits
in front of llama.cpp and enforces the standard role set before the
model's chat template ever sees the message.

Net effect: any ADK agent with at least one tool, running a Gemma-4
model through LiteLLM against LM Studio, fails on the very first
tool-result turn.

Steps to Reproduce

1. Load any Gemma-4-named model in LM Studio (tested with
   google/gemma-4-e4b) and start its local server
   (http://127.0.0.1:1234/v1, OpenAI-compatible).
2. Run the following minimal ADK agent, which has a single trivial
   tool unrelated to any specific application:

import asyncio
from google.adk.agents import Agent
from google.adk.models.lite_llm import LiteLlm
from google.adk.runners import InMemoryRunner
from google.adk.tools.function_tool import FunctionTool
from google.genai import types

def roll_dice() -> dict:
    """Roll a six-sided die.

    Returns:
        A dict with the result.
    """
    return {"result": 4}

model = LiteLlm(
    model="openai/google/gemma-4-e4b",
    api_base="http://127.0.0.1:1234/v1",
    api_key="not-needed",
)
agent = Agent(name="t", model=model, instruction="Be helpful.", tools=[FunctionTool(roll_dice)])

async def main():
    runner = InMemoryRunner(agent, app_name="t")
    session = await runner.session_service.create_session(app_name="t", user_id="u")
    message = types.Content(role="user", parts=[types.Part(text="Roll the dice for me")])
    async for event in runner.run_async(user_id="u", session_id=session.id, new_message=message):
        if event.is_final_response() and event.content and event.content.parts:
            print(event.content.parts[0].text)

asyncio.run(main())

Expected Behavior

The agent calls roll_dice, receives the result, and produces a final
text response.

Observed Behavior

The first request (tool declaration only, no prior tool call) succeeds
normally and the model correctly emits a tool_calls response. The
second request — sent after the tool runs, carrying the tool result —
is rejected by LM Studio:

2026-07-25 21:51:45 [ERROR]
 [google/gemma-4-e4b] Invalid 'content': 'content' field must be a string or an array of objects.

LM Studio's own debug log shows the exact outgoing message that trips
this:

{
  "role": "assistant",
  "tool_calls": [
    {
      "type": "function",
      "id": "200618556",
      "function": { "name": "roll_dice", "arguments": "{}" }
    }
  ]
},
{
  "role": "tool_responses",
  "tool_call_id": "200618556",
  "content": "{\"result\": 4}"
}

role: "tool_responses" is not a role LM Studio's OpenAI-compatible
validator accepts (only the standard system/user/assistant/tool
set is), so the request as a whole is rejected. The reported error text
("content field must be...") is misleading — the actual problem is
the unrecognized role value, not the content field.

Environment

• google-adk: 2.5.0
• litellm: 1.93.0
• LM Studio: OpenAI-compatible server at /v1, google/gemma-4-e4b
  (GGUF, llama.cpp-based runtime)
• Python: 3.13
• OS: Windows

Additional Context

Related: #5650 (Ollama, original fix), #6334 (broadened to hyphenated
gemma-4 naming for vLLM/llama.cpp). This report is about a case
neither of those covers: a llama.cpp-based server (LM Studio) that
does validate the OpenAI role enum strictly, unlike whatever
vLLM/llama.cpp setup #6334 was tested against.

Possible directions for a fix (not prescriptive, just what occurred to
us):

• Make the tool-result role a LiteLlm constructor option instead of
  an unconditional model-name-based inference, so callers can opt out
  per backend.
• Detect the actual serving backend (e.g. via a capability probe, or a
  custom_llm_provider/base-url heuristic) rather than matching on
  model name alone, since "serves a gemma-4-named GGUF" does not
  imply "accepts a non-standard role value."
• At minimum, catch the resulting BadRequestError and retry once with
  the standard role="tool", so a stricter backend degrades instead of
  hard-failing.

Happy to provide more logs or test a patch against our setup if useful.

[KXHXK]

I'd like to work on this if the direction is acceptable. To avoid brittle backend or base-URL heuristics, I propose adding an explicit LiteLlm tool-result role override while preserving the current Gemma-4 auto-detection as the default. I would thread the resolved role through both _content_to_message_param and _ensure_tool_results, so real and synthesized tool-result messages stay consistent, and add offline regression tests for the standard-role override plus the existing Ollama/Gemma behavior. Please let me know if this scope is approved or if you prefer retry/capability detection instead.

[surajksharma07]

@armandokeller Confirmed this on our end — openai/google/gemma-4-e4b trips the same gemma-?4 match #6334 broadened, so ADK sends role="tool_responses" even though LM Studio's stricter validator won't accept it.
The "content field" error is a red herring — the role is the real problem.

We've put together a tool_result_role override on LiteLlm so you can pin it to "tool" for your LM Studio setup without touching the Ollama/vLLM default.

Could you try that against your gemma-4-e4b server and confirm the tool-result turn goes through?

[armandokeller]

Thanks for confirming and for digging into a fix so quickly!

Just to make sure I'm looking in the right place: I found #6334 / PR #6340 (broadening _is_gemma4_model to also match the hyphenated HF-style naming), which is already merged in the google-adk 2.5.0 I have installed. That's actually what causes this report in the first place, since it now also matches LM Studio-hosted model names. I checked main and don't see a tool_result_role parameter anywhere yet, so I don't think the override you mentioned is there.

Could you point me to the branch or PR with the tool_result_role override? I'd like to build against it directly so I can test it against the exact repro from this issue (LM Studio serving google/gemma-4-e4b, same minimal roll_dice tool) and report back whether the tool-result turn completes cleanly.