Repository navigation
feat(live): update system instruction and tools in-session via LiveRquestQueue - #7945
Open
rfan-debug wants to merge 3 commits into
Open
rfan-debug wants to merge 3 commits into
rfan-debug wants to merge 3 commits into
Conversation
…questQueue
google-genai 2.32 added AsyncSession.send_context_update, but the live
runner offered no way to use it: callers reached into the private
_gemini_session, which left the new tools unknown to ADK's function-call
dispatch and reverted the update on every reconnect.
Add LiveContextUpdate and LiveRequestQueue.send_context_update. The flow
rewrites the LlmRequest first, registering the tools in tools_dict and
storing the instruction where Gemini.connect reads it, so function calls
execute and reconnects replay the latest context, then forwards the update
through BaseLlmConnection.send_context_update.
live_request_queue.send_context_update(
system_instruction=CHECKOUT_INSTRUCTION,
tools=[FunctionTool(lookup_order)],
)
Both fields are part of the model preamble, so an update invalidates the
Live prefix cache; the docstrings say so and point per-turn context to
send_content(partial=True) or tool responses instead.
…questQueue
google-genai 2.32 added AsyncSession.send_context_update, but the live
runner offered no way to use it: callers reached into the private
_gemini_session, which left the new tools unknown to ADK's function-call
dispatch and reverted the update on every reconnect.
Add LiveContextUpdate and LiveRequestQueue.send_context_update. The flow
rewrites the LlmRequest first, registering the tools in tools_dict and
storing the instruction where Gemini.connect reads it, so function calls
execute and reconnects replay the latest context, then forwards the update
through BaseLlmConnection.send_context_update.
live_request_queue.send_context_update(
system_instruction=CHECKOUT_INSTRUCTION,
tools=[FunctionTool(lookup_order)],
)
Both fields are part of the model preamble, so an update invalidates the
Live prefix cache; the docstrings say so and point per-turn context to
send_content(partial=True) or tool responses instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
run_live: change instruction/tools or transfer agents on the open Live connection (send_context_update, in-session transfer) #7692Proposed change
google-genai==2.32added AsyncSession.send_context_update, but the live runner offered no way to use it: callers reached into the private _gemini_session, which left the new tools unknown to ADK's function-call dispatch and reverted the update on every reconnect.This change adds LiveContextUpdate and LiveRequestQueue.send_context_update. The flow rewrites the LlmRequest first, registering the tools in tools_dict and storing the instruction where Gemini.connect reads it, so function calls execute and reconnects replay the latest context, then forwards the update through BaseLlmConnection.send_context_update.
Both fields are part of the model preamble, so an update invalidates the Live prefix cache; the docstrings say so and point per-turn context to send_content(partial=True) or tool responses instead.
Testing Plan
Unit tests (all under
tests/unittests/):live/test_live_request_queue.py: queueing,[]vsNone, missing-fieldValueError,extra='forbid'.live/test_live_llm_flow.py: tools replaced intools_dict/config.tools/live_connect_config.toolsand sent;[]clears; instruction stored whereGemini.connectreads it and sent asrole='system'Content;send_to_modelapplies the update before content in the same request.models/test_gemini_llm_connection.py: SDK forwarding (incl. empty list), unset fields stayNone, SDK-floor error.Unit Tests:
Please include a summary of passed
pytestresults.Manual End-to-End (E2E) Tests:
Manual E2E (to do before opening the PR):
adk webwith a live agent, send a context update from a tool callback, confirm (a) the model follows the new instruction, (b) a call to a swapped-in tool is executed by ADK, (c) after a forced reconnect the new tools are still declared.Doc change: google/adk-docs#2545
Checklist