Fix the simulations CI job: lk agent simulate needs the text subcommand - #107
Merged
Merged
Conversation
Since livekit-cli v2.18, the root `lk agent simulate` command only prints help and exits 0; the mode is a required subcommand (`text` or `audio`). The simulations job has been running the old flat form, so it has been passing without starting a single simulation. Switch the job and the four places that show the command to `lk agent simulate text --scenarios scenarios.yaml`. Text mode is the right mode for CI: it exercises the LLM, tools and conversation logic without the audio pipeline, and is scheduled so it doesn't compete with live sessions for inference.
u9g
approved these changes
Sep 25, 2026
bcherry
pushed a commit
that referenced
this pull request
Sep 25, 2026
Keeps the rewritten Testing paragraph and carries over #107's `lk agent simulate text` fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U7yszNU7t5aCHJsKLmU4hv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The simulations CI job runs
lk agent simulate --scenarios scenarios.yaml. Since livekit-cli v2.18 the rootsimulatecommand only prints help and exits 0 — the mode is a required subcommand (textoraudio). So the job has been green without running a single simulation. This switches the job, and the four places that show the command (README, AGENTS.md, thescenarios.yamlheader, and the workflow comment), to:Verify with
lk agent simulate --helpon a current CLI; the workflow installs the latest release, so it has been affected since that release.Why text mode
It exercises the LLM, tools and conversation logic without the STT/TTS pipeline, it's deterministic enough for CI, and LiveKit schedules text-mode simulations so they don't compete with live sessions for inference. Audio runs execute in real time and bill the speech providers every turn — right for a release candidate, not a recurring job.
Note
The first green run after this merges will be the first time the scenarios have actually executed in a while, so a failure there is information, not necessarily a regression from this change.