Skip to content

fix(AIC-3570): re-resolve graph flags on every invoke and stream - #187

Merged
jeffdupont merged 1 commit into
mainfrom
jdupont/aic-3570-graph-no-config-cache
Oct 21, 2026
Merged

fix(AIC-3570): re-resolve graph flags on every invoke and stream#187
jeffdupont merged 1 commit into
mainfrom
jdupont/aic-3570-graph-no-config-cache

Conversation

@jeffdupont

@jeffdupont jeffdupont commented Oct 20, 2026 •
edited by atlassian Bot
Loading

Copy link
Copy Markdown
Contributor

Fixes AIC-3570.

Why

GraphInstance kept the built graph in self._cache, keyed by the JSON of the context, for the life of the instance. An app that reuses one graph() never saw flag changes again: new node instructions, a new model, or turning the graph off (the kill switch) did not reach it. A transient variation failure was cached the same way, so the graph stayed disabled for that context. Flag evaluation is local and cheap, so there is nothing worth caching. The spec rule is updated to match in the monorepo PR (TESTING.md §3.11).

The other half of the ticket, one runId per run, was already fixed on main (invoke and stream each mint a fresh runId). Those tests still pass.

Fix

  • packages/client/src/launchdarkly_ai_server/graph.py: invoke and stream call _build_graph(...) on every call. Removed self._cache, MAX_GRAPH_CACHE_SIZE and the unused json import. Updated the _RUN_TRACK_DATA comment, which described the cache.
  • MAX_GRAPH_CACHE_SIZE was not exported from launchdarkly_ai_server (not in __init__.py or test_public_surface.py). It could still be imported from the launchdarkly_ai_server.graph module, so code that did that will break. Pre-1.0 removal is approved.

Testing

  • packages/client/tests/test_graph.py: test_node_variations_resolved_only_once encoded the bug, so it is now test_node_variations_resolved_on_every_call (the variation call count doubles across two calls). Deleted test_cache_is_bounded and test_equal_content_contexts_share_cache_entry, since there is no cache any more. Added test_flag_change_reaches_reused_graph: instructions changed from v1 to v2 reach the second call, then disabling the graph makes invoke and stream raise. Added test_failed_resolution_is_not_reused. test_per_call_variables_isolated is unchanged.
  • With the fix reverted, the 3 new or changed tests fail. With the fix, all 33 tests in test_graph.py pass.
  • make test: 2544 passed, 14 skipped. make typecheck: no issues in 75 files. make lint: all checks passed. make format-check: clean.

Related: launchdarkly/js-ai-sdk#145, launchdarkly/go-ai-sdk#11, https://github.com/launchdarkly/ai-sdks-monorepo/pull/71

🤖 Generated with Claude Code

@jeffdupont @claude
fix(AIC-3570): re-resolve graph flags on every invoke and stream
4dd2663 
graph() cached the built graph per context for the life of the instance, so
flag changes and the graph kill switch never reached a reused instance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 20, 2026
Merged
Merged
jeffdupont added a commit to launchdarkly/go-ai-sdk that referenced this pull request Oct 21, 2026
@jeffdupont @claude
fix(AIC-3570)!: re-resolve graph flags on every invoke and stream (#11)
ada5a25 
Fixes [AIC-3570](https://launchdarkly.atlassian.net/browse/AIC-3570).

## Why

`AgentGraph` kept each enabled resolved definition, keyed by the JSON of
the context, for its whole life, with up to `MaxGraphCacheSize` (512)
entries. A service that reuses one `Graph()` never saw flag changes
again: new node instructions, a new model, or turning the graph off (the
kill switch) did not reach it. Flag evaluation is local and cheap, so
there is nothing worth caching. The spec rule is updated to match in the
monorepo PR (TESTING.md §3.11).

The other half of the ticket, one `runId` per run, was already fixed on
main (each run sets its own `RunID`). Those tests still pass.

## Fix

- `aiserver/graph.go`: `definition()` is now `return
resolveDefinition(ctx, g.key, userCtx, g.opts)`. Removed the
`mu`/`cache`/`order` fields and the test-only `cacheLen`. Updated the
`RunID` comment, which described the cache.
- **Breaking:** removed the public constant `MaxGraphCacheSize`. Also
removed it from `aiserver/surface_test.go` and `aiserver/agents.md`.
Pre-1.0 removal is approved. The monorepo has no Go api-surface list, so
nothing else changes.

## Testing

- `aiserver/graph_test.go`: `TestGraphCachingAndIsolation` encoded the
bug, so it is now `TestGraphResolvesEachCallAndIsolation`. It checks
that the graph and node flags are evaluated on both calls, and keeps the
variables isolation check. The cache-size loop is gone. Added
`TestGraphSeesFlagChanges`: instructions changed from v1 to v2 reach the
second call, then disabling the graph makes `Invoke` and `Stream` return
`ErrGraphDisabled`. The transient-failure case is already covered by
`TestGraphFailedResolutionIsNotCached`, which still passes. Go already
skipped caching disabled results.
- With the fix reverted, the 2 new or changed tests fail ("flags not
re-resolved per call", "second call instructions = v1").
- `make test` (`go test -race ./...`): all 16 packages ok, 844 tests
passed. `go vet ./...` and `make lint` are clean.

Related: launchdarkly/js-ai-sdk#145,
launchdarkly/python-ai-sdk#187,
launchdarkly/ai-sdks-monorepo#71

🤖 Generated with [Claude Code](https://claude.com/claude-code)


[AIC-3570]:
https://launchdarkly.atlassian.net/browse/AIC-3570?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@jeffdupont
jeffdupont merged commit 8fdc4d4 into main Oct 21, 2026
8 checks passed
@jeffdupont
jeffdupont deleted the jdupont/aic-3570-graph-no-config-cache branch October 21, 2026 14:05
jeffdupont added a commit to launchdarkly/js-ai-sdk that referenced this pull request Oct 21, 2026
@jeffdupont @claude
fix(AIC-3570): re-resolve graph flags on every invoke and stream (#145)
ce4f72e 
Fixes [AIC-3570](https://launchdarkly.atlassian.net/browse/AIC-3570).

## Why

`graph()` kept the built graph (topology plus every node's config) in a
per-context `WeakMap` for the life of the returned caller. An app that
creates one `graph()` and reuses it for a user never saw flag changes
again: new instructions, a new model, or turning the graph off (the kill
switch) did not reach that caller. A transient variation failure was
cached the same way, so the graph stayed disabled for that context. Flag
evaluation is local and cheap, so there is nothing worth caching. The
spec rule is updated to match in the monorepo PR (TESTING.md §3.11).

The other half of the ticket, one `runId` per run, was already fixed on
main (`graph.ts` mints a fresh `runId` per stream call). Those tests
still pass.

## Fix

- `packages/client/src/graph.ts`: removed `nodeCache` and `resolveBuilt`
from `graphInternal`. `streamEvents` calls `buildGraph` directly on
every call.
- `invoke()` no longer resolves the graph itself before draining
`stream()`, so each call builds the graph once, not twice. `stream()`
still throws `Agent graph "<key>" is disabled`, and `invoke()` passes
that error up.
- Callers of `graphInternal` (`graph()`, claude-agents,
langchain-agents, vercel-agents, openai-agents) did not depend on the
cache.

## Testing

- `packages/client/src/__tests__/graph.test.ts`: the test "resolves node
variations only once per graph() instance" encoded the bug, so it now
asserts the opposite: the variation call count doubles across two calls.
Added: a node flag changed from v1 to v2 between calls reaches the
second call; disabling the graph flag makes the next `invoke()` and
`stream()` throw; a failed variation on call 1 does not carry over, and
call 2 succeeds. The per-call variables isolation test is unchanged.
- With the fix reverted, the 4 new or changed tests fail. With the fix,
all 66 tests in graph.test.ts pass.
- `yarn build`, `yarn typecheck`, `yarn code:check`, `yarn workspace
@launchdarkly/ai-server api:check` all pass. `yarn test` passes in every
workspace (client: 1404 passed, 11 skipped).

Related: launchdarkly/python-ai-sdk#187,
launchdarkly/go-ai-sdk#11,
launchdarkly/ai-sdks-monorepo#71

🤖 Generated with [Claude Code](https://claude.com/claude-code)


[AIC-3570]:
https://launchdarkly.atlassian.net/browse/AIC-3570?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot mentioned this pull request Oct 21, 2026
Merged
jeffdupont pushed a commit that referenced this pull request Oct 23, 2026
@github-actions
chore: release main (#166)
662d3f2 
🤖 I have created a release *beep* *boop*
---


<details><summary>launchdarkly-ai-server: 0.4.0</summary>

##
[0.4.0](launchdarkly-ai-server-0.3.0...launchdarkly-ai-server-0.4.0)
(2026-10-23)


### ⚠ BREAKING CHANGES

* **evaluations:** keep the management API client internal
([#202](#202))
* **evaluations:** rename base uri to match spec
([#199](#199))
* **evaluations:** add test proving harness edge case matches spec
([#200](#200))
* **evaluations:** route handlers by provider and mode
* **evaluations:** mark evaluations from code as experimental
([#157](#157))

### Experimental

* **evaluations:** add test proving harness edge case matches spec
([#200](#200))
([fa7162e](fa7162e))
* **evaluations:** keep the management API client internal
([#202](#202))
([17270ed](17270ed))
* **evaluations:** mark evaluations from code as experimental
([#157](#157))
([1389765](1389765))
* **evaluations:** rename base uri to match spec
([#199](#199))
([7d93f9c](7d93f9c))
* **evaluations:** route handlers by provider and mode
([f62591f](f62591f))
* **evaluations:** Set JSON as standard for output to pass to judge
([#203](#203))
([ffc728b](ffc728b))
* **skills:** Correct the HTTP 403 advice now that server-side FDv2 is
available to every account
([#178](#178))
([b17e322](b17e322))


### Features

* **agui:** add a shared reader for handler streams
([#164](#164))
([fc08dd5](fc08dd5))
* **AIC-3534:** add a TypeSafe Jev judge handler
([#149](#149))
([1b41ea4](1b41ea4))
* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))
* **langchain:** emit AG-UI text and tool events
([#163](#163))
([ab5e9e1](ab5e9e1))
* **openai:** emit AG-UI text and tool events
([#161](#161))
([2b1b2d4](2b1b2d4))


### Bug Fixes

* **AIC-3570:** re-resolve graph flags on every invoke and stream
([#187](#187))
([8fdc4d4](8fdc4d4))
* **AIC-3571:** visit every child before its parent in reverse_traverse
([#189](#189))
([87c73d9](87c73d9))
* **AIC-3572:** treat a variation without _ldMeta.enabled as disabled
([#184](#184))
([7a0d260](7a0d260))
* **AIC-3574:** build the LaunchDarkly client off the event loop
([#185](#185))
([bf03b6c](bf03b6c))
* **AIC-3576:** record deferred judge scores under the evaluated config
([#190](#190))
([354d3f1](354d3f1))
* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
* **AIC-3596:** name the [otel] extra in the missing-OTel warning
([#186](#186))
([90ce7bf](90ce7bf))
* **client:** match a handler's provides_for after normalizing its mode
([f62591f](f62591f))
* give native graph spans their LaunchDarkly identity
([#121](#121))
([e863b6d](e863b6d))
</details>

<details><summary>launchdarkly-ai-python: 0.3.0</summary>

##
[0.3.0](launchdarkly-ai-python-0.2.0...launchdarkly-ai-python-0.3.0)
(2026-10-23)


### ⚠ BREAKING CHANGES

* **evaluations:** mark evaluations from code as experimental
([#157](#157))

### Experimental

* **evaluations:** mark evaluations from code as experimental
([#157](#157))
([1389765](1389765))
</details>

<details><summary>launchdarkly-ai-claude-agents: 0.2.6</summary>

##
[0.2.6](launchdarkly-ai-claude-agents-0.2.5...launchdarkly-ai-claude-agents-0.2.6)
(2026-10-23)


### Features

* **claude:** emit AG-UI text and tool events
([#165](#165))
([3f3a0ce](3f3a0ce))
* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))


### Bug Fixes

* **AIC-3571:** visit every child before its parent in reverse_traverse
([#189](#189))
([87c73d9](87c73d9))
* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
* **claude-agents:** fail the run when the CLI reports an API error as
success
([#174](#174))
([3d1a768](3d1a768))
* give native graph spans their LaunchDarkly identity
([#121](#121))
([e863b6d](e863b6d))
</details>

<details><summary>launchdarkly-ai-claude-messages: 0.2.5</summary>

##
[0.2.5](launchdarkly-ai-claude-messages-0.2.4...launchdarkly-ai-claude-messages-0.2.5)
(2026-10-23)


### Features

* **claude:** emit AG-UI text and tool events
([#165](#165))
([3f3a0ce](3f3a0ce))
* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))


### Bug Fixes

* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
</details>

<details><summary>launchdarkly-ai-openai-agents: 0.2.6</summary>

##
[0.2.6](launchdarkly-ai-openai-agents-0.2.5...launchdarkly-ai-openai-agents-0.2.6)
(2026-10-23)


### Features

* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))
* **openai:** emit AG-UI text and tool events
([#161](#161))
([2b1b2d4](2b1b2d4))


### Bug Fixes

* **AIC-3571:** visit every child before its parent in reverse_traverse
([#189](#189))
([87c73d9](87c73d9))
* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
* give native graph spans their LaunchDarkly identity
([#121](#121))
([e863b6d](e863b6d))
</details>

<details><summary>launchdarkly-ai-vercel-agents: 0.1.1</summary>

##
[0.1.1](launchdarkly-ai-vercel-agents-0.1.0...launchdarkly-ai-vercel-agents-0.1.1)
(2026-10-23)


### Features

* **AIC-3408:** add Vercel AI SDK messages and agents adapters
([#103](#103))
([64d679e](64d679e))
</details>

<details><summary>launchdarkly-ai-litellm-agents: 0.1.1</summary>

##
[0.1.1](launchdarkly-ai-litellm-agents-0.1.0...launchdarkly-ai-litellm-agents-0.1.1)
(2026-10-23)


### Features

* add LiteLLM messages and agents integrations
([#102](#102))
([f2e9886](f2e9886))
</details>

<details><summary>launchdarkly-ai-openai-messages: 0.2.5</summary>

##
[0.2.5](launchdarkly-ai-openai-messages-0.2.4...launchdarkly-ai-openai-messages-0.2.5)
(2026-10-23)


### Features

* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))
* **openai:** emit AG-UI text and tool events
([#161](#161))
([2b1b2d4](2b1b2d4))


### Bug Fixes

* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
</details>

<details><summary>launchdarkly-ai-vercel-messages: 0.1.1</summary>

##
[0.1.1](launchdarkly-ai-vercel-messages-0.1.0...launchdarkly-ai-vercel-messages-0.1.1)
(2026-10-23)


### Features

* **AIC-3408:** add Vercel AI SDK messages and agents adapters
([#103](#103))
([64d679e](64d679e))
</details>

<details><summary>launchdarkly-ai-litellm-messages: 0.1.1</summary>

##
[0.1.1](launchdarkly-ai-litellm-messages-0.1.0...launchdarkly-ai-litellm-messages-0.1.1)
(2026-10-23)


### Features

* add LiteLLM messages and agents integrations
([#102](#102))
([f2e9886](f2e9886))
</details>

<details><summary>launchdarkly-ai-langchain-agents: 0.2.6</summary>

##
[0.2.6](launchdarkly-ai-langchain-agents-0.2.5...launchdarkly-ai-langchain-agents-0.2.6)
(2026-10-23)


### Features

* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))
* **langchain:** emit AG-UI text and tool events
([#163](#163))
([ab5e9e1](ab5e9e1))


### Bug Fixes

* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
* give native graph spans their LaunchDarkly identity
([#121](#121))
([e863b6d](e863b6d))
</details>

<details><summary>launchdarkly-ai-langchain-messages: 0.2.5</summary>

##
[0.2.5](launchdarkly-ai-langchain-messages-0.2.4...launchdarkly-ai-langchain-messages-0.2.5)
(2026-10-23)


### Features

* **client:** add configure for SDK defaults
([#158](#158))
([3736767](3736767))
* **langchain:** emit AG-UI text and tool events
([#163](#163))
([ab5e9e1](ab5e9e1))


### Bug Fixes

* **AIC-3577:** send object tool results to the model as JSON
([#188](#188))
([dead4bd](dead4bd))
</details>

<details><summary>launchdarkly-ai-opencode-agents: 0.2.0</summary>

##
[0.2.0](launchdarkly-ai-opencode-agents-0.1.0...launchdarkly-ai-opencode-agents-0.2.0)
(2026-10-23)


### ⚠ BREAKING CHANGES

* **opencode:** start and stop opencode serve in the handler
([#198](#198))

### Features

* **opencode-agents:** add an OpenCode handler
([#159](#159))
([fc08e8c](fc08e8c))
* **opencode-agents:** emit the AG-UI text and tool subset
([#162](#162))
([7677621](7677621))
* **opencode-agents:** open one OpenCode session per call
([#179](#179))
([b225113](b225113))
* **opencode-agents:** render agent instruction templates
([#182](#182))
([83175aa](83175aa))
* **opencode:** start and stop opencode serve in the handler
([#198](#198))
([0748dcf](0748dcf))


### Bug Fixes

* **opencode-agents:** own the HTTP client
([#194](#194))
([ed2d419](ed2d419))
* **opencode-agents:** skip judges and record structured history
([#177](#177))
([6fadf1d](6fadf1d))
</details>

<details><summary>launchdarkly-ai-typesafe: 0.1.1</summary>

##
[0.1.1](launchdarkly-ai-typesafe-0.1.0...launchdarkly-ai-typesafe-0.1.1)
(2026-10-23)


### Features

* **AIC-3534:** add a TypeSafe Jev judge handler
([#149](#149))
([1b41ea4](1b41ea4))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewklatzke andrewklatzke andrewklatzke approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jeffdupont @andrewklatzke