Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,14 +88,14 @@ jobs:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: tangle-network/agent-eval
ref: 4a848f092064e7e0320c716ff472c3f53446409e # v0.144.10
ref: dc592024c3049a677b90f3035fcda883b6efd9f3 # v0.144.12
path: .cohort/agent-eval
persist-credentials: false

- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: tangle-network/agent-knowledge
ref: 62dd504f0509d9795011efc8addcfdb6971745b8 # v7.2.0
ref: 5c5a9f58e35dc44992e7b8cd99911ef2c99cdd0b # v7.2.1
path: .cohort/agent-knowledge
persist-credentials: false

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -143,15 +143,15 @@ jobs:
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: tangle-network/agent-eval
ref: 4a848f092064e7e0320c716ff472c3f53446409e # v0.144.10
ref: dc592024c3049a677b90f3035fcda883b6efd9f3 # v0.144.12
path: .cohort/agent-eval
persist-credentials: false

- name: Check out exact Knowledge release
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: tangle-network/agent-knowledge
ref: 62dd504f0509d9795011efc8addcfdb6971745b8 # v7.2.0
ref: 5c5a9f58e35dc44992e7b8cd99911ef2c99cdd0b # v7.2.1
path: .cohort/agent-knowledge
persist-credentials: false

Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

## 0.132.1

- Add `candidatePopulation` to `improve(...)` results so consumers can inspect every verified optimizer candidate, including its exact profile, Interface diffs, parent lineage, and selection score.
- Consume Agent Eval 0.144.12 and Agent Knowledge 7.2.1 as one compatible dependency set.

## 0.131.7

- Add `superviseDispatch(...)` so `agent-eval` profile matrices admit and record a recursive Runtime tree before it spends.
Expand Down
46 changes: 44 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ pnpm add @tangle-network/agent-runtime @tangle-network/agent-eval @tangle-networ
- [Quickstart](#quickstart-offline-no-api-keys)
- [What you do with it](#what-you-do-with-it)
- [Run a chat turn](#run-a-chat-turn)
- [Retain and reconnect a run](#retain-and-reconnect-a-run)
- [Supervise a team of agents](#supervise-a-team-of-agents)
- [Improve an agent](#improve-an-agent)
- [Improve a knowledge base](#improve-a-knowledge-base)
Expand Down Expand Up @@ -115,6 +116,42 @@ return new Response(result.body, { headers: { 'content-type': result.contentType
For a stream reconnect, call `streamPrompt` with the same `executionId` and the last event id the client received.
For a repeated initial dispatch, reuse both `sessionId` and `turnId`; `executionId` alone is not an idempotency key.

### Retain and reconnect a run

Use the retained-run API when the provider owns a job that must outlive one HTTP reader or application process.
The provider must advertise exact run identity, replay, result identity, and idempotent cancellation.

```ts
import { reconnectRetainedRun, startRetainedRun } from '@tangle-network/agent-runtime'

const run = await startRetainedRun({
provider,
environment: { idempotencyKey: 'workspace-42', profile },
turn: { turnId: 'turn-7', prompt: 'Finish the migration and run its tests.' },
identity: { sessionId: 'thread-42', executionId: 'execution-7' },
})

await journal.write(run.controlRef)

for await (const event of run.events()) {
await journal.write(event)
}

const recovered = await reconnectRetainedRun({
provider: freshProvider,
controlRef: await journal.readControlRef(),
})
if (!recovered) throw new Error('the provider no longer retains this environment')

const snapshot = await recovered.status({ waitMs: 30_000 })
const result = await recovered.result()
```

Persist `controlRef` before acknowledging dispatch to the caller.
Persist each event cursor and sequence before advancing the visible transcript.
`reconnectRetainedRun` reconstructs a client from those values and rejects any provider, environment, session, execution, run, or digest mismatch.
An unknown provider result remains unknown; the runtime never converts it into success or confirmed cancellation.

### Supervise a team of agents

One supervisor spawns and steers workers toward a goal. Where the workers run (an in-process loop, or a sandboxed coding harness) is one data value; the budget, journaling, and stopping are handled for you.
Expand Down Expand Up @@ -238,11 +275,15 @@ With `resume: 'if-compatible'`, agent-eval resumes only when the saved run ident
Set `trustResumeState: true` only when that run directory is private to the current operator.
Use `resume: 'required'` to fail when no matching run exists.
`result.provenance` reports the upstream package, run ID, resume status, evaluation count, and artifact directory.
`result.candidatePopulation` verifies and joins callback observations with an optimizer's official candidate graph.
It returns every unique candidate as a complete profile with ordered Interface diffs, or as an explicit materialization refusal.
GEPA candidates retain exact parent indices and selection scores; callback-only proposals report lineage as unavailable.
Methods without either artifact return `status: 'unavailable'` instead of treating the winner as the full population.
There is no local fallback.
Install its optional Python process before using it:

```bash
python -m pip install "agent-eval-rpc==0.144.8"
python -m pip install "agent-eval-rpc==0.144.12"
python -m pip install "gepa[full]==0.1.4"
```

Expand All @@ -256,7 +297,7 @@ python -m pip install "gepa[full] @ git+https://github.com/gepa-ai/gepa.git@f919
Use `officialSkillOpt(...)` for Microsoft's SkillOpt:

```bash
python -m pip install "agent-eval-rpc==0.144.8"
python -m pip install "agent-eval-rpc==0.144.12"
python -m pip install "skillopt @ git+https://github.com/microsoft/SkillOpt.git@61735e3922efc2b90c6d6cab561e62e98452ca90"
```

Expand Down Expand Up @@ -490,6 +531,7 @@ The general-purpose pieces, by import path. Every export with its one-line summa
| Primitive | What it does | Import |
|---|---|---|
| Chat-turn runtime | Stream and persist one production chat turn (`handleChatTurn`); derive its stable execution and turn identity (`deriveExecutionId`); normalize any backend's stream into one event shape (`streamAgentTurn`) | `/durable` · `/kernel` |
| Retained provider runs | Start one detached provider job, replay exact events, reconnect after restart, continue its native context, and cancel idempotently (`startRetainedRun`, `reconnectRetainedRun`) | root |
| Tool-call loop | Run one model turn, execute requested tools, feed results back, and stop on completion, repetition, time, or cost limits (`runToolLoop`, `streamToolLoop`) | `/tool-loop` |
| Supervision | One agent spawns, budgets, and steers workers toward a goal (`supervise`, `delegate`), on an in-process loop or a sandboxed coding harness | `/kernel` · `/mcp` |
| Loop kernel + combinators | Write a driver (`plan`/`decide`) and run it (`runAgentRounds`), or compose fixed shapes: refine (`loopUntil`), best-of-N (`fanout`), chain (`pipeline`), multi-judge (`panel`) | `/kernel` |
Expand Down
4 changes: 4 additions & 0 deletions bench/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog

## 0.8.3

- Consume Runtime 0.132.1, Eval 0.144.12, and Knowledge 7.2.1 as one compatible dependency set.

## 0.8.2

- Consume Runtime 0.131.7, Eval 0.144.10, and Knowledge 7.2.0.
Expand Down
2 changes: 1 addition & 1 deletion bench/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@tangle-network/agent-bench",
"version": "0.8.2",
"version": "0.8.3",
"type": "module",
"description": "Benchmark adapters and execution for agent-runtime across coding, tool-use, RAG, memory, browser, and terminal tasks.",
"repository": {
Expand Down
2 changes: 1 addition & 1 deletion bench/src/swe-arena/gepa-seat.mts
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ export function innerSmokeJudge(): JudgeConfig<SmokeVerdict, GepaSeatScenario> {
// ---------------------------------------------------------------------------

export const GEPA_PYTHON_INSTALL_HINT =
'install `agent-eval-rpc==0.144.10`, then install ' +
'install `agent-eval-rpc==0.144.12`, then install ' +
'`gepa[full] @ git+https://github.com/gepa-ai/gepa.git@f919db0a622e2e9f9204779b81fe00cc1b2d808f`'

export type GepaMethodFactory = (
Expand Down
Loading