Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Changelog

## 0.134.2

- Make the supervised token charge additive: `input - cacheRead + output`, which equals `freshInput + cacheWrite + output` under a complete cache split. A spend that folded a classified turn together with an unclassified one previously fell back to the rolled-up prompt total for the whole aggregate, so one unreported turn re-charged every cached prefix beside it.
- Credit no cache read whose reported classes do not fit inside the prompt total they partition, at the record AND in the fold: `addTokenUsage` no longer accumulates the classes of a turn that overflowed its own `input`, because the accumulator's larger total would otherwise absorb the overflow and charge the aggregate less than its records. The charge is never below the output tokens, and a zero prompt total charges no prompt tokens.
- The token channel trusts a reported cache read the same way it trusts a reported `input`. It is an accounting unit, not a trust boundary against a provider that misreports its own usage.
- Accept a `Spend` whose cache classes cover only part of `input` when it carries `cacheBreakdownKnown: false`. Requiring an exact partition there rejected the shape aggregation produces, and a resumed pool restored from such a record failed at construction. Classes that EXCEED `input` are still refused, and a spend claiming a complete split must still partition `input` exactly.
- `equalKOnCost` now rates a rolled-up arm at what the pool charged it, including trees where one node reported no cache split.

## 0.134.1

- Preserve a cli-bridge root's known profile materialization when Runtime exhausts the token budget after the bridge emits its terminal receipt.
Expand Down
2 changes: 1 addition & 1 deletion docs/api/primitive-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

# Primitive catalog — the never-stale anti-reinvention inventory

> **GENERATED** from `@tangle-network/agent-runtime@0.134.1` and `@tangle-network/agent-eval@0.145.2` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`.
> **GENERATED** from `@tangle-network/agent-runtime@0.134.2` and `@tangle-network/agent-eval@0.145.2` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`.

## 1. agent-runtime — own public surface

Expand Down
6 changes: 4 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -378,10 +378,12 @@ The `Scope` it runs inside is the budget-conserving reactive control surface (`t
└─ budget → {tokensLeft,tokensKnown,cacheBreakdownKnown,usdLeft,usdKnown,iterationsLeft,deadlineMs,reservedTokens}
```

The token channel charges each token ONCE, when it first enters the context: `freshInput + cacheWrite + output`.
The token channel charges each token ONCE, when it first enters the context: `input - cacheRead + output`, which is `freshInput + cacheWrite + output` under a complete split.
A cache read re-presents content that was already charged when it was written, so charging it again charges the same tokens twice.
No price weight enters this channel; money is budgeted separately on `maxUsd`.
When a provider reports no usable cache split, the pool charges the rolled-up `input + output` and sets `cacheBreakdownKnown: false`, which marks `tokensLeft` an upper bound on newly-presented work rather than a measurement.
The charge is additive, so a rolled-up report agrees with the pool.
A cache class that does not fit inside the prompt total it partitions credits nothing.
Prompt tokens the provider never classified are charged in full, and `cacheBreakdownKnown: false` then marks `tokensLeft` an upper bound on newly-presented work rather than a measurement.

Two facts make this the whole game:
- `spawn` **reserves** from one root total and refunds the unspent remainder on settle.
Expand Down
2 changes: 1 addition & 1 deletion docs/canonical-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Generated signatures and the complete export list live in docs/api/.
Run pnpm docs:freshness after editing this file. -->

> **Version 0.134.1.**
> **Version 0.134.2.**
> [`docs/api/primitive-catalog.md`](./api/primitive-catalog.md) lists every export and import path.
> `agent-eval` must satisfy `>=0.145.2 <0.146.0`.
> `sandbox` must satisfy `>=0.21.1 <0.22.0`.
Expand Down
2 changes: 1 addition & 1 deletion docs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ The shape grows by LLM decision through the **coordination toolbox** over a live
|---|---|---|
| **Budget** | A ceiling envelope on a spawn/root: `{maxIterations, maxTokens, maxUsd?, deadlineMs?}`. (Keystone substrate.) `maxTokens` counts NEWLY-PRESENTED tokens (see **Charged tokens**). `deadlineMs` is currently classify-only, does not fire an abort — known gap. | `supervise/types.ts:189` |
| **Spend** | Conserved actual cost reconciled from `UsageEvent`s: `{iterations, tokens, usd, ms}`. Tokens and usd are separate channels, never folded. | `supervise/types.ts:198` |
| **Charged tokens** | The pool's token unit: `freshInput + cacheWrite + output`, so each token is counted once, when it first enters the context. A cache read re-presents content already charged at write time. With no readable cache split the charge falls back to `input + output` and `BudgetReadout.cacheBreakdownKnown` reads false, marking the balance an upper bound. | `runtime/util.ts` |
| **Charged tokens** | The pool's token unit: `input - cacheRead + output`, so each token is counted once, when it first enters the context. Under a complete split it equals `freshInput + cacheWrite + output`. The subtraction form is additive, so an aggregate charges what its records charged; a cache class that does not fit inside the prompt total it partitions credits nothing. Unclassified prompt tokens are charged in full and `BudgetReadout.cacheBreakdownKnown` reads false, marking the balance an upper bound. | `runtime/util.ts` |
| **BudgetPool / ReservationTicket** | The **conserved reservation pool**: each spawn *reserves* against the root then settles to actual `Spend`. This is what makes **equal-compute hold by construction** (the anti-confound invariant for the gate). | `supervise/budget.ts:48,29` |
| **UsageEvent** | The normalized usage increment every executor emits, so the pool meters all runtimes identically. | `supervise/types.ts:120` |
| `runAgentRounds`'s budget | Only `maxIterations` (count) + `maxConcurrency` (in-flight cap) + per-`Iteration` cost aggregation. The rigorous reservation pool is the keystone's, not `runAgentRounds`'s. | `run-loop.ts:88` |
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@tangle-network/agent-runtime",
"version": "0.134.1",
"version": "0.134.2",
"description": "Shared task-lifecycle skeleton for agents: a recursive loop kernel for chat turns, one-shot tasks, and multi-attempt loops, with trace capture and eval-gated self-improvement. Domain behavior lives in adapters; scoring and ship-gates in @tangle-network/agent-eval.",
"homepage": "https://github.com/tangle-network/agent-runtime#readme",
"repository": {
Expand Down
53 changes: 34 additions & 19 deletions src/runtime/supervise/budget.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,17 @@
* channels (`LoopTokenUsage` has no `usd`); iterations are conserved alongside them.
*
* The token unit is `chargedTokens`: every token counted ONCE, when it first enters the context
* (`freshInput + cacheWrite + output`). A cached prefix is re-presented content that was already
* charged when it was written, so charging a cache read again would charge the same tokens twice —
* on a real fleet cache is 98% of the rolled-up prompt total, which made a declared cap bite about
* 56x early. No price weight enters the token channel; money is budgeted separately on `maxUsd`.
* When a provider reports no usable cache split the pool keeps charging the rolled-up
* `input + output` and marks `readout().cacheBreakdownKnown` false, so the balance reads as an
* upper bound on newly-presented work rather than a measurement.
* (`input − cacheRead + output`, which is `freshInput + cacheWrite + output` under a complete
* split). A cached prefix is re-presented content that was already charged when it was written, so
* charging a cache read again would charge the same tokens twice — on a real fleet cache is 98% of
* the rolled-up prompt total, which made a declared cap bite about 56x early. No price weight
* enters the token channel; money is budgeted separately on `maxUsd`. The unit is additive, so a
* child's settlement charges what its turns charged and a rolled-up report agrees with the pool.
* Prompt tokens the provider never classified are charged in full, and
* `readout().cacheBreakdownKnown` then reads false, so the balance reads as an upper bound on
* newly-presented work rather than a measurement. The pool trusts a reported cache read the same way
* it trusts a reported `input`: the token channel is an accounting unit, not a trust boundary
* against a provider that misreports its own usage.
*
* Pure and deterministic: `now()` is injected, there is no I/O, and no wall-clock or
* RNG read. A `reserve`/`reconcile` ticket is single-use (fail-loud on double or
Expand Down Expand Up @@ -87,11 +91,13 @@ export type BudgetReadout = Readonly<{
*/
tokensKnown: boolean
/**
* False once the pool has charged work whose prompt-cache split it could not read. Those tokens
* were charged at the rolled-up prompt total, which counts a cached prefix again on every turn
* that reads it, so the debited amount is an upper bound on newly-presented work rather than a
* measurement. It is a separate fact from `tokensKnown`: the counts arrived, but their
* composition did not.
* False once the pool has charged a REPORTED spend whose prompt-cache split it could not read.
* The prompt tokens that carried no class were charged in full, and a cached prefix reaches the
* pool again on every turn that reads it, so the debited amount is an upper bound on
* newly-presented work rather than a measurement. It is a separate fact from `tokensKnown`: the
* counts arrived, but their composition did not. A restored uncertain reservation leaves this
* flag alone — it charges a declared ceiling, which has no composition to misread, and
* `tokensKnown` already reports that the balance is not a measurement.
*/
cacheBreakdownKnown: boolean
usdLeft: number
Expand Down Expand Up @@ -162,13 +168,22 @@ function assertValidSpend(spend: Spend, label: string): void {
}
}
const { freshInput, cacheRead, cacheWrite } = spend.tokens
if (
freshInput !== undefined &&
cacheRead !== undefined &&
cacheWrite !== undefined &&
freshInput + cacheRead + cacheWrite !== spend.tokens.input
) {
throw new Error(`${label}.tokens cache classes must sum to input`)
if (freshInput !== undefined && cacheRead !== undefined && cacheWrite !== undefined) {
const classified = freshInput + cacheRead + cacheWrite
// A spend that CLAIMS a complete split must partition `input` exactly — that invariant is what
// lets the charge credit a cache read. A spend that declares its split INCOMPLETE reports
// classes covering only part of `input`, which is exactly what `addTokenUsage` produces when it
// folds a classified turn together with an unclassified one. Demanding an exact partition there
// rejects the shape the incomplete flag exists to describe, and a resumed pool built from such
// an aggregate died at construction. Either way the classes may never EXCEED the total they
// partition; that direction would let bad telemetry credit tokens nobody presented.
if (spend.tokens.cacheBreakdownKnown !== false) {
if (classified !== spend.tokens.input) {
throw new Error(`${label}.tokens cache classes must sum to input`)
}
} else if (classified > spend.tokens.input) {
throw new Error(`${label}.tokens cache classes must not exceed input`)
}
}
for (const [field, value] of [
['usd', spend.usd],
Expand Down
67 changes: 49 additions & 18 deletions src/runtime/util.ts
Original file line number Diff line number Diff line change
Expand Up @@ -172,38 +172,69 @@ function hasClassifiedCacheBreakdown(usage: LoopTokenUsage): boolean {

/**
* The token charge for one observation: every token counted ONCE, at the moment it first enters
* the context.
* the context — `input − cacheRead + output`.
*
* `input` is a rolled-up prompt total that re-counts a cached prefix on every turn that reads it,
* so charging `input` makes a 100K prefix read 40 times cost 4.1M for content authored once. The
* three cache classes partition `input`, so `freshInput + cacheWrite` is exactly the prompt the
* provider had to present anew. No price weight is involved: this counts work, and money is a
* separate channel.
* so charging `input` makes a 100K prefix read 40 times cost 4.1M for content authored once. A
* cache read is content that was already charged when it was written, so it is the one class the
* charge subtracts. Under a complete split the result is exactly `freshInput + cacheWrite`. No
* price weight is involved: this counts work, and money is a separate channel.
*
* Without a readable split the charge stays the rolled-up `input + output`. That number is an
* UPPER BOUND on newly-presented work, not a measurement, and every caller that enforces a ceiling
* must report the difference (`BudgetPool` does it through `readout().cacheBreakdownKnown`).
* The subtraction form is what makes the charge ADDITIVE. `input` and `cacheRead` both accumulate
* through `addTokenUsage`, so the charge on an aggregate equals the sum of the charges on the
* records that built it — including an aggregate that mixes classified and unclassified turns,
* where the unclassified remainder is charged in full and only the reported cache reads are
* credited. A form that read `freshInput + cacheWrite` directly loses that: one unclassified turn
* would drop the whole aggregate to `input + output` and over-charge every classified turn with it.
*
* The one arithmetic guarantee: a set of classes that does not FIT inside the prompt total it
* partitions credits nothing, so the charge is never below `output` and a class that overflows its
* own total buys no tokens. It is not a guarantee against a provider that reports a cache read it
* never served — that provider can under-report `input` just as easily, and the token channel is an
* accounting unit, not a trust boundary. Prompt tokens the provider left unclassified are charged
* in full, so a spend with no readable split charges an upper bound on newly-presented work rather
* than a measurement, and every caller that enforces a ceiling must report the difference
* (`BudgetPool` does it through `readout().cacheBreakdownKnown`).
*/
export function chargedTokens(usage: LoopTokenUsage): number {
if (!hasCompleteCacheBreakdown(usage)) return usage.input + usage.output
// A complete split classifies every prompt token. The one complete case that carries no class
// fields is `input === 0`, where there is no prompt to charge.
const { freshInput = 0, cacheWrite = 0 } = usage
return freshInput + cacheWrite + usage.output
return usage.input - creditedCacheRead(usage) + usage.output
}

/** `cacheRead` if the reported classes fit inside `input`, else nothing. */
function creditedCacheRead(usage: LoopTokenUsage): number {
const { cacheRead } = usage
if (cacheRead === undefined) return 0
return classifiedTotal(usage, cacheRead) > usage.input ? 0 : cacheRead
}

/** Add the observed subtotal into `acc`; token and cache incompleteness are sticky. */
function classifiedTotal(usage: Partial<LoopTokenUsage>, cacheRead: number): number {
return (usage.freshInput ?? 0) + cacheRead + (usage.cacheWrite ?? 0)
}

/**
* Add the observed subtotal into `acc`; token and cache incompleteness are sticky.
*
* A delta whose classes do not FIT inside its own `input` contributes its prompt total and NO
* classes. Folding them in would hide the overflow: the accumulator's larger `input` can absorb a
* class total that overflowed the one delta that reported it, and `chargedTokens` would then credit
* at the aggregate a cache read it refuses at the record. That is the one way the charge could come
* out below the sum of the charges on the records that built it.
*/
export function addTokenUsage(acc: LoopTokenUsage, delta: Partial<LoopTokenUsage>): void {
acc.input += delta.input ?? 0
acc.output += delta.output ?? 0
if (delta.tokensKnown === false) acc.tokensKnown = false

if (delta.freshInput !== undefined) acc.freshInput = (acc.freshInput ?? 0) + delta.freshInput
if (delta.cacheRead !== undefined) acc.cacheRead = (acc.cacheRead ?? 0) + delta.cacheRead
if (delta.cacheWrite !== undefined) acc.cacheWrite = (acc.cacheWrite ?? 0) + delta.cacheWrite

const input = delta.input ?? 0
const fits = classifiedTotal(delta, delta.cacheRead ?? 0) <= input
if (fits) {
if (delta.freshInput !== undefined) acc.freshInput = (acc.freshInput ?? 0) + delta.freshInput
if (delta.cacheRead !== undefined) acc.cacheRead = (acc.cacheRead ?? 0) + delta.cacheRead
if (delta.cacheWrite !== undefined) acc.cacheWrite = (acc.cacheWrite ?? 0) + delta.cacheWrite
}

const classified =
fits &&
delta.freshInput !== undefined &&
delta.cacheRead !== undefined &&
delta.cacheWrite !== undefined &&
Expand Down
10 changes: 5 additions & 5 deletions src/testing/fixtures/agent-improvement-proposal.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"changedSurfaces": ["prompt"],
"digest": "sha256:1f060b6f2e36df287e9eb377763752983d558b17087e842675656bbac2225153",
"digest": "sha256:c03981e5cbc2395cbe1a5cf6fa530d8159b7ab4a063cb04d54926bfec9768300",
"evaluation": {
"decision": {
"contributingChecks": [
Expand Down Expand Up @@ -4870,7 +4870,7 @@
],
"metadata": {
"fixture": "agent-improvement-proposal",
"runtimeVersion": "0.134.1"
"runtimeVersion": "0.134.2"
},
"objectives": [
{
Expand Down Expand Up @@ -4981,8 +4981,8 @@
"baselineContentHash": "sha256:5c21ee53e513fc604cb09754e21c392b24a424da0ef37dbf8f1ee4a8a0b08f09",
"candidateContentHash": "sha256:60fcbb1c728194bd51d7d19cb732d1c3f1881dce7e0a6266b41c8b98cfd65693",
"kind": "agent-eval-loop",
"recordDigest": "sha256:6df4c838fd55296f7d6574171c1a3ad91ae43879869d8ba7f16b84605d379604",
"runId": "agent-runtime-0.134.1-proposal-fixture",
"recordDigest": "sha256:0ba1bee89ec30715187720834b632e6e1b886f341ca95273f8692d0c873757e4",
"runId": "agent-runtime-0.134.2-proposal-fixture",
"schema": "agent-candidate-experiment"
}
},
Expand All @@ -5009,5 +5009,5 @@
],
"kind": "agent-improvement-proposal",
"proposedAt": "2026-07-10T01:00:00.000Z",
"runId": "agent-runtime-0.134.1-proposal-fixture"
"runId": "agent-runtime-0.134.2-proposal-fixture"
}
6 changes: 3 additions & 3 deletions src/testing/fixtures/agent-profile-improvement-proposal.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"changedSurfaces": ["prompt", "skills"],
"digest": "sha256:359e5a1f5192d915f2d24bc1fb3558c017881b59320c72d44c3c3d5f70ec0daa",
"digest": "sha256:b4eb293ad75820b4ba6dfb1e0ab5c91f81e9c26104f868cad151ae54cb2b84f1",
"evaluation": {
"decision": {
"contributingChecks": [
Expand Down Expand Up @@ -1715,7 +1715,7 @@
],
"metadata": {
"fixture": "agent-profile-improvement-proposal",
"runtimeVersion": "0.134.1"
"runtimeVersion": "0.134.2"
},
"objectives": [
{
Expand Down Expand Up @@ -1826,7 +1826,7 @@
"baselineContentHash": "sha256:21c495a37c418c10bde64fbaa188beddeed31f1f051ea60a6a6582a9ee0db704",
"candidateContentHash": "sha256:103f77bc8481601eef1ad5fe6ba84a40dffabc3a44f421f8c8559121edab84e9",
"kind": "agent-eval-loop",
"recordDigest": "sha256:4d25ec3f7deecdb4f89bffaa39d4e757b26b790ed04c5c8ff9a822d46f89a25c",
"recordDigest": "sha256:3f13df4f69474d5421e983696aa5ebe4f306eb7d48c795972e40448e3240751e",
"runId": "profile-improvement-1",
"schema": "agent-profile-improvement-experiment"
}
Expand Down
Loading