Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .agents/skills/add-coding-agent-capture/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
name: add-coding-agent-capture
description: Add or review live coding-agent event capture through blocking command hooks or an in-process plugin such as a JavaScript adapter. Use when forwarding native lifecycle, model, tool, permission, or subagent events into the Braintrust daemon, or when fixing ordering, failure, or flush behavior in an adapter.
---

# Add coding-agent event capture

Confirm the canonical source identity and production translator exist. If not,
use `add-coding-agent-translator` first or include it earlier in the same plan.

## Choose the capture path

- Prefer blocking command hooks when the agent offers ordered lifecycle hooks.
- Prefer the shared long-lived daemon client for an in-process plugin API.
- Combine hooks with transcript paths only when hooks omit data required by the
translator; keep trace construction out of the adapter.

## Implement a thin adapter

- Forward raw native payloads with source, session, event, capture timestamp,
agent version, and adapter version. Add cwd or worktree only when supplied
separately by the agent API.
- Preserve per-session ordering. Await daemon acknowledgement for blocking
hooks; serialize requests on a long-lived connection for in-process plugins.
- Flush on terminal lifecycle events and close persistent clients cleanly.
- Fail open with bounded diagnostics when the daemon, host CLI, or network is
unavailable. Never break an agent turn because tracing failed.
- Forward only non-secret route selections. Never handle Braintrust credentials
or send spans directly to Braintrust.
- Capture every event required by the translator and quote commands safely on
every supported platform.

## Verify completion

Test hook or plugin registration, payload mapping, ordering, reconnection,
failure behavior, terminal flush, configuration precedence, disablement, and
the exact source identity accepted by the production translator.
4 changes: 4 additions & 0 deletions .agents/skills/add-coding-agent-capture/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "Add Agent Capture"
short_description: "Forward agent events through hooks or plugins"
default_prompt: "Use $add-coding-agent-capture to add daemon event capture for a coding agent."
35 changes: 35 additions & 0 deletions .agents/skills/add-coding-agent-import/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
name: add-coding-agent-import
description: Add or review historical transcript import and live attach support for a coding agent, including session lookup, native parsing, synthetic lifecycle envelopes, shared translator use, incremental tailing, destination overrides, and import tests. Use when implementing or fixing import, attach, resume-session lookup, or transcript-following behavior.
---

# Add coding-agent transcript import

Require a production translator and a viable native transcript containing the
facts to import. If either is missing, use `add-coding-agent-translator` or report
the fidelity blocker before implementing import.

## Implement import and attach

- Expose the agent, aliases, and display name through the public import command.
- Locate exactly one transcript from a validated session ID across documented
active and archived locations. Avoid unsafe symlink traversal and fail clearly
on zero or multiple matches.
- Parse records with contextual file and line errors, schema-version handling,
authoritative native session identity, and native timestamps.
- Convert transcript records into synthetic envelopes for the same source and
translator used by live capture. Never create a second trace builder or invent
facts that exist only in hooks.
- Keep historical import and live attach on the same parser, processor,
translator, and sink state. Vary only waiting and finalization.
- Tail incrementally without duplicates, tolerate partial final records, keep an
active turn open, and finalize it on interruption.
- Apply typed destination or exported parent overrides before transcript lookup
and fail fast when the host has not supplied resolved session configuration.

## Verify completion

Test active and archived discovery, invalid and ambiguous IDs, malformed and
empty files, one and many turns, incremental growth, no-growth polls, partial
writes, finalization, destination and parent attachment, resume behavior, and
deterministic replay through the production translator.
4 changes: 4 additions & 0 deletions .agents/skills/add-coding-agent-import/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "Add Agent Import"
short_description: "Import and follow native agent transcripts"
default_prompt: "Use $add-coding-agent-import to add transcript import and attach support for a coding agent."
48 changes: 48 additions & 0 deletions .agents/skills/add-coding-agent-integration/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
name: add-coding-agent-integration
description: Orchestrate a complete Braintrust coding-agent integration by delegating feasibility, daemon translation, event capture, setup, managed run, transcript import, verification, and shipping to the specialized repo-local skills. Use for end-to-end support for a new agent or a completeness audit of an existing integration; use the component skills directly for focused work.
---

# Orchestrate a coding-agent integration

Coordinate the integration; do not implement component details from this skill.
Inspect current source, keep a requirement-to-evidence inventory, and invoke the
specialized skill for every applicable workstream.

## Delegate the work

1. Use `audit-agent-tracing-support` to establish whether hooks, an in-process
plugin API, transcripts, or a combination can provide sufficient data.
2. Use `add-coding-agent-translator` to establish the canonical source identity
and production daemon translator first.
3. Use `add-coding-agent-capture` to forward live native events through command
hooks or an in-process plugin.
4. Once the translator exists, delegate independently:
- persistent installation and configuration to `add-coding-agent-setup`;
- invocation-local launching to `add-coding-agent-run`;
- historical import and live transcript following to
`add-coding-agent-import`, when a viable transcript exists.
5. Use `test-coding-agent-integration` to verify every implemented surface and
the combined end-to-end path.
6. Use `ship-coding-agent-integration` only after verification evidence exists.

Add every applicable skill to the working plan and follow that skill when its
workstream begins. Do not copy its instructions into the plan or preload skills
for workstreams that are out of scope.

## Maintain the shared contract

Keep one stable source identity across the translator, capture adapter, setup,
run, import, status, tests, and documentation. Keep plugins and hooks thin,
fail-open, credential-free forwarders; the daemon owns correlation, trace
construction, recovery, routing, and delivery.

Treat a missing capability as `blocked` or explicitly `not applicable` with
evidence, never as silently complete. Do not call the integration complete
until each applicable component skill has produced implementation and test
evidence and the shipping skill has confirmed a releasable distribution path.

## Hand off

Report a compact matrix with one row per delegated skill: status, source
evidence, test evidence, blockers or fidelity loss, and remaining release work.
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "Add Coding Agent Integration"
short_description: "Orchestrate all coding-agent integration layers"
default_prompt: "Use $add-coding-agent-integration to coordinate a complete coding-agent integration."
32 changes: 32 additions & 0 deletions .agents/skills/add-coding-agent-run/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
name: add-coding-agent-run
description: Add or review invocation-local managed-run support for a coding agent, including executable dispatch, temporary hook or plugin injection, route isolation, duplicate-capture suppression, process behavior, trust safety, and run-command tests. Use when implementing or fixing the command that launches an agent with tracing for one invocation.
---

# Add a managed coding-agent run

Require a production translator and a capture mechanism that can be injected for
one process invocation. Use the translator or capture skill first when missing.

## Implement managed run

- Add the agent to public run parsing, aliases, executable dispatch, and the
canonical daemon source mapping.
- Require a resolved destination before launch and scope route overrides to the
child process tree without rewriting persistent setup.
- Inject the full capture event set using agent-specific arguments, settings,
environment, or plugin configuration.
- Suppress inherited Braintrust capture only for the managed process tree while
allowing the injected adapter to run, preventing duplicate traces.
- Preserve normal hook or plugin trust review. Never enable a global trust
bypass or authorize unrelated hooks.
- Inherit stdio, forward agent arguments verbatim, return its exit status, and
terminate and reap it on interruption.
- Quote generated commands safely across supported platforms and unusual paths.

## Verify completion

Test parsing, aliases, generated configuration, every injected event, route
isolation, duplicate suppression, paths with spaces and non-ASCII text, missing
executables, failure status, interruption, and the absence of transcript polling
when live hooks or plugins are the selected capture path.
4 changes: 4 additions & 0 deletions .agents/skills/add-coding-agent-run/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "Add Agent Run"
short_description: "Launch an agent with invocation-local tracing"
default_prompt: "Use $add-coding-agent-run to add managed tracing runs for a coding agent."
30 changes: 30 additions & 0 deletions .agents/skills/add-coding-agent-setup/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
name: add-coding-agent-setup
description: Add or review persistent setup for a coding-agent tracing integration, including public CLI exposure, plugin or hook installation, non-secret route configuration, idempotent updates, disablement, and isolated setup tests. Use when implementing or fixing the setup, install, enable, disable, or uninstall experience for an agent.
---

# Add persistent coding-agent setup

Require a registered translator and an installable capture adapter. If either is
missing, use `add-coding-agent-translator` or `add-coding-agent-capture` before
considering setup complete.

## Implement setup

- Expose the agent through the public setup command and help surfaces.
- Resolve the selected profile, organization, and typed trace destination in the
profile-aware host before writing configuration.
- Install or enable the adapter through the agent's supported mechanism.
- Persist only non-secret route settings. Keep credentials, resolved backend
URLs, and credential leases in the host.
- Make repeated setup idempotent and atomic. Preserve unrelated plugins, hooks,
settings, and user configuration; avoid duplicate Braintrust entries.
- Keep generated hook definitions stable when the agent uses definition-based
trust, and preserve the agent's normal review boundary.
- Provide a reversible disable or uninstall path.

## Verify completion

Test setup in an isolated home and configuration tree, including first install,
repeat setup, route changes, existing unrelated configuration, disablement,
missing dependencies, and a real agent launch using the installed adapter.
4 changes: 4 additions & 0 deletions .agents/skills/add-coding-agent-setup/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "Add Agent Setup"
short_description: "Install and configure persistent agent tracing"
default_prompt: "Use $add-coding-agent-setup to add persistent tracing setup for a coding agent."
35 changes: 35 additions & 0 deletions .agents/skills/add-coding-agent-translator/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
name: add-coding-agent-translator
description: Implement or review a coding agent's daemon translator, including source registration, native event correlation, trace shape, deterministic recovery, routing metadata, and translator tests. Use when adding a new agent translator or changing how an existing agent's hooks or transcripts become Braintrust spans.
---

# Add a coding-agent translator

Start from official agent documentation, captured native payloads, and native
transcripts. Establish one lowercase source identity for every integration
surface and record any version-dependent schema behavior.

## Implement the translator

- Add one stateful translator per daemon session and register its factory in the
production registry and advertised capabilities.
- Build the supported hierarchy of session, turn, LLM, tool, permission,
compaction, and recursive subagent spans. Preserve correct parents and the
effective trace root.
- Populate native timing, inputs, outputs, model, token and cache metrics,
errors, outcomes, tags, agent and adapter versions, session identity, cwd,
and useful metadata without inventing unavailable facts.
- Honor resolved destinations, exported parent/root attachment, additional
metadata, and flush behavior. Keep internal routing data out of user metadata.
- Use shared git enrichment and deterministic span identities so repeated,
duplicate, late, and journal-replayed events converge instead of duplicating.
- Bound transcript observations to what was available when an event was
captured. Handle unknown or drifted events without panicking.
- Flush late records and close defensibly any work left open by missing events.

## Verify completion

Add fixture-driven tests for happy paths, failures, duplicate or missing event
pairs, multiple turns, resume, supported subagents and permissions, token
accounting, attachment, cwd changes, version variants, flush, and replay.
Confirm the production registry selects this translator for the agreed source.
4 changes: 4 additions & 0 deletions .agents/skills/add-coding-agent-translator/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "Add Agent Translator"
short_description: "Build an agent event-to-trace translator"
default_prompt: "Use $add-coding-agent-translator to implement daemon translation for a coding agent."
32 changes: 32 additions & 0 deletions .agents/skills/ship-coding-agent-integration/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
name: ship-coding-agent-integration
description: Package and release a coding-agent tracing integration through its marketplace, package registry, or distribution repository, including manifests, builds, validation, versioning, CI, documentation, publishing automation, and installed-artifact smoke tests. Use when preparing an integration for release or auditing whether it has a complete distribution path.
---

# Ship a coding-agent integration

Require verification evidence from `test-coding-agent-integration`. Do not
publish, tag, or mutate a live distribution unless the user explicitly requests
that release action.

## Prepare the distribution

- Integrate the agent with the repository's build, validation, and publishing
system using its supported marketplace or package format.
- Validate manifests, entrypoints, generated artifacts, package allowlists,
platform binaries, dependency compatibility, and installability.
- Ensure distributed hooks and plugins remain thin daemon forwarders without
direct Braintrust credential or ingest logic.
- Add versioning, tags, release notes, provenance, and release automation suited
to the distribution channel, including a safe dry-run path.
- Document prerequisites, setup, run, import or attach, disablement,
troubleshooting, fidelity limitations, privacy behavior, and release steps.
- Configure only the narrowly required publishing and optional smoke-test
secrets.

## Prove release readiness

Install the built artifact through the user-facing distribution path. Verify
setup, one live traced session, managed run, import or attach when supported,
status or permalink output, and disablement without damaging unrelated agent
configuration. Add a post-deploy smoke test where the channel permits it.
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "Ship Agent Integration"
short_description: "Package and release a coding-agent integration"
default_prompt: "Use $ship-coding-agent-integration to prepare and ship a coding-agent integration."
33 changes: 33 additions & 0 deletions .agents/skills/test-coding-agent-integration/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
name: test-coding-agent-integration
description: Test or audit a coding-agent tracing integration across translator fixtures, capture adapters, setup, managed run, transcript import, daemon recovery, real-agent execution, package installation, and CI. Use when adding verification, reviewing integration completeness, diagnosing coverage gaps, or proving that emitted events become correct Braintrust traces.
---

# Test a coding-agent integration

Identify the changed integration surfaces and use their specialized skills when
their intended behavior is unclear. Derive current commands and test seams from
source rather than copying another agent's test layout.

## Build layered evidence

- Add translator fixtures covering trace hierarchy, content, timing, metrics,
errors, duplicates, missing events, flush, and deterministic replay.
- Test adapter mapping, ordering, failure-open behavior, reconnection, and flush.
- Test setup, managed run, and import or attach independently when supported.
- Exercise the full envelope-to-journal-to-translator-to-sink pipeline, including
routing, attachment, credential redaction, restart recovery, and status errors.
- Run a real packaged agent against deterministic mock inference and mock ingest.
Assert session, turn, LLM, tool, failure, version, provenance, and terminal
delivery semantics rather than only process success.
- Install the adapter exactly as users will, including package and peer
resolution. Add minimum and current compatibility coverage when applicable.
- Cover every claimed operating system and architecture, or document the
deliberate product limitation.

## Validate and report

Run the repositories' canonical formatting, linting, unit, integration, locked
dependency, package, generated-artifact, and diff checks. Explicitly execute
real-agent tests that the default runner skips and confirm they actually ran.
Report exact commands, versions, results, skipped tests, and remaining gaps.
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "Test Agent Integration"
short_description: "Verify agent tracing from events to delivered spans"
default_prompt: "Use $test-coding-agent-integration to verify a coding-agent integration end to end."
8 changes: 8 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,14 @@ Everything an agent installs lives under `src/plugins/<agent>/content/`.
Plugin hooks are thin fail-open shell and Windows command launchers that invoke
`bt daemon hook`; they do not contain or compile a second tracing runtime.

## Adding a coding-agent integration

For complete integrations, use `$add-coding-agent-integration` to delegate the
tracing audit, translator, capture, setup, managed run, import/attach, testing,
and shipping work to the corresponding repo-local skills. Use a component skill
directly for focused work. Every integration uses one source identity and daemon
translator; hooks and plugins remain thin, fail-open, credential-free forwarders.

## Local development

```bash
Expand Down
Loading