feat(desktop): move persisted secrets from JSON into keyring via gen-ref protocol

[wpfleger96]

What

Provider API keys and other secret-shaped values set as agent env vars were persisted plaintext in global-agent-config.json, managed-agents.json, agent-definitions.json, and the custom_harnesses/*.json files. This moves every persisted secret out of those JSON files and into the OS keyring, leaving each JSON file holding only an opaque generation reference. The nostr identity keys already lived in the keyring; agent env vars were the missed surface, and custom-harness env maps were a second plaintext surface on the same footing.

How

Gen-ref protocol

Each secret field (env_vars, auth_tag, provider_config, and harness env) gets a companion *_ref: Option<String> holding a generation UUID. On load, when the inline field is empty and a ref is present, the value is hydrated from the keyring. On save, a non-empty inline value is written to the keyring under a versioned coordinate, the inline JSON field is cleared, and the ref is updated to the new generation. Superseded generations are never deleted eagerly — they are retired by a two-cycle GC only once no committed JSON references them.

Coordinates are keyed by namespace: global:env:<gen>, agent:<pubkey>:env|auth_tag|provider_config:<gen>, definition:<slug>:env:<gen>, and harness:<id>:env:<gen>. All coordinates are keys inside the single JSON blob the keyring already uses for identity (service buzz-desktop, or buzz-desktop-dev in debug builds; username secrets) — one blob entry, one OS prompt per process, shared with the identity store.

Fail-closed hierarchy

Unavailability is tracked at three tiers, and none silently degrades to an empty env:

1. Instance tier — ManagedAgentRecord.secrets_unavailable, set when an instance ref fails to hydrate.
2. Definition tier — AgentDefinition.secrets_unavailable, set when a definition env_vars_ref fails to hydrate.
3. Global tier — load_global_agent_config returns Err when a global env_vars_ref is unresolvable.

All three gate spawn, readiness (local_setup: false), and deploy through the single require_effective_secrets_available gate. The definition-tier spawn refusal is one shared predicate (unavailable_definition_id) rather than a per-call-site copy.

Commit-point discipline

Old-generation deletion is removed from every save path. The invariant is: write keyring → cancel GC candidacy → clear the inline field → set the ref → commit the JSON atomically. A failed JSON commit still finds the on-disk ref's generation live, because nothing was deleted before the commit landed.

Batched write with generation reuse

A metadata-only save no longer churns the keyring. write_secrets_batched persists env / auth_tag / provider_config in a single blob mutation and, for any field whose bytes byte-equal what the live ref already stores, keeps the existing generation and writes nothing — no new UUID, no mutation, no GC candidacy. Changed fields are staged and committed together in one atomic store_batch_verified; if that one write fails, all staged fields fall back to KeptInline together, so there is no torn partial state. The boot-migration path keeps its explicit cancel_gc_candidacy; the command path drops it as provably redundant under the transaction lock (a freshly minted or a live-reused generation can never carry a candidate marker at save time).

GC exclusion and validation

The full extraction + global save + GC runs under managed_agents_store_lock; the GC's final JSON read and remove_batch are indivisible against a concurrent save. collect_live_refs validates every coordinate before the sweep — malformed, duplicate, or inline+ref-conflict states abort the sweep as a no-op rather than reclaiming a live generation.

Cross-process transaction lock

Each save_managed_agents / save_agent_definitions acquires a cross-process transaction lock (acquire_secret_txn_lock) before reading the other half, and holds it across the gen-writes and the atomic JSON commit. The identity-persist path takes the same lock, so an identity save cannot interleave with an agent save against the shared blob. The lock is keyed by the symlink-resolved store directory inode (store_txn_lock_dir), so two Desktop processes sharing one JSON (e.g. just staging + just production on shared worktrees) cannot interleave a read of one half with the other's write. There is no /tmp lock file: the lock is taken directly on the canonical store-directory inode in owner-only app-data (Unix flock on the directory fd; Windows a named kernel mutex via CreateMutexW derived from the resolved path), so a tmp-cleaner cannot unlink the lock target from under a live transaction. The blob lockfile itself is additionally hardened: after the flock is granted, locked_inode_is_live re-checks that the held fd still refers to the live pathname's inode, catching the classic unlink/recreate split where two processes would otherwise each "hold" the lock over different inodes.

Boot migration and scrub

At boot, before the spawn registry warms, migrate_inline_secrets_to_keyring and migrate_harness_secrets_to_keyring lift any remaining inline secrets into the keyring, rewrite the JSON stripped at 0o600, and — once every projected generation reads back cleanly — scrub the plaintext-bearing backup/temp artifacts (*.json.bak, temp files) the save path can leave behind. Extraction is idempotent across launches: an already-projected file (empty inline + live ref) is re-read and nothing is rewritten. Phase-2 cleanup (legacy managed-agents.json scrub/delete) is gated on a verified reload showing zero secrets_unavailable flags.

Dev-service migration

migrate_agent_secrets_to_dev_service copies projection keys from the release keyring service to a scoped dev service for standalone worktree launches. The completion marker (_dev_secrets_migration_v2) is written only on a fully clean run; any conflict withholds it so the migration retries next boot. Per-coordinate conflict:<coord> markers are cleared only on proven convergence or proven non-liveness of the coordinate — a source that merely dropped the coordinate does not clear it.

Known decisions and limitations

• Harness generations are not GC'd. Superseded harness:<id>:env:<gen> entries are outside is_projection_key, so the two-cycle sweep never reclaims them; they accrete in the keyring blob (encrypted at rest — not the plaintext surface this PR closes) on the rare harness-env edit. Follow-up: make custom-harness saves a second consumer of write_secrets_batched, whose gen-reuse makes most accretion never happen.
• Boot migration relies on single-instance serialization. The boot extraction path is not itself cross-process-locked; it is safe because Desktop runs single-instance (tauri_plugin_single_instance, lib.rs:116) — a duplicate launch is focused into the existing window rather than running a second boot migration.
• Caller-snapshot race (pre-existing, deferred). The transaction lock makes each save_* atomic cross-process, but it cannot make a caller's pre-lock snapshot transactional. A command that reads its half under the process-local managed_agents_store_lock, mutates, then calls save_* still races a second OS process reading the same JSON before the first commits (last-writer-wins, as wholesale rewrite always was). The gen-ref protocol's new exposure is that a lost write can orphan a just-committed generation. Closing it means promoting ~60 save_* call sites to caller-level load → mutate → commit transactions, several spanning .await — beyond this security fix's seam. The seam is marked at acquire_secret_txn_lock in storage.rs.

Test coverage

• Two-launch ref preservation and GC-cycle survival across the projection seams.
• Raw re-read + ? propagation on the save paths (no silent inline re-materialization).
• Projection / hydrate / 0o600 / boot-migration coverage including two-launch generation stability, and 64 harness tests (16 migration-harness-secrets + 48 custom-harness).
• Batched-mutation gen-reuse and single-mutation atomicity.
• Blob-lock inode recheck, identity txn-lock entry point, and an identity-vs-agent-save interleave test.
• Instance-vs-definition save_* interleave test (handshake-deterministic flock EWOULDBLOCK probe), asserting neither half is lost or re-inlined.
• Definition-tier spawn/readiness driven through the production predicate (mutation-checked), with a positive control isolating the definition gate.
• The two global-config Err-path boundaries (refuse_save_on_unavailable_current, resolve_snapshot_global), each with an Ok pass-through control and an Err-arm regression. The get_agent_models, card-mint, and profile-signing boundaries route through require_effective_secrets_available, whose all-tier refusal is already saturated by the effective-config gate tests — no per-command duplicates.