fix(agents-server): stop orphaned cron wakes#4559
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4559 +/- ##
===========================================
- Coverage 74.35% 55.55% -18.80%
===========================================
Files 52 318 +266
Lines 6897 36894 +29997
Branches 2190 10569 +8379
===========================================
+ Hits 5128 20497 +15369
- Misses 1753 16364 +14611
- Partials 16 33 +17
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
balegas
approved these changes
Jun 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes orphaned cron wakes in Agents Server after a manifest-backed schedule is deleted. Users should no longer see agents repeatedly woken by a cron schedule that
list_schedulescorrectly reports as absent.Root Cause
Cron schedules are represented by two linked pieces of state:
list_schedulesFor
/horton/RbYyjlX3ey, the manifest schedule had been deleted, solist_schedulesreturned[]. However, cron wake delivery kept going because stale wake/scheduler side effects could outlive the manifest deletion:WakeRegistryremoved cached registrations on Electric shape deletes by deriving the DB id from the shape message key. Shape keys are protocol-level identifiers and are not guaranteed to be the table primary key, so the DB row could be gone while the in-memory registration remained.wake_registrationsrows still subscribed to that cron stream.Approach
This PR makes the cron side effects self-healing in both places.
Wake registry delete handling
WakeRegistry.applyShapeMessagenow treatsold_value.idas authoritative for delete messages. The shape usesreplica: full, so deletes should carry the deleted row; if the id is missing, the registry resets its cache to fail closed instead of keeping a potentially stale wake registration alive.Cron chain rescheduling
Before inserting the next cron tick, the scheduler checks whether any wake registration still points at that cron stream:
If there are no subscribers, it completes the current task and stops the chain. A future schedule registration will seed a fresh tick through
getOrCreateCronStream/ rehydration.Key Invariants
Non-goals
Trade-offs
An alternative would be to always reschedule cron ticks forever for every cron stream ever created. That is simpler, but it allows orphaned cron streams to keep waking stale in-memory subscribers and creates unnecessary scheduler churn. Checking for subscribers before rescheduling keeps cron streams lazy and aligns their lifecycle with active wake registrations.
Another option would be to rely only on wake-registry cache invalidation. The scheduler guard is still useful as a second line of defense and prevents orphaned cron task chains even if a process restarts or a wake cache bug reappears.
Verification
Commands run:
Notes:
pnpm installcompleted dependency linking but reported warnings, includingexamples/.shared preparefailing due SST.wake-registry.test.tslocally reached the integration section and timed out with a webhook deliveryfetch failedwarning, so the new unit path was verified with-t.check-changesetpasses.Files changed
packages/agents-server/src/wake-registry.ts— remove cached registrations usingold_value.idfrom shape delete messages; reset the cache if a delete id is unavailable.packages/agents-server/src/scheduler.ts— stop rescheduling cron ticks when no registrations subscribe to the cron stream; clarify cron payload stream-path extraction and subscriber lookup.packages/agents-server/test/scheduler.test.ts— cover subscriber-present and no-subscriber cron reschedule paths.packages/agents-server/test/wake-registry.test.ts— cover non-id shape keys with delete ids coming fromold_value.id..changeset/fix-orphaned-cron-wakes.md— patch changeset for@electric-ax/agents-server.