Commit cc8e2e7
fix(run-engine): sweeper LREMs the worker queue list when acking marked runs
processMarkedRun previously called acknowledgeMessage with
removeFromWorkerQueue: false. The Lua script always DELs the message body
but only LREMs the worker queue list when the flag is set, leaving a
stale messageKey on the list whenever the swept run had been pushed to
the worker queue list (fast-path enqueue or processQueueForWorkerQueue
promotion) but never BLPOP'd.
The next BLPOP returns the tombstone, GET on the messageKey returns nil,
and the dequeue path logs "Failed to dequeue message from worker queue".
Switch to removeFromWorkerQueue: true. Cost is one extra LREM per swept
run (O(N) on the list); production telemetry shows sweeper acks at
<0.33/sec hard upper bound on worker queue lists averaging 200-500
entries — under 0.05% Redis CPU even at peak. The repro test added in
the previous commit flips to passing with this change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent dc42cdf commit cc8e2e7
1 file changed
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2847 | 2847 | | |
2848 | 2848 | | |
2849 | 2849 | | |
2850 | | - | |
| 2850 | + | |
2851 | 2851 | | |
2852 | 2852 | | |
2853 | 2853 | | |
| |||
0 commit comments