chore(webapp,core,sdk): upgrade @s2-dev/streamstore to 0.25 and migrate S2 hosts - #4349
Merged
Conversation
🦋 Changeset detectedLatest commit: a3689e0 The changes in this PR will be included in the next version bump. This PR includes changesets to release 27 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughS2 Streamstore dependencies and REST endpoint defaults are updated to use the newer 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
@trigger.dev/build
trigger.dev
@trigger.dev/core
@trigger.dev/python
@trigger.dev/react-hooks
@trigger.dev/redis-worker
@trigger.dev/rsc
@trigger.dev/schema-to-json
@trigger.dev/sdk
commit: |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
ericallam
force-pushed
the
feat/s2-caught-up-reads
branch
7 times, most recently
from
July 27, 2026 21:29
fdcd835 to
f202ccc
Compare
ericallam
marked this pull request as ready for review
July 27, 2026 21:45
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
ericallam
force-pushed
the
feat/s2-caught-up-reads
branch
from
July 30, 2026 13:07
c02115a to
cab3b03
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
ericallam
changed the title
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
ericallam
changed the title
ericallam
force-pushed
the
feat/s2-caught-up-reads
branch
from
July 31, 2026 10:58
d43512f to
9186328
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Resuming a chat session stream (page reload or reconnect) held the SSE connection open for the whole long-poll window even after every buffered output record had already arrived. The client now detects when it has caught up to the latest output and closes the resumed stream right away, so reconnecting to an idle chat settles immediately. Detection reuses the stream's tail-carrying heartbeat: batch and ping frames now carry the tail, and CaughtUpTracker from @s2-dev/streamstore 0.25.0 turns "last delivered seq + 1 === tail" into a caught-up signal. When the tail is absent (older self-hosted stream backends) the client keeps its previous behavior, so nothing regresses. Also moves to the current S2 hosts that 0.25.0 defaults to.
In local development, Vite's dev-server CORS middleware reflected the request Origin on every Express response, on top of the app's own CORS handling. The two layers produced a duplicated Access-Control-Allow-Origin header, which browsers reject, breaking cross-origin API calls from a separate frontend in dev. Disabling Vite's dev CORS lets the app be the single source of CORS headers. Dev-only; production is unaffected.
caughtUp() is documented to reject when the stream ends before reaching the tail, but cancel(), non-retryable HTTP responses, and the user-abort branches closed or errored the stream without ending the tracker, so a caller awaiting caughtUp() could stay pending forever. End the tracker on every terminal path. Also removes a stray closing tag from the changeset and applies oxfmt formatting.
Two follow-up fixes to the caught-up close, from review of the streaming path: - A resumed stream that only receives a tail-bearing heartbeat (nothing new to deliver) now settles promptly. The client used to wait for the first visible record before it could observe caught-up, so a quiescent reconnect never triggered the fast close. - Caught-up is now reported only after the consumer has actually drained every record up to the tail, not when records are merely decoded. A resumed stream carrying a backlog that reaches the tail delivers all of it before the stream can close, with no dropped records. The decode path now hands records to a demand-driven reader that marks the tail boundary as the consumer pulls past it, and ends tracking at that same point.
The client-protocol reference now describes the tail carried on the heartbeat ping, the caught-up test a hand-rolled client can run against it, and the SSEStreamSubscription's own close-on-caught-up on the reconnect path.
Boots the real webapp plus Postgres, Redis, and s2-lite via testcontainers and drives the session `.out` wire protocol directly: a producer appends records straight to S2 while the client subscribes through the webapp SSE proxy using the real stream-subscription code. Covers basic delivery, multi-turn continuation, resume without duplication, trim and command-record filtering, a quiescent reconnect settling at the tail, and a backlog that reaches the tail delivering every record before the stream closes.
Adds a real-Chromium leg to the session-stream e2e: a page on a different origin than the webapp cross-origin fetches and streams the session `.out` through the proxy, exercising the browser CORS preflight and streaming read that a node-fetch client skips. Runs against the same testcontainer stack.
Extends the chat.agent e2e harness to cover the run-lifecycle scenarios that previously needed the run-engine and a supervisor: idle-timeout suspend and resume, chat.endRun() continuation, and chat.requestUpgrade(). session.in.wait() suspends a run on a waitpoint whose only task-visible effect is that runtime.waitUntil() eventually resolves with the next .in record. A TestRuntimeManager plus a SessionWaitpointBackend reproduce that in one process: the two waitpoint apiClient calls are stubbed, and the backend opens its own tail on the session channel and resolves the wait with the next record, so the same run() invocation continues in place (matching local-dev warm resume and the task-observable semantics of a deployed CRIU restore). endRun/requestUpgrade exit the run; a small session orchestrator then spawns the next run as a continuation, mirroring the server re-triggering on the next append. It deliberately does not model server-side waitpoint bookkeeping or process checkpoint/restore, none of which are visible to task code on the resume path.
…ation legs Builds on the waitpoint backend to add the run-lifecycle scenarios it unlocks: a HITL tool approval that crosses a suspend/resume boundary (the answer arrives after the run suspends on the idle waitpoint), a run that suspends and resumes across multiple turns, a requestUpgrade that defers the message so the continuation run processes it, and a three-hop endRun chain that restores history across each continuation.
Rounds out the run-lifecycle coverage: onChatSuspend/onChatResume fire around a real suspend/resume, an OutOfMemoryError on the first attempt fails the run the way a machine swap needs and the attempt-2 retry recovers the in-flight message, and a run with no next message times out on the waitpoint and exits. Adds waitpoint-timeout support to the in-process backend (honors the wire timeout, resolves ok:false) so the idle-timeout path is exercised.
A preloaded chat.agent run (started via transport.preload) that retried after an out-of-memory error dropped the message it was processing at crash time. Boot recovery restores the in-flight message into the dispatch queue and advances the .in cursor past it, but the preload branch fired onPreload and waited for a first message without draining that queue, so the recovered message was stranded behind the advanced cursor and the run sat waiting. The preload branch now dispatches a recovered message as the first turn before waiting. Includes a full-stack e2e that reproduces the drop (preloaded run, OOM on attempt 1, in-flight message on .in) and passes with the fix.
Cancelling the returned stream aborted the internal connection, but the retry path only bailed on the caller-provided abort signal, so a consumer cancel was treated as a transient drop and kept reconnecting (up to maxRetries), issuing SSE requests after the consumer was gone. Track consumer cancellation separately, wake any pending backoff, and short-circuit the retry. Adds a regression test. Also from review: drop the unused s2 networkEndpoint field, order the webapp test-server env so the worker-disable vars win over extraEnv as documented, abort the cross-origin browser read on a timer instead of a between-reads deadline, and correct the client-protocol docs (the caught-up check counts the highest raw seq_num including skipped command records; the chat transport closes on caught-up, SSEStreamSubscription only exposes the signal).
The e2e-webapp job timed out at 20 min and E9 (server-peek fast-close) failed. Three causes: - The e2e vitest config ran files in parallel, so several files each booted a full Docker stack (Postgres, Redis, s2-lite, MinIO) plus a webapp process at once and thrashed the runner. Run e2e files serially, matching the repo pnpm test --no-file-parallelism. - The wire test seeded a Session with no triggerConfig.basePayload, so .in/append threw a server ZodError. Seed an empty basePayload. - E9 raw peek fetch had no client timeout, so a peek that did not settle under load hung ~66s until the socket dropped. Bound the fetch with an abort deadline and retry the peek until it settles. Also raises the job timeout to 30 min for the larger suite.
The in-process chat.agent e2e harness runs every leg against the single globalThis API registry. When a leg closed while its run was still parked on a session waitpoint, that run stayed alive into the next leg and its later teardown unregistered the globals out from under the active run. In CI this surfaced as Noop run-metadata and "can only be used from inside task.run()" errors across most legs. close() now aborts the run and awaits full teardown instead of racing a 10s timeout; the abort disables the session-waitpoint backend so any in-flight or later wait() returns at once; and the backend keeps its pending entry until the wait resolves so disable() can abort it. Runs no longer overlap, so the shared globals stay consistent.
The e2e .out collector subscribed once and stopped at the first graceful SSE close. The proxy serves a bounded wait window and then closes; real clients reconnect with Last-Event-ID to keep draining. A pre-populated stream fits one window, so the wire legs passed, but a streaming agent turn spans several windows and a slower runner reliably lands the content past the first close, leaving the collector with empty output. collectSessionOut now re-subscribes from the last seq it saw (deduping) until its predicate holds or the deadline passes, matching how the session manager consumes the channel.
…pped Self-hosted deployments that run S2 with REALTIME_STREAMS_S2_SKIP_ACCESS_TOKENS and no configured access token got an empty token back from the session channel initialize call. The SDK's session-stream writer treats an empty access token as missing S2 credentials and never opens the writer, so a chat.agent turn's assistant output was silently dropped: .out stayed empty with no error surfaced. The initialize response now hands back a non-empty placeholder token when access tokens are skipped (S2 ignores it in that mode, but the client requires a token to be present). The session-agent e2e runs the skip-access-tokens path with no token so the regression stays covered.
The server-to-server AgentChat client's reconnect() now sends X-Peek-Settled, so resuming an .out that is already at a turn-complete tail closes promptly instead of holding the full SSE long-poll window. This matches the browser transport's reconnectToStream and gives server-to-server consumers the same fast tail-close. The active send-message path is unchanged (no peek, so it never races a newly-triggered turn's first chunk).
… server fast-close
ericallam
force-pushed
the
feat/s2-caught-up-reads
branch
from
July 31, 2026 13:54
a819882 to
5a9b176
Compare
0ski
approved these changes
Jul 31, 2026
…th cloud defaults
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Upgrades
@s2-dev/streamstorefrom 0.22 to 0.25 and points the S2 client at its current hosts (a.s2.dev/{basin}.b.s2.dev, replacingaws.s2.dev). The host change is the operationally important piece: it tracks where S2 lives now, independent of any feature. It also folds in several chat.agent session-stream reliability fixes and full-stack e2e coverage that came out of this work.Included:
AgentChat.reconnect()now sendsX-Peek-Settled, so a server-to-server reconnect settles on the existing server fast-close instead of waiting out the long poll..outwritable on self-hosted setups that skip S2 access tokens.