Skip to content

Commit d833d43

Browse files
committed
fix(streaming): address validated review findings across provider gating and reset reconciliation
Three-reviewer pass over the branch, findings validated against staging: - agent-handler forwards agentEvents to executeProviderRequest — the flag was computed but dropped in the field-by-field copy, so provider-side thinking requests (OpenAI summaries, Gemini includeThoughts, Anthropic summarized display) never activated on opted-in runs - openai: restore summary:'auto' alongside explicit reasoning effort — staging always paired them; gating summary purely on agentEvents changed legacy payloads - gemini: Gemini 2 + tools + responseFormat falls back to the silent path; the live loop never applied the deferred responseSchema for AUTO tools - openai-compat loop: malformed tool-argument JSON fails the call instead of executing with defaulted {} args (staging parsed inside the execution try) - openai-compat parser: a vendor id arriving after a synthesized start no longer renames the call (start/end ids stayed consistent) - stream-pump: abort closes the byte projection so a drain blocked on backpressure cannot deadlock teardown - chunk_reset removes the block from the client text order (deployed chat + panel chat) so a reset block re-registers at arrival position — fixes separator/order corruption when parallel blocks stream around a reset - resume route echoes the negotiated X-Sim-Stream-Protocol response header (parity with the chat route); docs: [DONE] wire shape + final-vs-error terminal semantics corrected
1 parent 854a6db commit d833d43

17 files changed

Lines changed: 288 additions & 26 deletions

File tree

apps/docs/content/docs/en/workflows/deployment/agent-events.mdx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -35,10 +35,10 @@ Answer text stays on `chunk`. Thinking and tools never reuse `chunk` (so older c
3535
| `{ "blockId", "event": "chunk_reset" }` | Opted-in only: discard the block’s streamed answer text — it belonged to a turn that resolved to tool calls |
3636
| `{ "blockId", "event": "thinking", "data": "…" }` | Thinking / reasoning summary delta |
3737
| `{ "blockId", "event": "tool", "phase": "start"\|"end", "id", "name", "status?" }` | Tool lifecycle (no args / results) |
38-
| `{ "event": "final", "data": … }` | Terminal success envelope |
39-
| `{ "event": "error", "error": "…" }` | Terminal failure — followed by `[DONE]`, never by `final` |
38+
| `{ "event": "final", "data": … }` | Terminal result envelope for a settled execution. `data.success` may be `false` with `data.error` when the workflow itself failed |
39+
| `{ "event": "error", "error": "…" }` | Terminal stream failure (timeout, client abort, processing error) — followed by `[DONE]`, never by `final` |
4040
| `{ "event": "stream_error", "blockId?", "error" }` | Non-terminal mid-block read issue; the stream keeps going |
41-
| `data: [DONE]` | Stream closed (always follows the terminal `final` or `error` frame) |
41+
| `data: "[DONE]"` | Stream closed (JSON-encoded sentinel; always follows the terminal `final` or `error` frame) |
4242

4343
### Live answer text and intermediate turns
4444

apps/sim/app/(interfaces)/chat/hooks/use-chat-streaming.test.tsx

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,6 +186,30 @@ describe('useChatStreaming thinking + abort', () => {
186186
expect(assistant?.content).not.toContain('Let me check')
187187
})
188188

189+
it('re-registers a reset block at the end so multi-block order matches arrival', async () => {
190+
mockReadSSEEvents.mockImplementation(async (_source, options) => {
191+
// Agent A streams provisional text, then resets (tools follow).
192+
await options.onEvent({ blockId: 'agent-a', chunk: 'Checking the weather…' })
193+
await options.onEvent({ blockId: 'agent-a', event: 'chunk_reset' })
194+
// Another block streams while A's tools run.
195+
await options.onEvent({ blockId: 'block-b', chunk: 'B output' })
196+
// A's final turn re-streams; the server bakes in the cross-block separator.
197+
await options.onEvent({ blockId: 'agent-a', chunk: '\n\nIt is 68°F.' })
198+
await options.onEvent({
199+
event: 'final',
200+
data: { success: true, output: {} },
201+
})
202+
})
203+
204+
await act(async () => {
205+
await handle.latest().handleStreamedResponse(makeSseResponse(), setMessages, vi.fn(), vi.fn())
206+
})
207+
await flushUiBatch()
208+
209+
const assistant = messages.find((m) => m.id === 'msg-assistant-1')
210+
expect(assistant?.content).toBe('B output\n\nIt is 68°F.')
211+
})
212+
189213
it('settles thinking chrome when a tool starts', async () => {
190214
let midStreamThinking: boolean | undefined
191215
mockReadSSEEvents.mockImplementation(async (_source, options) => {

apps/sim/app/(interfaces)/chat/hooks/use-chat-streaming.ts

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -552,9 +552,16 @@ export function useChatStreaming() {
552552
if (isChatChunkResetFrame(json)) {
553553
// The block's live-streamed text belonged to an intermediate turn
554554
// (tool calls follow); drop it — the final turn re-streams after.
555+
// Remove the block from the order too: its re-streamed text
556+
// re-registers at the end, keeping render order = arrival order
557+
// (the server re-computes the cross-block separator on re-stream).
555558
const { blockId } = json
556559
if (blockTextSegments.has(blockId)) {
557-
blockTextSegments.set(blockId, '')
560+
blockTextSegments.delete(blockId)
561+
const orderIndex = blockTextOrder.indexOf(blockId)
562+
if (orderIndex !== -1) {
563+
blockTextOrder.splice(orderIndex, 1)
564+
}
558565
recomputeAccumulatedText()
559566
// Spoken audio cannot be unplayed; clamp so slicing stays valid.
560567
lastAudioPosition = Math.min(lastAudioPosition, accumulatedText.length)

apps/sim/app/api/resume/[workflowId]/[executionId]/[contextId]/route.ts

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,10 @@ import { getBaseUrl } from '@/lib/core/utils/urls'
2020
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
2121
import { preprocessExecution } from '@/lib/execution/preprocessing'
2222
import { PauseResumeManager } from '@/lib/workflows/executor/human-in-the-loop-manager'
23-
import { createStreamingResponse } from '@/lib/workflows/streaming/streaming'
23+
import {
24+
agentStreamProtocolResponseHeaders,
25+
createStreamingResponse,
26+
} from '@/lib/workflows/streaming/streaming'
2427
import { validateWorkflowAccess } from '@/app/api/workflows/middleware'
2528
import type { ResumeExecutionPayload } from '@/background/resume-execution'
2629
import { ExecutionSnapshot } from '@/executor/execution/snapshot'
@@ -278,6 +281,11 @@ export const POST = withRouteHandler(
278281
return new NextResponse(stream, {
279282
headers: {
280283
...SSE_HEADERS,
284+
// Echo the negotiated stream protocol (same as the public chat route).
285+
...agentStreamProtocolResponseHeaders({
286+
includeThinking: persistedSnapshot.metadata.includeThinking === true,
287+
requestHeaders: request.headers,
288+
}),
281289
'X-Execution-Id': enqueueResult.resumeExecutionId,
282290
},
283291
})

apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/chat/chat.tsx

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -567,8 +567,15 @@ export function Chat() {
567567
}
568568

569569
if (event === 'chunk_reset' && blockId) {
570+
// Drop the block's provisional text and its order slot — the
571+
// final turn re-registers at the end, keeping render order =
572+
// arrival order (separators are recomputed on re-stream).
570573
if (blockSegments.has(blockId)) {
571-
blockSegments.set(blockId, '')
574+
blockSegments.delete(blockId)
575+
const orderIndex = blockOrder.indexOf(blockId)
576+
if (orderIndex !== -1) {
577+
blockOrder.splice(orderIndex, 1)
578+
}
572579
recomputeContent()
573580
contentDirty = true
574581
scheduleFlush()

apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -634,12 +634,13 @@ export function useWorkflowExecution() {
634634

635635
/**
636636
* Intermediate-turn reconciliation: drop the block's streamed text
637-
* (chunk_reset frame) and re-arm separator bookkeeping so the
638-
* final turn's text starts a clean segment.
637+
* (chunk_reset frame) and remove its bookkeeping entirely so
638+
* separator counting ignores it and the final turn (or, if none
639+
* re-streams, onBlockComplete's output fallback) starts clean.
639640
*/
640641
const onStreamReset = (blockId: string) => {
641642
if (!streamedChunks.has(blockId)) return
642-
streamedChunks.set(blockId, [])
643+
streamedChunks.delete(blockId)
643644
processedFirstChunk.delete(blockId)
644645
safeEnqueue(encodeSSE({ blockId, event: 'chunk_reset' }))
645646
}
@@ -1249,7 +1250,7 @@ export function useWorkflowExecution() {
12491250
onStreamChunkReset: (data) => {
12501251
// Live-streamed text belonged to an intermediate turn (tools
12511252
// follow); the final turn re-streams as regular chunks.
1252-
streamedChunks.set(data.blockId, [])
1253+
streamedChunks.delete(data.blockId)
12531254
if (onStreamReset && isExecutingFromChat) {
12541255
onStreamReset(data.blockId)
12551256
}

apps/sim/executor/handlers/agent/agent-handler.test.ts

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1875,6 +1875,52 @@ describe('AgentBlockHandler', () => {
18751875
expect(providerCallArgs.billingAttribution).toEqual(billingAttribution)
18761876
})
18771877

1878+
it('forwards agentEvents to executeProviderRequest on opted-in streaming runs', async () => {
1879+
const inputs = {
1880+
model: 'gpt-4o',
1881+
userPrompt: 'Stream this',
1882+
apiKey: 'test-api-key',
1883+
}
1884+
1885+
const streamingContext = {
1886+
...mockContext,
1887+
stream: true,
1888+
selectedOutputs: ['test-agent-block'],
1889+
metadata: { ...mockContext.metadata, agentEvents: true },
1890+
} as ExecutionContext
1891+
1892+
mockGetProviderFromModel.mockReturnValue('openai')
1893+
1894+
await handler.execute(streamingContext, mockBlock, inputs)
1895+
1896+
expect(mockExecuteProviderRequest).toHaveBeenCalled()
1897+
const providerCallArgs = mockExecuteProviderRequest.mock.calls[0][1]
1898+
expect(providerCallArgs.stream).toBe(true)
1899+
expect(providerCallArgs.agentEvents).toBe(true)
1900+
})
1901+
1902+
it('does not set agentEvents on runs without the run-level opt-in', async () => {
1903+
const inputs = {
1904+
model: 'gpt-4o',
1905+
userPrompt: 'Stream this',
1906+
apiKey: 'test-api-key',
1907+
}
1908+
1909+
const streamingContext = {
1910+
...mockContext,
1911+
stream: true,
1912+
selectedOutputs: ['test-agent-block'],
1913+
} as ExecutionContext
1914+
1915+
mockGetProviderFromModel.mockReturnValue('openai')
1916+
1917+
await handler.execute(streamingContext, mockBlock, inputs)
1918+
1919+
expect(mockExecuteProviderRequest).toHaveBeenCalled()
1920+
const providerCallArgs = mockExecuteProviderRequest.mock.calls[0][1]
1921+
expect(providerCallArgs.agentEvents).toBe(false)
1922+
})
1923+
18781924
it('should handle multiple MCP tools from the same server efficiently', async () => {
18791925
const fetchCalls: any[] = []
18801926

apps/sim/executor/handlers/agent/agent-handler.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1042,6 +1042,7 @@ export class AgentBlockHandler implements BlockHandler {
10421042
verbosity: providerRequest.verbosity,
10431043
thinkingLevel: providerRequest.thinkingLevel,
10441044
previousInteractionId: providerRequest.previousInteractionId,
1045+
agentEvents: providerRequest.agentEvents,
10451046
streamToolCalls: providerRequest.streamToolCalls,
10461047
abortSignal: ctx.abortSignal,
10471048
})

apps/sim/providers/gemini/core.ts

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1024,7 +1024,16 @@ export async function executeGeminiRequest(
10241024
}
10251025

10261026
const initialCallTime = Date.now()
1027-
const shouldStreamToolCalls = request.streamToolCalls ?? false
1027+
/**
1028+
* Gemini 2 cannot combine responseSchema with tools, so structured output
1029+
* is applied on a final schema-configured request after tools settle — the
1030+
* silent path does this; the live loop would break as soon as a turn has
1031+
* no calls and skip the schema. Gemini 3 carries responseJsonSchema
1032+
* alongside tools, so its live loop keeps structured output.
1033+
*/
1034+
const responseFormatNeedsFinalPass = Boolean(request.responseFormat) && !isGemini3Model(model)
1035+
const shouldStreamToolCalls =
1036+
(request.streamToolCalls ?? false) && !responseFormatNeedsFinalPass
10281037
const shouldStream = request.stream && !tools?.length
10291038

10301039
// Live streaming tool loop

apps/sim/providers/openai-compat/stream-events.test.ts

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,47 @@ describe('createOpenAICompatibleAgentEventStream', () => {
9595
])
9696
})
9797

98+
it('keeps a synthesized tool id stable when the vendor id arrives after start', async () => {
99+
const onComplete = vi.fn()
100+
const stream = createOpenAICompatibleAgentEventStream(
101+
(async function* () {
102+
// Name first (no id) → start emits with a synthesized id.
103+
yield {
104+
choices: [
105+
{
106+
delta: {
107+
tool_calls: [{ index: 0, function: { name: 'lookup', arguments: '{"q"' } }],
108+
},
109+
},
110+
],
111+
} as any
112+
// Vendor id arrives late — must not rename the started call.
113+
yield {
114+
choices: [
115+
{
116+
delta: {
117+
tool_calls: [{ index: 0, id: 'call_real', function: { arguments: ':1}' } }],
118+
},
119+
},
120+
],
121+
usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 },
122+
} as any
123+
})(),
124+
{ providerName: 'Loose', emitToolCallStarts: true, onComplete }
125+
)
126+
127+
const events = await collectEvents(stream)
128+
const start = events.find((e) => e.type === 'tool_call_start')
129+
expect(start).toBeDefined()
130+
expect(start!.id).not.toBe('call_real')
131+
132+
// The assembled call keeps the same id as the emitted start.
133+
const assembled = onComplete.mock.calls[0][0].toolCalls
134+
expect(assembled).toHaveLength(1)
135+
expect(assembled[0].id).toBe(start!.id)
136+
expect(assembled[0].function).toEqual({ name: 'lookup', arguments: '{"q":1}' })
137+
})
138+
98139
it('always enqueues text deltas live and assembles content for onComplete', async () => {
99140
const onComplete = vi.fn()
100141
const stream = createOpenAICompatibleAgentEventStream(

0 commit comments

Comments
 (0)