fix(streaming): drain remaining bytes after [DONE] before closing response

[Sahith59]

Closes #3440

Problem

When Stream.__stream__ (and AsyncStream.__stream__) encounters the [DONE] SSE event, it immediately breaks out of the loop and the finally block calls response.close() / await response.aclose().

At that moment, the underlying HTTP/1.1 chunked transfer iterator has not been read to EOF. Specifically, the chunked terminator (0\r\n\r\n) may still be buffered in the kernel or in h11's receive buffer. h11 tracks the remote connection state machine. When response.close() is called while h11's their_state is still SEND_RESPONSE (terminator not yet parsed), httpcore takes the destroy-the-connection branch instead of the return-to-pool (IDLE) branch. This emits an immediate TCP FIN on the socket.

Observable symptoms:

• Upstream proxy logs: spike of downstream_remote_disconnect (Envoy / nginx / any HTTP/1.1 chunked-forwarding gateway)
• Client side: occasional httpcore.RemoteProtocolError / httpx.RemoteProtocolError: peer closed connection without sending complete message body on the next request

Root Cause & Regression History

This was originally fixed in 7e2b2544 ("fix(client): correctly flush the stream response body"). It was accidentally removed in 6132922c ("fix(client): close streams without requiring full consumption") and has been present in all releases since 2.15.0, including the current 2.44.0.

Fix

After observing [DONE], exhaust the remaining SSE iterator before breaking. This drains the chunked terminator through h11's state machine so that their_state reaches DONE, and response.close() then takes the graceful path (back to IDLE, connection returned to the pool).

Sync path (Stream.__stream__)

if sse.data.startswith("[DONE]"):
    # Drain remaining bytes so h11 state reaches DONE before close.
    for _ in iterator:
        pass
    break

Async path (AsyncStream.__stream__)

if sse.data.startswith("[DONE]"):
    # Drain remaining bytes so h11 state reaches DONE before aclose.
    async for _ in iterator:
        pass
    break

Impact

This fix restores the connection-pooling behaviour that existed before 6132922c. High-throughput applications using streaming completions behind a reverse proxy (Envoy, nginx, etc.) will immediately see a reduction in spurious downstream_remote_disconnect errors and improved connection reuse / latency.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ec966628f3

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Avoid waiting past the terminal SSE marker

When a server or proxy emits the app-level [DONE] marker but leaves the SSE response open, this nested drain keeps reading until the HTTP body reaches EOF, so the final next()/until_done() call can block until the read timeout (or forever with no read timeout) instead of completing at [DONE]; the async path has the same pattern. This regresses OpenAI-compatible SSE intermediaries that use [DONE] as the stream terminator but do not close immediately, so the post-DONE drain should be bounded or otherwise avoid waiting for an unbounded EOF.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Suppress post-DONE drain failures

If the transport reports a read/protocol error while consuming bytes after [DONE] (for example, a proxy closes the chunked response without the terminating chunk), this newly added drain propagates that exception even though the SDK has already received the terminal application marker; before this change the stream completed at [DONE] and closed the unusable connection. For OpenAI-compatible gateways with occasional abrupt closes after the sentinel, fully consumed streams can now fail on the final iteration, so drain errors after the sentinel should be ignored while still closing the response.

Useful? React with 👍 / 👎.

[Sahith59]

Thanks for the Codex review! Addressed both P2 items in the latest push:

1. Suppress post-DONE drain failures: Wrapped both sync and async drain loops in try/except Exception so that transport errors after [DONE] (e.g. a proxy closing abruptly without sending the chunked terminator) are silently swallowed. The application-level stream is complete at [DONE], so any subsequent transport noise is irrelevant.

2. Bound the drain: Because the drain is now wrapped in try/except, the user's configured read timeout will fire as a caught exception if the server keeps the SSE connection open indefinitely after [DONE]. The drain is therefore best-effort: it drains if possible, and gives up cleanly on any error or timeout.

The updated comments in the code explain both rationales inline.