Aborted mid-stream LiteLLM tool-call finalizes partial arguments into an unhandled JSONDecodeError — truncation guard only covers finish_reason=='length'

[cyyeong]

A mid-stream-aborted LiteLLM tool-call finalizes partial arguments into an unhandled JSONDecodeError — the truncation guard only covers finish_reason == "length"

This issue has been reattributed since it was filed — the original title
blamed 2.5.0 and the version A/B below explains why that was wrong. What
remains is a robustness bug present, byte-identically, in 2.4.0 through
2.6.3.

The bug

_finalize_tool_call_response in models/lite_llm.py has a graceful branch
for truncated tool-call arguments — it returns an LlmResponse with
error_code=MAX_TOKENS and the message "Tool call arguments were truncated
while streaming and could not be parsed as valid JSON. Increase
max_output_tokens and retry." — but the validation is guarded:

for index, func_data in function_calls.items():
  if func_data["id"]:
    if finish_reason == "length":          # <-- only this case is handled
      try:
        _parse_tool_call_arguments(func_data["args"])
      except json.JSONDecodeError:
        has_incomplete_tool_call_args = True
        continue
    tool_calls.append(...)                 # partial args pass through here

A stream that aborts mid-tool-call — provider incident, transport cut,
proxy timeout — ends with no finish_reason at all (and no usage chunk).
The guard is bypassed, the partial arguments reach
_parse_tool_call_arguments, and a bare JSONDecodeError propagates out of
generate_content_async and kills the whole invocation:

  File ".../google/adk/models/lite_llm.py", line 2956, in generate_content_async
    _finalize_tool_call_response(
  File ".../google/adk/models/lite_llm.py", line 2184, in _message_to_generate_content_response
    args=_parse_tool_call_arguments(tool_call.function.arguments),
  File ".../google/adk/models/lite_llm.py", line 199, in _parse_tool_call_arguments
    return json.loads(arguments)
json.decoder.JSONDecodeError: Expecting value: line 1 column 16 (char 15)

The buffer held exactly {"agent_name": — the head of ADK's own
transfer_to_agent arguments; the value never arrived.

Suggested fix: treat "stream ended while a tool call's arguments do not
parse" the same as the length case regardless of finish_reason — the graceful
MAX_TOKENS-style LlmResponse (or a dedicated aborted-stream error code)
instead of an uncaught parse exception. The discriminator is already in hand at
that point: tool-call deltas were received and the stream ended with no
terminal finish_reason. (The missing usage chunk — despite include_usage — corroborates the abnormal
end: 27/27 clean tee'd calls carried one, and every failure in the preserved
incident log is flagged by ADK's own Skipping missing token usage metadata
warning.)

How we hit it, and why the original version-attribution was wrong

During a provider-side incident window (2026-08-13, ~15:56–19:00 UTC, model
openrouter/z-ai/glm-5.2 through LiteLLM), a deployed multi-agent app died on
its first transfer_to_agent in 9 of 10 turns on 2.5.0 while an
interleaved 2.4.0 arm passed 10/10 — which looked exactly like a 2.5.0
regression, and was filed as one.

Re-measured twelve hours later with a wire tee on the same estate, same probe:

Arm (deployed)	Turns	char 15 failures	Transfer calls on the wire
2.4.0	6	0	6/6 complete, finish=tool_calls, usage present
2.5.0	10	0	10/10 complete
2.6.3	10	0	10/10 complete

Same requests (byte-identical messages; the versions' tools payloads differ by
two words of docstring), zero failures on any version. The failing window's
errors each correlate with ADK's own Skipping missing token usage metadata
warning — the streams ended abnormally. OpenRouter serves this model from a
32-provider pool and per-generation lookups show each window's calls
concentrating on one upstream (one arm: 10/10 CoreWeave; another: 8/11 Baidu),
so a window whose routing lands on a misbehaving upstream fails almost every
turn while an adjacent window passes — which is what masqueraded as a version
difference. A minimal standalone harness (coordinator + two sub-agents, same
model) reproduces on no version under either StreamingMode.

So: the 9/10-vs-0/10 A/B was real but confounded; 2.4.0 survived the incident
by routing luck, not by handling — its finalize path is byte-identical. Any ADK
version converts an aborted upstream stream mid-tool-call into a hard
invocation failure, and that is the actionable bug.

Environment

google-adk	2.4.0 / 2.5.0 / 2.6.3 (guard byte-identical in all three)
litellm	1.89.4 (pinned across all measurements)
Model	openrouter/z-ai/glm-5.2 (32-provider pool)
Streaming	StreamingMode.SSE, stream_options={"include_usage": true}
Failing tool	transfer_to_agent (ADK-generated)

Possibly related

• PROGRESSIVE_SSE_STREAMING is not honored by the LiteLlm adapter — function-call argument deltas are buffered until finish_reason #5342 — PROGRESSIVE_SSE_STREAMING is not honored by the LiteLlm adapter
• [LiteLlm] JSONDecodeError on malformed tool_call.function.arguments crashes run before callbacks can recover #5008 (closed) — [LiteLlm] JSONDecodeError on malformed tool_call.function.arguments

[cyyeong]

Correction to the original text (edited above).

The first version of this issue said a standalone harness "reproduced only once the run config was corrected to SSE". That was wrong — that run was killed before it produced a result, and I reported an outcome I had not observed. I have now run it properly against a clean google-adk==2.5.0 + litellm==1.89.4 venv:

Harness	StreamingMode	Turns	Failures
standalone	NONE	5	0
standalone	SSE	6	0
deployed app	SSE	10	9

The minimal harness does not reproduce it under either run config. The 9/10-vs-0/10 bisect through the deployed application path stands and is unaffected; what changed is that I can no longer hand you a small reproduction, and the streaming mode is not the discriminator I claimed it was. Sorry for the noise — better to correct it before anyone spends time running a recipe that passes.

[cyyeong]

Reattributed — title and body rewritten; the version claim is withdrawn.

The original report blamed 2.5.0 on a 9/10-vs-0/10 A/B. Re-measured twelve hours later with a wire-level tee on the same deployment: 2.4.0, 2.5.0 and 2.6.3 all pass the identical probe with zero failures (26/26 transfer calls complete on the wire). The failing window's streams all ended abnormally — no finish_reason, no usage chunk despite include_usage — during what the evidence indicates was a provider-side incident behind OpenRouter's 32-provider pool for this model, with per-window routing concentration making it masquerade as a version difference.

What remains for ADK — and it is byte-identical in 2.4.0 through 2.6.3 — is the robustness gap: _finalize_tool_call_response handles truncated tool-call arguments gracefully only when finish_reason == "length". A stream that aborts mid-tool-call has no finish_reason, bypasses the guard, and the partial arguments raise an uncaught JSONDecodeError that kills the invocation. Full wire evidence and the suggested fix are in the rewritten body.

[chelsealong]

Picking this one up now — opening a PR shortly. Flagging it here so nobody duplicates the work; if someone is already on it, say so and I will drop mine.

[cyyeong]

Update: reproduced on 2.4.0 during a second provider window — the version attribution is conclusively dead.

A new incident window on the same OpenRouter pool (z-ai/glm-5.2) began ~2026-08-14 02:00 UTC. The same deployment that measured 0/10 failures on 2.4.0 during a healthy window has now logged 124 Expecting value: line 1 column 16 (char 15) errors on google-adk 2.4.0, each paired with ADK's own Skipping missing token usage metadata warning (abnormal stream end, no finish_reason).

This is the second independent confirmation that the failure follows the provider window, not the ADK version — and that the actionable bug is the one in this issue's title: an aborted mid-tool-call stream bypasses the finish_reason == "length" guard and kills the invocation with an uncaught JSONDecodeError on every version.

[surajksharma07]

@cyyeong The guard in _finalize_tool_call_response only checks JSON validity when finish_reason=='length' so a stream that just dies mid-tool-call slips right past it.
#6720 patches that by validating args unconditionally.
Could you pull that branch and confirm it clears the transfer_to_agent failures on your traffic before we lean on it?