fix(vl): raise a clear error on malformed data URLs - #4837
Conversation
_load_data_url did bare tuple-unpacks on the comma and the ';<type>'
separators, so a malformed data: URL (e.g. "data:image/png", "data:,",
"data:image/png;base64") crashed with an opaque
"not enough values to unpack" ValueError. That opaque error was swallowed
by the generic except in async_engine into a generic "in prompt processing
error", hiding the root cause from users and debuggers.
Replace the bare unpacks with length-checked splits that raise a clear,
debuggable ValueError('Malformed data URL: ...') at the media-loader
boundary (the correct layer), mirroring the empty-prompt guard pattern of
InternLM#4803. The async_engine generic catch is intentionally left untouched; it
now surfaces a clear message instead of an opaque unpack error.
Adds test_load_data_url_rejects_malformed (red on master, green here) plus
a positive well-formed base64 case to test_safe_url.py.
There was a problem hiding this comment.
Pull request overview
Improves robustness and debuggability of VL media loading by validating data: URL structure inside lmdeploy.vl.media.connection._load_data_url, replacing opaque tuple-unpack failures with clear, actionable ValueErrors and adding regression tests.
Changes:
- Add length-checked parsing for
data:URLs and raise explicitValueErrors for malformed shapes (missing comma/payload, missing;<type>). - Keep existing behavior for non-
base64data URLs (NotImplementedError) while making malformed input errors clearer. - Add tests covering malformed
data:URLs and a well-formed base64data:URL.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| lmdeploy/vl/media/connection.py | Adds explicit validation and clearer errors for malformed data: URL parsing in _load_data_url. |
| tests/test_lmdeploy/test_vl/test_safe_url.py | Adds regression tests for malformed data: URLs and a well-formed base64 case. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| url_spec_path = url_spec.path or '' | ||
| data_spec, data = url_spec_path.split(',', 1) | ||
| media_type, data_type = data_spec.split(';', 1) | ||
| parts = url_spec_path.split(',', 1) |
There was a problem hiding this comment.
Thanks for your contribution. I understand your intuition, but we can make it simpler for maintenance purposes.
We can simply raise one invalid error rather than the current over-detailed information, which I prefer. And we can also remove unit test for this improvement.
There was a problem hiding this comment.
Thanks for the pointer to vllm's connector. Simplified as suggested: both detailed messages are collapsed into a single concise ValueError: Invalid data URL: data:<path>, and the dedicated unit test for it has been removed (5779fc9). Happy to tweak the wording if you'd prefer it phrased differently.
Per review feedback, replace the two detailed ValueError messages with a single concise 'Invalid data URL' error for simpler maintenance, and remove the dedicated unit test for it.
Motivation
_load_data_url(inlmdeploy/vl/media/connection.py) parsesdata:URLs with two bare tuple-unpacks:A malformed
data:URL that omits the comma and payload (e.g.data:image/png,data:,,data:image/png;base64) makesstr.splitreturn a single-element list, so the unpack raises an opaqueValueError: not enough values to unpack (expected 2, got 1). That error is swallowed by the surroundingexceptin the request path (lmdeploy/serve/core/async_engine.py) into a generic "in prompt processing error", hiding the actual cause from both the caller and the logs.Change
Replace the bare unpacks with length-checked splits that raise a clear, debuggable
ValueErrorat the media-loader boundary (the layer that ownsdata:URL parsing), pointing the caller at the expecteddata:<media-type>;<type>,<payload>shape:Malformed data URL: expected "data:<media-type>;<type>,<payload>" but got "data:...";<type>→Malformed data URL media type: expected "<media-type>;<type>" but got "..."The existing
NotImplementedError('Only base64 data URLs are supported for now.')for non-base64 payloads is unchanged. The generic catch inasync_engine.pyis intentionally left untouched — it now surfaces a clear root-cause message instead of an opaque unpack error.Why a guard rather than relying on the existing catch
The generic catch already prevents the crash from surfacing as an internal error, but it converts a structural input bug into an unactionable "in prompt processing error": a caller sending
data:image/pngcannot tell from the response that their data URL is malformed. Raising the clear error at the loader boundary (the layer that owns data-URL parsing) is the smallest root-cause clarification at the correct layer — no engine-path change, mirroring the empty-prompt guard landed in #4803.Tests
Added
test_load_data_url_rejects_malformed(parametrized over the three malformed shapes) andtest_load_data_url_loads_well_formed_base64. The malformed cases are red onmain(raisenot enough values to unpack, which does not matchMalformed data URL) and green on this branch. Well-formed base64 loading is unchanged.Scope
Two files:
lmdeploy/vl/media/connection.py(the guard) andtests/test_lmdeploy/test_vl/test_safe_url.py(the tests). No engine-layer or serving-path changes.