Describe the bug
BufferExec drives its input on a background task (MemoryBufferedStream) that fills a channel ahead of the consumer. If the input panics while being polled, the spawned task unwinds, tokio catches the panic at the task boundary, and the channel's sender gets dropped. The consumer (poll_next) then sees the closed channel and returns Ok(None) — a normal end of stream.
A panic isn't a Result::Err, so it never goes through the batch_tx.send(Err(..)) path either. So the panic is effectively swallowed: the partition just stops early and the query finishes "successfully" with whatever was buffered before the panic, instead of failing.
I ran into this with a query that panicked deep in an aggregate (an oversized FixedSizeBinary array) underneath a BufferExec. Instead of an error, the query came back with a partial, under-counted result, which made it look like a data problem rather than a crash.
To Reproduce
Give MemoryBufferedStream an input that panics partway through and drain it — the stream ends cleanly instead of surfacing an error.
Expected behavior
A panic in the input should propagate (an error on the output stream, or a re-raise), not turn into a silent early EOF.
Additional context
BufferExec / MemoryBufferedStream were added in #19760. RecordBatchReceiverStream already propagates panics from its producer tasks; MemoryBufferedStream rolls its own channel and doesn't.
I have a fix and will put up a PR.
Version
main (54.x)
Describe the bug
BufferExecdrives its input on a background task (MemoryBufferedStream) that fills a channel ahead of the consumer. If the input panics while being polled, the spawned task unwinds, tokio catches the panic at the task boundary, and the channel's sender gets dropped. The consumer (poll_next) then sees the closed channel and returnsOk(None)— a normal end of stream.A panic isn't a
Result::Err, so it never goes through thebatch_tx.send(Err(..))path either. So the panic is effectively swallowed: the partition just stops early and the query finishes "successfully" with whatever was buffered before the panic, instead of failing.I ran into this with a query that panicked deep in an aggregate (an oversized
FixedSizeBinaryarray) underneath aBufferExec. Instead of an error, the query came back with a partial, under-counted result, which made it look like a data problem rather than a crash.To Reproduce
Give
MemoryBufferedStreaman input that panics partway through and drain it — the stream ends cleanly instead of surfacing an error.Expected behavior
A panic in the input should propagate (an error on the output stream, or a re-raise), not turn into a silent early EOF.
Additional context
BufferExec/MemoryBufferedStreamwere added in #19760.RecordBatchReceiverStreamalready propagates panics from its producer tasks;MemoryBufferedStreamrolls its own channel and doesn't.I have a fix and will put up a PR.
Version
main (54.x)