Skip to content

Conversation

@gabotechs
Copy link
Contributor

@gabotechs gabotechs commented Jan 12, 2026

Which issue does this PR close?

  • Closes #.

Rationale for this change

Prerequisite for the following PRs:

Even if the api on the MemoryPool does not require &mut self for growing/shrinking the reserved size, the api in MemoryReservation does, making simple implementations irrepresentable without synchronization primitives. For example, the following would require a Mutex for concurrent access to the MemoryReservation in different threads, even though the MemoryPool doesn't:

let mut stream: SendableRecordBatchStream = SendableRecordBatchStream::new();
let mem: Arc<MemoryReservation> = Arc::new(MemoryReservation::new_empty());

let mut builder = ReceiverStreamBuilder::new(10);
let tx = builder.tx();
{
    let mem = mem.clone();
    builder.spawn(async move {
        while let Some(msg) = stream.next().await {
            mem.try_grow(msg.unwrap().get_array_memory_size()); // ❌ `mem` is not mutable
            tx.send(msg).unwrap();
        }
    });
}
builder
    .build()
    .inspect_ok(|msg| mem.shrink(msg.get_array_memory_size()));  // ❌ `mem` is not mutable

What changes are included in this PR?

Make the methods in MemoryReservation require &self instead of &mut self for allowing concurrent shrink/grows from different tasks for the same reservation.

Are these changes tested?

yes, by current tests

Are there any user-facing changes?

Users can now safely call methods of MemoryReservation from different tasks without synchronization primitives.

This is a backwards compatible API change, as it will work out of the box for current users, however, depending on their clippy configuration, they might see some new warnings about "unused muts" in their codebase.

@github-actions github-actions bot added core Core DataFusion crate execution Related to the execution crate datasource Changes to the datasource crate physical-plan Changes to the physical-plan crate labels Jan 12, 2026
@gabotechs gabotechs force-pushed the do-not-require-mut-in-memory-reservation branch from fa504b1 to 2a538b1 Compare January 13, 2026 09:51
@gabotechs gabotechs force-pushed the do-not-require-mut-in-memory-reservation branch from 2a538b1 to a578d54 Compare January 13, 2026 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate datasource Changes to the datasource crate execution Related to the execution crate physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant