Skip to content

Reduce coordinator memory growth in citus_move_shard_placement#8652

Draft
ihalatci with Copilot wants to merge 2 commits into
mainfrom
copilot/fix-citus-move-shard-placement-memory-usage
Draft

Reduce coordinator memory growth in citus_move_shard_placement#8652
ihalatci with Copilot wants to merge 2 commits into
mainfrom
copilot/fix-citus-move-shard-placement-memory-usage

Conversation

Copilot AI commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

DESCRIPTION: Bound coordinator memory use during shard moves with many colocations

citus_move_shard_placement() could retain large per-shard DDL structures on the coordinator, causing memory to climb dramatically for large colocated move sets (as in the attached WSS graph). This change makes the block-writes transfer path release transient allocations per shard instead of accumulating them for the full operation.

  • What changed

    • Per-shard memory release in setup phase
      • Reset localContext during shard table recreation loop, so DDL/list allocations are freed each iteration.
    • Removed deferred command accumulation
      • Dropped ShardCommandList buffering of attach/FK DDL across all shards.
      • Execute partition-attach and foreign-key command lists incrementally per shard.
    • Ordering preserved
      • Keep operational ordering: partition attach work runs before foreign-key creation work.
  • Illustrative change

SendCommandListToWorkerOutsideTransaction(targetNodeName, targetNodePort,
                                          tableOwner, ddlCommandList);
MemoryContextReset(localContext);

Copilot AI changed the title [WIP] Fix memory usage in citus_move_shard_placement() Reduce coordinator memory growth in citus_move_shard_placement Jul 8, 2026
Copilot AI requested a review from ihalatci July 8, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

citus_move_shard_placement() uses lots of memory on the coordinator

2 participants