Skip to content

[To dev/1.3] Pipe: dynamically share idle floating memory (#18422) - #18437

Open
Caideyipi wants to merge 1 commit into
apache:dev/1.3from
Caideyipi:cp-10950e13-dev-1.3
Open

[To dev/1.3] Pipe: dynamically share idle floating memory (#18422)#18437
Caideyipi wants to merge 1 commit into
apache:dev/1.3from
Caideyipi:cp-10950e13-dev-1.3

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Summary

  • Backport commit 10950e1 (Pipe: dynamically share idle floating memory with TsFile sync #18422) to dev/1.3.
  • Treat pipe_total_floating_memory_proportion as the upper limit for floating InsertNode memory.
  • Allow non-floating Pipe allocations to borrow unused floating memory and reclaim it symmetrically.
  • Adapt the change to dev/1.3's counter-based PipeMemoryManager and update capacity consumers to use the manager instance.
  • Preserve the dev/1.3 resize admission hard limits and add regression coverage.

Testing

  • Spotless apply for datanode and node-commons: passed.
  • DataNode test-compile with Checkstyle and Spotless: passed.
  • PipeMemoryManagerResizeTest, PipeMemoryManagerTest, and PipeDataNodeTaskAgentTest: 12 tests, 0 failures, 0 errors.
  • git diff --check origin/dev/1.3...HEAD: passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant