Implement atomic hashing and chunking in PulpExport #7220
Conversation
vycius
force-pushed
the
atomic-chunked-export
branch
from
83fc336 to
6226daa
Compare
vycius
force-pushed
the
atomic-chunked-export
branch
6 times, most recently
from
0579e8b to
aa02e28
Compare
|
Thank you for this PR! Seems like good improvements all around. Just FYI, if you have used AI in any way, just ensure it is documented properly in compliance with our AI policy: https://pulpproject.org/help/more/governance/ai_policy/ |
Replace the external 'split' subprocess and secondary hashing pass with a custom HashingFileWriter. - Resolves a race condition where files moved/deleted between the write and hash phases caused export failures. - Eliminates redundant I/O by calculating checksums while streaming data to disk. - Removes OS dependency on the 'split' binary. - Ensures the Table of Contents (TOC) is generated from in-memory metadata for 100% consistency. closes pulp#7218 Assisted By: Gemini 3
vycius
force-pushed
the
atomic-chunked-export
branch
from
aa02e28 to
58112aa
Compare
Thank you for pointing that out! I wasn't aware of the policy, but I have now updated the commit messages with the 'Assisted-by' tag. For clarity, I used AI to help refactor the logic, generate the initial test suite, and brainstorm potential edge cases. All suggestions were manually reviewed and verified |
2 participants
Replace the external 'split' subprocess and secondary hashing pass with a custom HashingFileWriter.
closes #7218