HDDS-15171. Add available space check on follower during bootstrap.#10185
Draft
sadanand48 wants to merge 5 commits into
Draft
HDDS-15171. Add available space check on follower during bootstrap.#10185sadanand48 wants to merge 5 commits into
sadanand48 wants to merge 5 commits into
Conversation
sadanand48
changed the title
sadanand48
changed the title
Contributor
smengcl
reviewed
May 7, 2026
Comment on lines
+2383
to
+2384
| <name>ozone.om.bootstrap.min.space</name> | ||
| <value>5GB</value> |
Contributor
There was a problem hiding this comment.
This is a good first step, but the best approach is to get an estimate on how much this space it would actually need to download and unpack, because it could be well exceeding 5GB? CMIIW
Contributor
There was a problem hiding this comment.
+1. Maybe reuse the estimates from org.apache.hadoop.ozone.om.snapshot.logEstimatedTarballSize in the preemptive space check before the transfer.
| for (Throwable t = ioe; t != null; t = t.getCause()) { | ||
| if (t instanceof FileSystemException) { | ||
| FileSystemException fse = (FileSystemException) t; | ||
| String reason = fse.getReason(); |
Contributor
There was a problem hiding this comment.
Can do the error message string matching once
Suggested change
| String reason = fse.getReason(); | |
| String reason = (t instanceof FileSystemException fse && fse.getReason() != null) | |
| ? fse.getReason() : t.getMessage(); |
| @@ -4101,6 +4052,13 @@ public synchronized TermIndex installSnapshotFromLeader(String leaderId) throws | |||
| omDBCheckpoint = omRatisSnapshotProvider. | |||
| downloadDBSnapshotFromLeader(leaderId); | |||
| } catch (IOException ex) { | |||
| if (OmRatisSnapshotProvider.isDiskFullOrQuotaIOException(ex)) { | |||
Contributor
There was a problem hiding this comment.
Isn't an error already logged in downloadDBSnapshotFromLeader? Why do we need to log it again here?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Currently if follower doesn't have enough space to accomodate the tarball from the leader, it fails the attempt however leader will keep trying to install snapshot,
This PR is to add a space check before starting the transfer. The space check is against a predefined config whose default size is set to 5GB.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-15171
How was this patch tested?
unit tests