YARN-11921. Replace unsupported org.xolstice.maven.plugins in hadoop-yarn-csi by edwardcapriolo · Pull Request #8156 · apache/hadoop

edwardcapriolo · 2025-12-31T20:23:23Z

Description of PR

xolstice plugin is unsupported thus causing you to pull old out of date binaries from maven. This replaces the plugin with a different one that is supported.

How was this patch tested?

Completed the build even on alpine linux as well as locally on fedora 42.

hadoop-yetus · 2025-12-31T22:16:40Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	15m 23s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	34m 23s		trunk passed
+1 💚	compile	0m 34s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 35s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	mvnsite	0m 40s		trunk passed
+1 💚	javadoc	0m 35s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 33s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	shadedclient	63m 9s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 32s		the patch passed
+1 💚	compile	0m 27s		the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 27s		the patch passed
+1 💚	compile	0m 26s		the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 26s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 29s		the patch passed
+1 💚	javadoc	0m 24s		the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 23s		the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	shadedclient	27m 22s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 41s		hadoop-yarn-csi in the patch passed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		111m 10s

Subsystem	Report/Notes
Docker	ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8156/1/artifact/out/Dockerfile
GITHUB PR	#8156
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint
uname	Linux f5d208cd817f 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `4a6f665`
Default Java	Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8156/1/testReport/
Max. process+thread count	614 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8156/1/console
versions	git=2.25.1 maven=3.9.11
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

ayushtkn

LGTM

@cnauroth / @aajisaka anyone with some bandwidth for an additional review?

aajisaka · 2026-01-12T15:56:42Z

            </plugin>
            <plugin>
-                <groupId>org.xolstice.maven.plugins</groupId>
+                <groupId>io.github.ascopes</groupId>


The plugin version is unset. Which plugin version do you want to use?

aajisaka · 2026-01-12T16:01:57Z

This PR is targeted only for YARN. Do you want to replace the plugin in Common and HDFS as well as YARN in this PR?

Common and HDFS config:

hadoop/hadoop-project/pom.xml

Lines 2380 to 2420 in 1dab293

    
           <plugin> 
        
             <groupId>org.xolstice.maven.plugins</groupId> 
        
             <artifactId>protobuf-maven-plugin</artifactId> 
        
             <version>${protobuf-maven-plugin.version}</version> 
        
             <extensions>true</extensions> 
        
             <configuration> 
        
               <protocArtifact> 
        
                 com.google.protobuf:protoc:${hadoop.protobuf.version}:exe:${os.detected.classifier} 
        
               </protocArtifact> 
        
               <attachProtoSources>false</attachProtoSources> 
        
             </configuration> 
        
             <executions> 
        
               <execution> 
        
                 <id>src-compile-protoc</id> 
        
                 <phase>generate-sources</phase> 
        
                 <goals> 
        
                   <goal>compile</goal> 
        
                 </goals> 
        
                 <configuration> 
        
                   <includeDependenciesInDescriptorSet>false</includeDependenciesInDescriptorSet> 
        
                   <protoSourceRoot>${basedir}/src/main/proto</protoSourceRoot> 
        
                   <outputDirectory>${project.build.directory}/generated-sources/java</outputDirectory> 
        
                   <clearOutputDirectory>false</clearOutputDirectory> 
        
                   <skip>true</skip> 
        
                 </configuration> 
        
               </execution> 
        
               <execution> 
        
                 <id>src-test-compile-protoc</id> 
        
                 <phase>generate-test-sources</phase> 
        
                 <goals> 
        
                   <goal>test-compile</goal> 
        
                 </goals> 
        
                 <configuration> 
        
                   <protoTestSourceRoot>${basedir}/src/test/proto</protoTestSourceRoot> 
        
                   <outputDirectory>${project.build.directory}/generated-test-sources/java</outputDirectory> 
        
                   <clearOutputDirectory>false</clearOutputDirectory> 
        
                   <skip>true</skip> 
        
                 </configuration> 
        
               </execution> 
        
             </executions> 
        
           </plugin>

edwardcapriolo · 2026-01-15T14:39:33Z

This PR is targeted only for YARN. Do you want to replace the plugin in Common and HDFS as well as YARN in this PR?

Common and HDFS config:

hadoop/hadoop-project/pom.xml

Lines 2380 to 2420 in 1dab293

<plugin>

<groupId>org.xolstice.maven.plugins</groupId>

<artifactId>protobuf-maven-plugin</artifactId>

<version>${protobuf-maven-plugin.version}</version>

<extensions>true</extensions>

<configuration>

<protocArtifact>

com.google.protobuf:protoc:${hadoop.protobuf.version}:exe:${os.detected.classifier}

</protocArtifact>

<attachProtoSources>false</attachProtoSources>

</configuration>

<executions>

<execution>

<id>src-compile-protoc</id>

<phase>generate-sources</phase>

<goals>

<goal>compile</goal>

</goals>

<configuration>

<includeDependenciesInDescriptorSet>false</includeDependenciesInDescriptorSet>

<protoSourceRoot>${basedir}/src/main/proto</protoSourceRoot>

<outputDirectory>${project.build.directory}/generated-sources/java</outputDirectory>

<clearOutputDirectory>false</clearOutputDirectory>

<skip>true</skip>

</configuration>

</execution>

<execution>

<id>src-test-compile-protoc</id>

<phase>generate-test-sources</phase>

<goals>

<goal>test-compile</goal>

</goals>

<configuration>

<protoTestSourceRoot>${basedir}/src/test/proto</protoTestSourceRoot>

<outputDirectory>${project.build.directory}/generated-test-sources/java</outputDirectory>

<clearOutputDirectory>false</clearOutputDirectory>

<skip>true</skip>

</configuration>

</execution>

</executions>

</plugin>

I will commit to to doing the rest in the next commit, but is it possible we merge this as it is. It does no harm and I am working mostly with nodemanager right now. This poses a challenge for me.

aajisaka · 2026-01-16T02:17:44Z

I will commit to to doing the rest in the next commit, but is it possible we merge this as it is.

Thanks, I'll merge this.

…yarn-csi (#8156) Signed-off-by: Ayush Saxena <ayushsaxena@apache.org> Signed-off-by: Akira Ajisaka <aajisaka@apache.org> (cherry picked from commit d785981)

cnauroth · 2026-01-30T16:47:26Z

I think I'll need to revert this from branch-3.4. I noticed that the build in start-build-env.sh is failing now with a bytecode version compatibility error trying to use the new plugin. On branch-3.4, the build Docker image uses Java 8.

The change can remain in trunk, where we've upgraded to Java 17.

… hadoop-yarn-csi (#8156)" This reverts commit 7b63fb9. This change is incompatible with Java 8, which is used by the build Docker image.

cnauroth · 2026-01-31T05:28:11Z

Reverted from branch-3.4.

LamiumAmplexicaule · 2026-02-17T19:51:39Z

Since the version is not specified, using the newly released v5.0.0 results in build failure due to breaking changes.

$ ./start-build-env.sh ubuntu_24
$ git log --oneline -n1
56192be95af (HEAD -> trunk, origin/trunk, origin/HEAD) HDFS-17861. The mis-behavior of commitBlockSynchronization may cause standby namenode and observer namenode crash. (#8120)
$ mvn package -f hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml
~~~
[ERROR] Failed to execute goal io.github.ascopes:protobuf-maven-plugin:5.0.0:generate (default) on project hadoop-yarn-csi: The parameters 'protoc' for goal io.github.ascopes:protobuf-maven-plugin:5.0.0:generate are missing or invalid -> [Help 1]

https://github.com/ascopes/protobuf-maven-plugin/releases/tag/v5.0.0

Specifying a version earlier than v5.0.0 does not cause the build to fail.

$ git diff hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml
index fd780bd6963..91c96d749f6 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml
@@ -197,6 +197,7 @@
             <plugin>
                 <groupId>io.github.ascopes</groupId>
                 <artifactId>protobuf-maven-plugin</artifactId>
+                <version>4.1.3</version>
                 <configuration>
                     <protocVersion>${hadoop.protobuf.version}</protocVersion>
                     <binaryMavenPlugins>
$ mvn package -f hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml
~~~
[INFO] BUILD SUCCESS
~~~

cnauroth · 2026-02-17T21:40:22Z

@LamiumAmplexicaule , thanks for pointing this out. I'm testing a slightly different version of the patch and will send a pull request shortly.

cnauroth · 2026-02-17T21:59:09Z

@LamiumAmplexicaule , thanks for pointing this out. I'm testing a slightly different version of the patch and will send a pull request shortly.

This is now tracked in YARN-11930 and #8255 .

HADOOP-19772 replace end of life plugin

4a6f665

github-actions Bot added YARN build trunk labels Dec 31, 2025

ayushtkn approved these changes Jan 10, 2026

View reviewed changes

Comment thread hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml

aajisaka reviewed Jan 12, 2026

View reviewed changes

aajisaka approved these changes Jan 16, 2026

View reviewed changes

aajisaka changed the title ~~HADOOP-19772 replace end of life plugin~~ YARN-11921. Replace unsupported org.xolstice.maven.plugins Jan 16, 2026

aajisaka changed the title ~~YARN-11921. Replace unsupported org.xolstice.maven.plugins~~ YARN-11921. Replace unsupported org.xolstice.maven.plugins in hadoop-yarn-csi Jan 16, 2026

aajisaka merged commit d785981 into apache:trunk Jan 16, 2026
1 of 3 checks passed

slfan1989 mentioned this pull request Jan 19, 2026

YARN-11921. [Addendum] Use consistent hadoop.protobuf.version for protoc #8192

Merged

6 tasks

cnauroth added a commit that referenced this pull request Jan 31, 2026

Revert "YARN-11921. Replace unsupported org.xolstice.maven.plugins in…

65dce02

… hadoop-yarn-csi (#8156)" This reverts commit 7b63fb9. This change is incompatible with Java 8, which is used by the build Docker image.

cnauroth mentioned this pull request Feb 17, 2026

YARN-11930: Newly released protobuf-maven-plugin incompatible with hadoop-yarn-csi build #8255

Closed

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

YARN-11921. Replace unsupported org.xolstice.maven.plugins in hadoop-yarn-csi#8156

YARN-11921. Replace unsupported org.xolstice.maven.plugins in hadoop-yarn-csi#8156
aajisaka merged 1 commit intoapache:trunkfrom
edwardcapriolo:HADOOP-19772

edwardcapriolo commented Dec 31, 2025

Uh oh!

hadoop-yetus commented Dec 31, 2025

Uh oh!

ayushtkn left a comment

Uh oh!

Uh oh!

aajisaka Jan 12, 2026

Uh oh!

aajisaka commented Jan 12, 2026

Uh oh!

edwardcapriolo commented Jan 15, 2026

Uh oh!

aajisaka commented Jan 16, 2026

Uh oh!

Uh oh!

cnauroth commented Jan 30, 2026

Uh oh!

cnauroth commented Jan 31, 2026

Uh oh!

LamiumAmplexicaule commented Feb 17, 2026

Uh oh!

cnauroth commented Feb 17, 2026

Uh oh!

cnauroth commented Feb 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

edwardcapriolo commented Dec 31, 2025

Description of PR

How was this patch tested?

Uh oh!

hadoop-yetus commented Dec 31, 2025

Uh oh!

ayushtkn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

aajisaka Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

aajisaka commented Jan 12, 2026

Uh oh!

edwardcapriolo commented Jan 15, 2026

Uh oh!

aajisaka commented Jan 16, 2026

Uh oh!

Uh oh!

cnauroth commented Jan 30, 2026

Uh oh!

cnauroth commented Jan 31, 2026

Uh oh!

LamiumAmplexicaule commented Feb 17, 2026

Uh oh!

cnauroth commented Feb 17, 2026

Uh oh!

cnauroth commented Feb 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants