[BUG] Assembly fails with Calcite classes from unified-query-api #1228
Description
What is the bug?
After introducing the unified-query-api module (which integrates Calcite) into the opensearch-spark project, the sbt assembly command fails with a java.lang.ArrayIndexOutOfBoundsException. The error appears to be caused by ASM exceeding its class file limits during shading.
How can one reproduce the bug?
Steps to reproduce the behavior:
- Add a dependency on
unified-query-apiin the opensearch-spark project. - Run
sbt clean assembly. - Observe the failure during the shading phase.
What is the expected behavior?
The sbt assembly task should complete successfully and generate a shaded fat JAR.
What is your host/environment?
- Version JDK 11
Do you have any screenshots?
N/A
Do you have any additional context?
Here is the stack trace of the failure:
[error] java.lang.ArrayIndexOutOfBoundsException: Index 65536 out of bounds for length 334
[error] at org.objectweb.asm.ClassReader.readLabel(ClassReader.java:2679)
[error] at org.objectweb.asm.ClassReader.createLabel(ClassReader.java:2695)
[error] at org.objectweb.asm.ClassReader.readTypeAnnotations(ClassReader.java:2761)
[error] at org.objectweb.asm.ClassReader.readCode(ClassReader.java:1928)
[error] at org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1514)
[error] at org.objectweb.asm.ClassReader.accept(ClassReader.java:744)
[error] at org.objectweb.asm.ClassReader.accept(ClassReader.java:424)
[error] at com.eed3si9n.jarjar.ScalaSigProcessor.process(ScalaSigProcessor.scala:15)
[error] at com.eed3si9n.jarjar.util.JarProcessorChain.process(JarProcessorChain.java:38)
[error] at com.eed3si9n.jarjar.JJProcessor.process(JJProcessor.scala:108)
[error] at com.eed3si9n.jarjarabrams.Shader$.$anonfun$bytecodeShader$6(Shader.scala:75)
[error] at sbtassembly.Assembly$.$anonfun$shader$2(Assembly.scala:494)
[error] at sbtassembly.Assembly$.$anonfun$assemble$27(Assembly.scala:304)
[error] at scala.collection.parallel.AugmentedIterableIterator.flatmap2combiner(RemainsIterator.scala:133)
[error] at scala.collection.parallel.AugmentedIterableIterator.flatmap2combiner$(RemainsIterator.scala:130)
[error] at scala.collection.parallel.immutable.ParVector$ParVectorIterator.flatmap2combiner(ParVector.scala:66)
[error] at scala.collection.parallel.ParIterableLike$FlatMap.leaf(ParIterableLike.scala:1082)
[error] at scala.collection.parallel.Task.$anonfun$tryLeaf$1(Tasks.scala:53)
[error] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[error] at scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:67)
[error] at scala.collection.parallel.Task.tryLeaf(Tasks.scala:56)
[error] at scala.collection.parallel.Task.tryLeaf$(Tasks.scala:50)
[error] at scala.collection.parallel.ParIterableLike$FlatMap.tryLeaf(ParIterableLike.scala:1078)
[error] at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal(Tasks.scala:170)
[error] at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal$(Tasks.scala:157)
[error] at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:440)
[error] at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:150)
[error] at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute$(Tasks.scala:149)
[error] at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)
[error] at java.base/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
[error] at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
[error] at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
[error] at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
[error] at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
[error] at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
This seems related to known issues: