[SPARK-56020][SQL] Improve GroupPartitions Spark UI#54842
Open
szehon-ho wants to merge 4 commits intoapache:masterfrom
Open
[SPARK-56020][SQL] Improve GroupPartitions Spark UI#54842szehon-ho wants to merge 4 commits intoapache:masterfrom
szehon-ho wants to merge 4 commits intoapache:masterfrom
Conversation
szehon-ho
changed the title
szehon-ho
commented
Mar 17, 2026
| } | ||
|
|
||
| test("SPARK-55992: GroupPartitions textual representatin in plans") { | ||
| test("SPARK-55992: GroupPartitions textual representation in simple and extended explain") { |
peter-toth
reviewed
Mar 17, 2026
sql/core/src/test/scala/org/apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala
Outdated
Show resolved
Hide resolved
peter-toth
reviewed
Mar 17, 2026
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/GroupPartitionsExec.scala
Outdated
Show resolved
Hide resolved
peter-toth
reviewed
Mar 17, 2026
| } | ||
|
|
||
| /** Summary parts for plan string representation. */ | ||
| private def planSummaryParts(joinKeyMaxFields: Int): Seq[String] = { |
Contributor
There was a problem hiding this comment.
It is a bit weird that planSummaryParts() is used at 2 places but its result is immediately filtered by a .filter(_.nonEmpty) at both places. So how about returning only the non-empty parts and probably as Iterator[String].
Member
Author
|
@peter-toth thanks for taking a look, cleaned up the pr a bit |
szehon-ho
force-pushed
the
group_partitions_string_follow
branch
from
0a7db13 to
34f989c
Compare
peter-toth
reviewed
Mar 18, 2026
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/GroupPartitionsExec.scala
Outdated
Show resolved
Hide resolved
peter-toth
approved these changes
Mar 18, 2026
Contributor
peter-toth
left a comment
peter-toth
left a comment
There was a problem hiding this comment.
LGTM, only minor suggestion.
…rces/v2/GroupPartitionsExec.scala Co-authored-by: Peter Toth <peter.toth@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Follow up for #54459
Why are the changes needed?
Improve UX for the new SPJ group operator
Does this PR introduce any user-facing change?
Adds to Explain Extended, and changes unreleased Spark UI for the GroupPartitions node a bit
How was this patch tested?
Add unit test
Was this patch authored or co-authored using generative AI tooling?
Yes, cursor