Skip to content

Core: Read content stats from v4 Manifest - #17433

Draft
nastra wants to merge 6 commits into
apache:mainfrom
nastra:read-content-stats-from-manifest
Draft

Core: Read content stats from v4 Manifest#17433
nastra wants to merge 6 commits into
apache:mainfrom
nastra:read-content-stats-from-manifest

Conversation

@nastra

@nastra nastra commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

This reads the content stats in the v4 Manifest Reader.

Used Claude for the initial prototyping but reviewed and adjusted the code manually

@github-actions github-actions Bot added the core label Jul 30, 2026
@nastra
nastra marked this pull request as draft July 30, 2026 15:45
@nastra
nastra force-pushed the read-content-stats-from-manifest branch from 0e72e44 to 44cadbc Compare July 30, 2026 15:53
@nastra
nastra force-pushed the read-content-stats-from-manifest branch from 44cadbc to c62305e Compare July 30, 2026 18:21
@nastra
nastra force-pushed the read-content-stats-from-manifest branch from c62305e to 6cf0fd0 Compare July 30, 2026 18:25
Comment thread core/src/test/java/org/apache/iceberg/TestV4ManifestReader.java Outdated
Comment thread core/src/main/java/org/apache/iceberg/V4ManifestReader.java Outdated
Comment thread core/src/test/java/org/apache/iceberg/TestV4ManifestReader.java Outdated
Comment thread core/src/main/java/org/apache/iceberg/V4ManifestReader.java Outdated
readBuilder.setCustomType(TrackedFile.CONTENT_STATS_ID, ContentStatsStruct.class);
// content_stats holds one stats struct per projected column
for (Types.NestedField fieldStats : statsField.type().asStructType().fields()) {
readBuilder.setCustomType(fieldStats.fieldId(), FieldStatsStruct.class);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does geo and variant need any special handling here? Can we add a test to make sure those types work?

@anoopj anoopj left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the file pruning coming in a followup?

@nastra
nastra force-pushed the read-content-stats-from-manifest branch from 5420c62 to ad7a0fe Compare July 31, 2026 07:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants