[refine](column) enforce nullable nested types for array #63088
Open
Mryange wants to merge 3 commits into
Open
[refine](column) enforce nullable nested types for array #63088Mryange wants to merge 3 commits into
Mryange wants to merge 3 commits into
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
/review |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Automated code review summary:
No blocking issues found in this PR.
Critical checkpoint conclusions:
- Goal and tests: The PR makes array element types explicitly nullable in BE and aligns FE array type modeling with that invariant. The changed BE serde unit test covers one Arrow nested-array path, and FE tests were adjusted for array nullability matching. I did not run the test suite in this review runner.
- Scope: The change is focused on array nested nullability and related FE/BE type conversion call sites.
- Concurrency and lifecycle: No new concurrent state, locks, threads, or non-trivial lifecycle/static initialization concerns were introduced.
- Configuration and compatibility: No new configuration items. Existing serialized FE metadata keeps the
containsNullfield for Gson compatibility while treating it as true at runtime. BE thrift/protobuf type descriptors continue to emit nullable array elements. - Parallel paths: I checked FE parser/type conversion/function signatures, BE DataTypeArray/DataTypeArraySerDe, and representative BE factory/consumer paths; the nullable nested type invariant is applied consistently in the modified paths.
- Error handling and invariants: The new
DORIS_CHECKassertions are used for the intended array nested-nullable invariant rather than silently accepting inconsistent columns/serdes. - Data correctness and persistence: No transaction/version/delete-bitmap/write path changes were introduced.
- Memory/performance: The change adds no significant allocations beyond normal nullable type wrapping and does not alter hot data loops except invariant checks on deserialize entry points.
- Observability: No additional observability appears necessary for this type-system refinement.
- User focus: No additional user-provided review focus was present.
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29557 ms |
Contributor
TPC-DS: Total hot run time: 170984 ms |
Contributor
Author
|
run buildall |
Contributor
FE UT Coverage ReportIncrement line coverage |
Contributor
TPC-H: Total hot run time: 29429 ms |
Contributor
TPC-DS: Total hot run time: 171726 ms |
Contributor
Author
|
run buildall |
Contributor
FE UT Coverage ReportIncrement line coverage |
Contributor
TPC-H: Total hot run time: 29622 ms |
Contributor
TPC-DS: Total hot run time: 172004 ms |
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29687 ms |
Contributor
TPC-DS: Total hot run time: 171519 ms |
Contributor
Author
|
run beut |
Contributor
FE UT Coverage ReportIncrement line coverage |
Mryange
force-pushed
the
enforce-nullable-nested-types-for-array-map-2
branch
from
1c1bdc1 to
88ca38a
Compare
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29725 ms |
Contributor
TPC-DS: Total hot run time: 170615 ms |
Contributor
FE UT Coverage ReportIncrement line coverage |
Contributor
Author
|
run beut |
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29817 ms |
Contributor
TPC-DS: Total hot run time: 170923 ms |
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29652 ms |
Contributor
TPC-DS: Total hot run time: 172120 ms |
Contributor
FE UT Coverage ReportIncrement line coverage |
Contributor
Author
|
run p0 |
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
This PR makes the nested types inside Array explicitly nullable in BE type implementations, instead of relying on implicit caller-side conventions.
DataTypeArray now always stores nullable nested element type
DataTypeArraySerDe updated to follow the same invariant
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)