fix: preserve non-empty theta v2 exact sketches - #189
Conversation
There was a problem hiding this comment.
Pull request overview
Fixes legacy Theta compact serialization v2 (“exact”, preamble_longs == 2) being deserialized as logically empty even when it contains retained entries, aligning Rust behavior with the C++ and Go legacy readers and preventing downstream set-operation short-circuiting.
Changes:
- Fix v2 exact deserialization to set
emptybased onnum_entries == 0instead of unconditionallytrue. - Add regression tests that construct valid v2 exact images and assert non-empty behavior, estimate/bounds correctness, iteration stability, and current-format round-trip preservation.
- Add a regression test ensuring zero-entry v2 exact images remain logically empty.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
datasketches/src/thetafamily/theta/sketch.rs |
Corrects v2 exact logical-emptiness reconstruction by tying empty to num_entries. |
datasketches/tests/serde_tests/theta.rs |
Adds compatibility tests and a helper to synthesize v2 exact images for regression coverage. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
ZENOTME
left a comment
There was a problem hiding this comment.
LGTM. Maybe we refine test later.
|
|
||
| use crate::serialization_test_data; | ||
|
|
||
| fn serialize_v2_exact(entries: &[u64]) -> Vec<u8> { |
There was a problem hiding this comment.
I'm not sure there are better way to provide a v2 serialize snapshot instead of construct by hand here. 🤔
There was a problem hiding this comment.
v2 serialize snapshot
Maybe. But existing C++/Java/Go impls do not provide one. We may provide one when implementing #176 , and if you'd like to, you can go to other impls to ask them to provide one and update https://github.com/apache/datasketches-tck.
Summary
Fix legacy Theta serialization version 2 exact images being deserialized as logically empty when they contain retained entries.
The v2 exact branch now derives logical emptiness from
num_entries == 0instead of setting it unconditionally.Closes #186.
Regression coverage
The new serialization compatibility tests construct valid v2 exact images using the current Theta family ID and default seed hash, then verify:
is_empty()is false,Relationship to other implementations
This restores agreement with the current C++ and Go legacy readers. Both treat a v2 exact image (
preamble_longs == 2) as empty only whennum_entries == 0:Go retains compatibility fixtures identified as Java-generated v2 images. Current Java reads compact versions 3 and 4 rather than v2 directly, so the active legacy-reader behavior is represented by C++ and Go:
The fix changes no serialization bytes emitted by Rust; it only corrects the state reconstructed from legacy input.
Validation
cargo x prepare-testdatacargo x checkcargo x testcargo x lint