Commit 0a429a3
authored
fix: emit unmatched NULL-key right rows in RIGHT/FULL PiecewiseMergeJoin (#24336)
## Which issue does this PR close?
- Closes #24335.
## Rationale for this change
A `RIGHT`/`FULL` `PiecewiseMergeJoin` with a range predicate drops an
unmatched right-side row whose join key is `NULL`. A `NULL` key never
matches (`NULL < x` is UNKNOWN), so in a `RIGHT`/`FULL` join the row is
unmatched and must still be emitted with NULLs on the left — but
`PiecewiseMergeJoinExec` omits it, diverging from `NestedLoopJoin`.
```sql
create table l(v int) as values (5);
create table r(v int) as values (10), (NULL);
select l.v, r.v from l right join r on l.v < r.v; -- drops (NULL, NULL)
```
Root cause: `resolve_classic_join` starts the match scan past the
streamed side's `NULL`-keyed rows (they sort to the front under
`nulls_first`). Those rows are never revisited, so for `Right`/`Full`
they were never added to `unmatched_indices` and got dropped.
## What changes are included in this PR?
- In `resolve_classic_join`, when skipping the streamed side's leading
`NULL`-key rows, record them as unmatched for `Right`/`Full` joins so
they are emitted (with NULLs on the buffered side).
## Are these changes tested?
Yes.
- Regression test in `pwmj.slt`: a `RIGHT JOIN` over the existing
`null_join_*` tables now emits the `(NULL, NULL)` row. The test fails on
`main` (the row is dropped) and passes with this change.
- Verified more broadly with a differential fuzz against
`NestedLoopJoin` (same SQL, `enable_piecewise_merge_join` on vs off):
1200 checks over random `RIGHT JOIN` inputs with `<`/`<=`/`>`/`>=` and
high right-side NULL density, 0 mismatches.
## Are there any user-facing changes?
`RIGHT`/`FULL` range joins via `PiecewiseMergeJoin` (behind
`enable_piecewise_merge_join`, default off) now return unmatched right
rows with `NULL` keys, matching `NestedLoopJoin`. No API changes.1 parent e76f1af commit 0a429a3
2 files changed
Lines changed: 55 additions & 0 deletions
File tree
- datafusion
- physical-plan/src/joins/piecewise_merge_join
- sqllogictest/test_files
Lines changed: 10 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
481 | 481 | | |
482 | 482 | | |
483 | 483 | | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
484 | 494 | | |
485 | 495 | | |
486 | 496 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
342 | 342 | | |
343 | 343 | | |
344 | 344 | | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
345 | 390 | | |
346 | 391 | | |
0 commit comments