Describe the bug
pairwise_diff and pairwise_diff_checked produce incorrect results when the input array has a nonzero offset, such as an array created with slice().
The kernel reads values from the parent array before the logical slice instead of consistently indexing relative to the sliced input.
Reproduction
import pyarrow as pa
import pyarrow.compute as pc
base = pa.array([99, 1, 4, 9, 16, 88], type=pa.int64())
sliced = base.slice(1, 4)
rebuilt = pa.array(sliced.to_pylist(), type=sliced.type)
print("offset:", sliced.offset)
print("input:", sliced.to_pylist())
print("actual:", pc.pairwise_diff(sliced, period=1).to_pylist())
print("expected:", pc.pairwise_diff(rebuilt, period=1).to_pylist())
Output:
offset: 1
input: [1, 4, 9, 16]
actual: [None, -98, 3, 5]
expected: [None, 3, 5, 7]
The -98 result comes from subtracting the parent array's prefix value 99, which is outside the logical slice.
The same behavior is reproducible through the public C++ API.
Expected behavior
A sliced array should produce the same result as an equivalent zero-offset array containing the same logical values:
Actual behavior
The first computed difference reads from the parent array before the slice:
Root cause
In cpp/src/arrow/compute/kernels/vector_pairwise.cc, PairwiseExecImpl copies the input ArraySpan and then calls:
left.SetSlice(left_start, computed_length);
right.SetSlice(right_start, computed_length);
ArraySpan::SetSlice() assigns the provided value as an absolute offset rather than adding it to the existing input offset.
For a sliced input, the shifted spans therefore index from the parent buffers rather than from input.offset.
The validity calculation still uses the original input offset, so the value and validity spans may also refer to different logical elements.
Affected inputs
The issue affects sliced inputs where:
offset > 0
0 < abs(period) < length
It is reproducible for both positive and negative periods and affects numeric, decimal, and temporal kernel signatures that share this implementation.
Proposed direction
Preserve the original input offset when creating the shifted operand spans, and add focused C++ regression coverage for sliced inputs.
I searched existing open and closed issues, pull requests, commits, release notes, tests, and source history and did not find an existing report for this sliced-array offset bug.
Component(s)
C++, Python
Describe the bug
pairwise_diffandpairwise_diff_checkedproduce incorrect results when the input array has a nonzero offset, such as an array created withslice().The kernel reads values from the parent array before the logical slice instead of consistently indexing relative to the sliced input.
Reproduction
Output:
The
-98result comes from subtracting the parent array's prefix value99, which is outside the logical slice.The same behavior is reproducible through the public C++ API.
Expected behavior
A sliced array should produce the same result as an equivalent zero-offset array containing the same logical values:
Actual behavior
The first computed difference reads from the parent array before the slice:
Root cause
In
cpp/src/arrow/compute/kernels/vector_pairwise.cc,PairwiseExecImplcopies the inputArraySpanand then calls:ArraySpan::SetSlice()assigns the provided value as an absolute offset rather than adding it to the existing input offset.For a sliced input, the shifted spans therefore index from the parent buffers rather than from
input.offset.The validity calculation still uses the original input offset, so the value and validity spans may also refer to different logical elements.
Affected inputs
The issue affects sliced inputs where:
It is reproducible for both positive and negative periods and affects numeric, decimal, and temporal kernel signatures that share this implementation.
Proposed direction
Preserve the original input offset when creating the shifted operand spans, and add focused C++ regression coverage for sliced inputs.
I searched existing open and closed issues, pull requests, commits, release notes, tests, and source history and did not find an existing report for this sliced-array offset bug.
Component(s)
C++, Python