Add lossy option to CastOptions for controlling precision loss in casts

[klion26]

Which issue does this PR close?

• Closes Add lossy flag in CastOptions #9172.

What changes are included in this PR?

This PR adds lossy in CastOptions, which aims to control precision loss in casts.

Behavior matrix:

safe	lossy	Behavior
true	false	Default. Invalid casts return Null, Lossy casts(e.g., float to int) return NULL.
true	true	Invalid casts return Null, Lossy casts are allowed (truncation occurs).
false	false	Invalid casts return an error. Lossy casts return an error.
false	true	Invalid casts return an error. Lossy casts are allowed (truncation occurs).

The generated docs

Are these changes tested?

No tests, this added a new flag.

Are there any user-facing changes?

Yes, changed the CastOptions

[klion26]

@alamb @scovich Please help review this when you're free, thanks.

This wants to add lossy in CastOptions so that we can support lossy cast in variant-get, as this adds new fields in pub struct, so we can only merge this in a major version.

It seems I can't add the api-change tag for this PR. @alamb, please help to add the tag. Thank you.

[tustvold]

I'm possibly missing some context here, but FWIW we already support lossy casts for decimals. I'm not sure about float-> int casting but I thought it rounded to zero, and only returnee null on overflow.

[scovich]

I'm possibly missing some context here, but FWIW we already support lossy casts for decimals.

As in, losing scale in e.g. Decimal(6, 4) -> Decimal(4, 2) Just Works (rounding to zero, so e.g. 99.9999 becomes 99.99)? What about losing precision, e.g. Decimal(6, 2) -> Decimal(4, 2)? Presumably values larger than 99 would have to become error/null?

[tustvold]

Yes, truncation (i.e. precision loss) is permitted, overflow will error (or NULL).

[klion26]

Sorry for the late reply, I missed this notification.

Thanks for the reply。 This is raised in variant_get related code, hoping to support conversion like decimal/float to int, and TimeUnit::Nano -> TimeUnit::Second， etc.

I found the Decimal-related tests(test_decimal_to_decimal_decrease_scale_and_precision_unchecked/test_decimal_to_decimal_increase_scale_and_precision_unchecked) in `arrow-cast/src/cast/mod.rs(Copied some test case below)

test behavior for decimal cast

input -> casted_result

• 99999(Decimal(5, 0)) -> 100(Decimal(3, -3))
• -99999(Decimal(5, 0)) -> -1(Decimal(1, -5))
• 123456789(Decimal(10, 2)) -> 12346(5, -2)
• -9876543210(Decimal(10, 4)) -> -987654(Decimal(7, 0))
• 9999999(Decimal(7, 4)) -> Err(Invalid argument error: 1000.000 is too large to store in a Decimal128 of precision 6. Max is 999.999)
• 99999(Decimal(5,0)) -> 9999900000(Decimal(10,5))
• -99999(Decimal(5, 0)) -> -9999900000(Decimal(10, 5))
• 99999(Decimal(5, 2)) -> 99999000(10, 5)
• -99999(Decimal(5, -2)) -> -9999900000(Decimal(10, 3))
• -12345(Decimal(5, 3)) -> Decimal(6, 5) Err("Invalid argument error: -12.34500 is too small to store in a Decimal128 of precision 6. Min is -9.99999"

Looking at cast_with_option, it seems it has implemented many conversions, so maybe we can directly use or improve cast_with_option? I'll try to figure it out and come back.

[klion26]

Currently, cast/cast_with_option supports different types of transformations, and lossy transformations are performed regardless of whether safe is true or false (e.g., 3.1f64 -> 3i32, 864000003005(TimeUnit::Milli) -> 864000003(TimeUnit::Second), etc.). Considering that this logic has been in place since its introduction in 2019, modifying it would result in a significant behavior change. Perhaps we should leave it as is and reuse the cast logic in variant_get.

[alamb]

Considering that this logic has been in place since its introduction in 2019, modifying it would result in a significant behavior change. Perhaps we should leave it as is and reuse the cast logic in variant_get.

yes, please let's not change how the cast kernels work.

[klion26]

I'm closing this pr/issue now. We can improve the variant_get according to the logic of the cast kernel. Thanks all for the feedback.