feat: Cast numeric (non int) to timestamp#3559
Conversation
41393a5 to
5aa49b0
Compare
5aa49b0 to
88e0c83
Compare
bd49514 to
4d3a6a3
Compare
|
@andygrove I moved the 'assertDataFrameEquals' inside cast test to leverage existing Try and ANSI checks . This should help us gain confidence with native code and perhaps merge it |
|
Test failure: |
|
Thank you @andygrove There seems to be failure per spark version after I fixed the error message handling. I am making changes to fix tests with the right error message |
67d6e37 to
0546da6
Compare
0546da6 to
3f3dd19
Compare
|
it seems like there is some regression between Spark's cast from Float / Double to Timestamp in terms handling extreme values . Investigating into this |
|
Only spark 4 test is failing and it seems like we might not be using right exception parsing |
Would you like to wait forhttps://github.com//pull/3580 ? |
|
@parthchandra sure ! |
|
@andygrove , marking it as a draft to not accidentally merge until @parthchandra 's PR is reviewed |
|
@coderfender I've marked this as ready for review if you could rebase and resolve the ci failures. |
|
Sure thank you |
|
It seems like the errors are transient @parthchandra . Can you please rerun failed actions whenever you get a chance |
7bbe49b to
e593089
Compare
|
@parthchandra @andygrove , Rebased with main . Please review whenever you get a chance . Thank you |
|
Thank you |
Which issue does this PR close?
Closes ##3560 .
Rationale for this change
Adding support for further more native cast support.
This should more or less close out the cast matrix in terms of support (barring other not planned casts such as Timestamp -> Int etc)
What changes are included in this PR?
How are these changes tested?
CometCastSuite.scalaand added new ones (along with benchmarking scripts) on the rust side to test cast at all Eval Modes