[PECOBLR-330] Support for complex params

[jprakash-db]

Description

This PR introduces comprehensive support for complex parameter types (ARRAY and MAP) in the Databricks SQL Python client.

Key Changes

• Added new parameter classes: ArrayParameter and MapParameter
• These classes recursively wrap Python lists and dictionaries and serialize them into the appropriate Thrift structures.
• The type inference logic dbsql_parameter_from_primitive is updated to recognize and construct these parameter types, including for nested complex values.
• Added the TSparkParamValueArgs which is necessary to be passed as arguements for complex types

Testing

e2e

• Added tests to insert and compare the results for nested structures ARRAY<ARRAY<STRING>>, ARRAY<MAP<STRING,INTEGER>>, MAP<STRING,ARRAY<STRING>>

unit

• Added tests to verify the convertion of the params into the respective server contract format

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[shivam2680]

Timezone will not be there for TIMESTAMP_NTZ param.

[vikrantpuppala]

+1, would like to know how we've accounted/tested for NTZ

[jprakash-db]

If it is not there it will be an empty space, there is already a test suite that inserts NTZ and none NTZ and reads back to compare whether it is equal or not

[jprakash-db]

@vikrantpuppala @shivam2680 There are already existing tests that insert NTZ and non NTZ values and reads back from table to ensure everything is working as expected -

databricks-sql-python/tests/e2e/test_parameterized_queries.py

Line 283 in 3842583

def test_dbsqlparameter_single(

[vikrantpuppala]

+1, would like to know how we've accounted/tested for NTZ

[gopalldb]

Is this not an opt-in feature, can't user disable this in favor of existing behavior?

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[jprakash-db]

@gopalldb We are just adding support for more types, don't think there is anything to opt in. If the users don't use complex types they won't need this

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[gopalldb]

Thanks!