fix: validate time zone config before applying changes#23224
Open
Probablism wants to merge 2 commits into
Open
fix: validate time zone config before applying changes#23224Probablism wants to merge 2 commits into
Probablism wants to merge 2 commits into
Conversation
Contributor
|
would it be possible to enforce it more strictly by changing the type of the config to datafusion/datafusion/common/src/config.rs Lines 770 to 773 in ea2ffdb |
5ab1228 to
dbd8d10
Compare
dbd8d10 to
5ab1228
Compare
5ab1228 to
758a3a9
Compare
Author
|
Good point. I had kept this as a string initially to avoid changing the config type, but I agree the typed version is better here. I updated it to store a parsed timezone via |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
datafusion.execution.time_zoneaccepted a raw string value, so invalidtimezones could be stored in the session config and fail later when timestamp
operations tried to parse them.
This changes the config value to store a parsed timezone, so invalid timezone
values are rejected when the setting is applied and the previous valid value
remains active.
What changes are included in this PR?
ConfigTimeZonewrapper around Arrow's parsed timezone type.ExecutionOptions::time_zonefromOption<String>toOption<ConfigTimeZone>.value and only convert back to a string where Arrow metadata/display requires
it.
SQL
SETbehavior, and UDF/FFI config propagation.Are these changes tested?
Yes.
cargo fmt --allcargo clippy --all-targets --all-features -- -D warningscargo test -p datafusion-common test_execution_time_zone_validationcargo test -p datafusion --test user_defined_integration test_config_options_work_for_scalar_funccargo test -p datafusion-ffi --features integration-tests --test ffi_udf test_config_on_scalar_udfcargo test -p datafusion-functions timezonecargo test --profile=ci --test sqllogictests -- set_variablecargo check -p datafusion-common -p datafusion-sql -p datafusion-functions -p datafusion-spark -p datafusion-ffi -p datafusion --testsAre there any user-facing changes?
Yes. Invalid
SET TIME ZONEvalues now fail immediately instead of beingaccepted and causing later timestamp operations to fail.