Skip to content

Remove from string column#19790

Open
ishanema03 wants to merge 7 commits intoapache:mainfrom
ishanema03:remove-from-string-column
Open

Remove from string column#19790
ishanema03 wants to merge 7 commits intoapache:mainfrom
ishanema03:remove-from-string-column

Conversation

@ishanema03
Copy link

Which issue does this PR close?

Rationale for this change

As mentioned in the issue, the From trait implementations for Column were misleading - they invoked Column::from_qualified_name() which parses and lower-cases field names, making conversions like field.name().into() behave unexpectedly. Requiring explicit calls to Column::from_qualified_name() makes the behavior clear and prevents misuse.

What changes are included in this PR?

  • Removed From, From<&str>, and From<&String> trait implementations for Column
  • Updated col() and unnest() functions to accept strings directly
  • Replaced all .into() usages with explicit Column::from_qualified_name() calls across the codebase

Are these changes tested?

Yes, all existing tests have been updated and pass with the new API.

Are there any user-facing changes?

Yes, this is a breaking API change:

  • Users must now call Column::from_qualified_name() explicitly instead of using .into()
  • The helper functions col() and unnest() still accept strings directly

@github-actions github-actions bot added sql SQL Planner logical-expr Logical plan and expressions optimizer Optimizer rules core Core DataFusion crate substrait Changes to the substrait crate common Related to common crate labels Jan 13, 2026
@ishanema03 ishanema03 marked this pull request as draft January 13, 2026 12:31
@ishanema03 ishanema03 force-pushed the remove-from-string-column branch from 97d3400 to 56a14c5 Compare January 13, 2026 12:45
@ishanema03 ishanema03 marked this pull request as ready for review January 13, 2026 12:52
@findepi findepi requested a review from alamb February 18, 2026 16:17
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @ishanema03

I think this is a non trivial breaking API (as you can see by the number of API changes)

I understand the rationale for trying to make the API easier to use, but I think we need to balance that against the need to avoid breaking downstream users who are already using the APIs

Can someone help me understand if there is some particular bug this is solving?

I think some of the nuance on API change guidelines isn't well documented, so I'll make a PR to update it https://datafusion.apache.org/contributor-guide/api-health.html

@alamb
Copy link
Contributor

alamb commented Feb 26, 2026

I wrote up some thoughts here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common Related to common crate core Core DataFusion crate logical-expr Logical plan and expressions optimizer Optimizer rules sql SQL Planner substrait Changes to the substrait crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove From<String> for Column

2 participants