Skip to content

[lake/paimon] Support custom Paimon path#3523

Open
wzx140 wants to merge 2 commits into
apache:mainfrom
wzx140:codex/custom-paimon-path-from-fork-main
Open

[lake/paimon] Support custom Paimon path#3523
wzx140 wants to merge 2 commits into
apache:mainfrom
wzx140:codex/custom-paimon-path-from-fork-main

Conversation

@wzx140

@wzx140 wzx140 commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Allow users to set the Paimon path option through custom table properties.
  • Reject altering paimon.path after datalake has been enabled for the table.
  • Add regression coverage for unsettable Paimon options and paimon.path alter validation.

Test Plan

  • JAVA_HOME=/opt/homebrew/Cellar/openjdk@11/11.0.24/libexec/openjdk.jdk/Contents/Home ./mvnw -pl fluss-lake/fluss-lake-paimon -am -Dtest=LakeEnabledTableCreateITCase#testCreateLakeEnableTableWithUnsettablePaimonOptions+testAlterPaimonPathOnlyWhenLakeDisabled -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false test

@wzx140 wzx140 marked this pull request as ready for review June 25, 2026 08:55
@luoyuxia luoyuxia requested a review from Copilot July 3, 2026 06:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Enables configuring a custom Paimon table path via table custom properties, and adds server-side validation to prevent changing paimon.path once a table is datalake-enabled (plus regression coverage for the new behavior).

Changes:

  • Allow paimon.path to be set by removing CoreOptions.PATH from Paimon’s “unsettable options” list.
  • Extend alter-table property validation to consider custom property keys and reject paimon.path changes when datalake is enabled.
  • Add IT coverage for unsettable Paimon options and paimon.path alter validation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
fluss-server/src/main/java/org/apache/fluss/server/utils/TableDescriptorValidation.java Adds validation for altering paimon.path via custom properties when datalake is enabled.
fluss-server/src/main/java/org/apache/fluss/server/coordinator/MetadataManager.java Passes custom-keys-to-change into alter validation during table property updates.
fluss-lake/fluss-lake-paimon/src/main/java/org/apache/fluss/lake/paimon/utils/PaimonConversions.java Makes Paimon path user-settable by removing it from the unsettable-options list.
fluss-lake/fluss-lake-paimon/src/test/java/org/apache/fluss/lake/paimon/LakeEnabledTableCreateITCase.java Adds regression tests for unsettable options and paimon.path alter restrictions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@luoyuxia luoyuxia left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wzx140 Thanks for the pr.
IIUC, paimon.path is intended to customize the physical location only when the Paimon table is created. Allowing it to be changed after the table has already been created looks unsafe to me, because changing this option does not migrate the existing Paimon metadata/data from the old path to the new one.

The current check only rejects changing paimon.path while table.datalake.enabled is true. However, a table can be enabled once, then disabled later, while the Paimon table still exists.
In that state, altering paimon.path would still be allowed. Could we make paimon.path a create-time-only option, or reject changing it once the lake table has ever been created?

@wzx140

wzx140 commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

@wzx140 Thanks for the pr. IIUC, paimon.path is intended to customize the physical location only when the Paimon table is created. Allowing it to be changed after the table has already been created looks unsafe to me, because changing this option does not migrate the existing Paimon metadata/data from the old path to the new one.

The current check only rejects changing paimon.path while table.datalake.enabled is true. However, a table can be enabled once, then disabled later, while the Paimon table still exists. In that state, altering paimon.path would still be allowed. Could we make paimon.path a create-time-only option, or reject changing it once the lake table has ever been created?

Thanks for the review! Good catch. I’ll update the implementation accordingly.

@wzx140 wzx140 force-pushed the codex/custom-paimon-path-from-fork-main branch from 5641661 to f07558d Compare July 3, 2026 09:24
@wzx140

wzx140 commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

@luoyuxia I’ve updated the implementation to reject changes to paimon.path once the lake table has been created. Could you please review it again?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants