bundle/direct: add a feature-flag list to the direct deployment state by shreyas-goenka · Pull Request #5660 · databricks/cli

shreyas-goenka · 2026-06-19T08:42:51Z

Why

The direct deploy engine records its state on disk with a state_version. Today, any backwards-incompatible change forces a version bump, which older CLIs reject wholesale. That's too coarse: an additive capability that only some deployments use shouldn't lock every older CLI out of every new state.

This adds feature flags to the state: a deployment records the named features it depends on, and an older CLI that doesn't understand a recorded feature refuses that state (pointing at docs) while still reading every state that uses no features.

What

Header.Features — a map[feature-name]FeatureInfo recording the features a state depends on. FeatureInfo is intentionally empty (presence of the key is what matters); it's a struct so per-feature fields can be added later without reshaping the state.
On load, checkSupportedFeatures rejects a state recording a feature this CLI doesn't know, with an actionable error: upgrade to the latest CLI version and see <doc-url>. The doc URL is derived from the feature name (docBaseURL + name), so only the name lives in the state.
Feature names are defined in a static supportedFeatures set. A dummy feature exercises the mechanism in tests; no real deployment records a feature yet — this PR is the mechanism only.

Version handling (the subtle part)

The baseline currentStateVersion stays 2 — normal deploys are unchanged, so there's no golden churn and no forced upgrade.

A state that records a feature is written at featureStateVersion (3), specifically so an older CLI (which reads only up to 2) refuses it rather than silently ignoring a feature it depends on.

On load, a version-3 state with no features is normalized to version 2. This is the deferral hook: a later release can start writing version 3 unconditionally — the real bump — and CLIs carrying this change already tolerate those states instead of rejecting them. We get the mechanism in now and bake it in before flipping the switch.

eng-dev-ecosystem-bot · 2026-06-19T09:10:31Z

Integration test report

Commit: 38c6f2c

Run: 28954883906

	Env	🔄flaky	💚RECOVERED	🙈SKIP	✅pass	🙈skip	Time
💚	aws linux		4	4	230	1070	4:04
💚	aws windows		4	4	232	1068	5:08
💚	aws-ucws linux		4	4	314	988	8:05
💚	aws-ucws windows		4	4	316	986	7:03
💚	azure linux		4	4	230	1069	4:21
💚	azure windows		4	4	232	1067	4:15
🔄	azure-ucws linux	3	1	4	316	985	12:03
🔄	azure-ucws windows	1	3	4	318	983	8:44
💚	gcp linux		4	4	229	1071	4:00
💚	gcp windows		4	4	231	1069	4:26

8 interesting tests: 4 SKIP, 3 flaky, 1 RECOVERED

	Test Name	aws linux	aws windows	aws-ucws linux	aws-ucws windows	azure linux	azure windows	azure-ucws linux	azure-ucws windows	gcp linux	gcp windows
💚	TestAccept	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R
🙈	TestAccept/bundle/invariant/no_drift	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/ssh/connection	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🔄	TestFetchRepositoryInfoAPI_FromRepo	💚R	💚R	💚R	💚R	💚R	💚R	🔄f	🔄f	💚R	💚R
🔄	TestFetchRepositoryInfoAPI_FromRepo/root	💚R	💚R	💚R	💚R	💚R	💚R	🔄f	💚R	💚R	💚R
🔄	TestFetchRepositoryInfoAPI_FromRepo/subdir	💚R	💚R	💚R	💚R	💚R	💚R	🔄f	💚R	💚R	💚R

Top 7 slowest tests (at least 2 minutes):

duration	env	testname
3:58	aws windows	TestAccept
3:29	aws-ucws windows	TestAccept
3:23	gcp windows	TestAccept
3:10	azure windows	TestAccept
3:02	azure-ucws windows	TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform
3:00	azure-ucws windows	TestAccept
2:01	azure-ucws linux	TestImportDirDoesNotOverwrite

shreyas-goenka · 2026-06-29T12:49:29Z

+	for name := range db.Features {
+		names = append(names, name)
+	}
+	slices.Sort(names)


makes tests with multiple features deterministic.

github-actions · 2026-06-29T12:58:30Z

Approval status: pending

`/acceptance/bundle/` - needs approval

9 files changed
Suggested: @denik
Also eligible: @pietern, @janniklasrose, @andrewnester, @anton-107, @lennartkats-db

`/bundle/` - needs approval

Files: bundle/direct/dstate/migrate.go, bundle/direct/dstate/state.go, bundle/direct/dstate/state_test.go
Suggested: @denik
Also eligible: @pietern, @janniklasrose, @andrewnester, @anton-107, @lennartkats-db

_{Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @simonfaltum, @renaudhartert-db, @janniklasrose) can approve all areas.

See OWNERS for ownership rules.}

shreyas-goenka · 2026-06-29T13:41:22Z

+// RecordFeature marks the state as depending on the named feature, stamping the
+// minimum CLI version required to read it (from featureMinCLIVersion). It must be
+// called before the WAL is started (UpgradeToWrite) so the change is persisted.
+func (db *DeploymentState) RecordFeature(name string) {


Get rid of this function - make this decision at open.

shreyas-goenka · 2026-06-29T13:41:50Z

+{
+ "state_version": 3,
+ "features": {
+  "dummy": "1.2.0"


talk to Julia, if aligned get rid of versions from state and consider hosting every feature on our docs page.

denik · 2026-07-08T10:40:29Z

We can consider adding a "per-feature" version but omitting this for now.

This is outdated, right? We discussed that version in the state file cannot work because it's frozen and we might need/want to recommend a more recent one.

denik · 2026-07-08T10:41:33Z

+
+=== a state recording an unknown feature flag is rejected
+>>> errcode [CLI] bundle plan
+Error: the deployment state requires feature "future_feature" which this CLI ([DEV_VERSION]) does not support; upgrade to 9.9.9 or newer and see https://docs.databricks.com/future-feature for more information on how to proceed


upgrade to 9.9.9

where does 9.9.9 from from? Can we replace recommendation to "Upgrade to latest CLI version. See URL for more information".

Just a random version for the test that was persisted in the state file. We can - either message works.

denik · 2026-07-08T10:43:29Z

+ "features": {
+  "dummy": {
+   "min_cli_version": "1.2.0",
+   "doc_url": "https://docs.databricks.com/dev-tools/bundles/deployment-modes"


We don't need the URL here, we can just hard code url with anchor based on feature name:

https://docs.databricks.com/dev-tools/bundles/state-features#dummy

That is more fragile. This is more forward looking and allows us to change the structure of the URLs if necessary. If we hardcode in the binary we'll be stuck with a format (or would be forced to use the same parent URL)

shreyas-goenka · 2026-07-08T10:50:17Z

This is outdated, right? We discussed that version in the state file cannot work because it's frozen and we might need/want to recommend a more recent one.

This is different. I mean like each feature flag having it's own version. Incase we need to make more breaking changes within the context of a single feature flag. We don't need it because we can just introduce new flags.

Bumps the direct state schema to version 3, adding a per-state feature map (Header.Features) recording each feature flag the state depends on. On load the CLI rejects any state recording a feature it does not know, pointing the user at the feature's documentation (derived from its name), so older CLIs fail with an actionable message instead of silently mishandling the state. Feature flags are defined in a static set (supportedFeatures). A dummy feature exercises the mechanism in tests; no real deployment records one yet. Co-authored-by: Isaac

shreyas-goenka · 2026-07-08T13:18:22Z

+// docBaseURL is the documentation prefix a feature name is appended to, to form
+// the link shown when a feature is unsupported. The full URL is derived from the
+// feature name (docBaseURL + name), so only the name is recorded in the state.
+const docBaseURL = "https://docs.databricks.com/aws/en/dev-tools/bundles/"


Not done yet. We need to update it once teh docs page lands and fix it here to use anchors

Adds a per-state feature map (Header.Features) recording each feature flag a state depends on. The baseline state version stays 2; a state that records a feature is written at version 3 so an older CLI (which reads only up to 2) refuses it instead of silently ignoring the feature. On load, a version-3 state with no features is treated as version 2, so a later release can start writing version 3 unconditionally — the actual version bump — while CLIs with this change already tolerate such states. A CLI that loads a state recording a feature it does not know rejects it and points the user at the feature's documentation (derived from its name). Feature flags are defined in a static set (supportedFeatures); a dummy feature exercises the mechanism in tests, and no real deployment records one yet. Co-authored-by: Isaac

The WAL header now records the state's schema version and feature flags, and recovery restores them. Previously a crash between opening the WAL for write and Finalize would replay the WAL onto the pre-crash state and silently drop the features the interrupted deploy recorded, downgrading the recovered state. Co-authored-by: Isaac

Collapse the currentStateVersion / featureStateVersion "nothing to migrate" checks into one branch. No behavior change. Co-authored-by: Isaac

…l case Adds an acceptance case for a version-3 state with an empty features map (the CLI accepts it, treating it as version 2), so all three feature-flag cases — known, unknown, empty-v3 — are exercised together. Documents that the "version 3 + empty features == version 2" equivalence is special-cased to version 3 only, and renames the unit test to TestEmptyFeatureStateEquivalentToVersion2 as a forcing function: it must be removed when the baseline is actually bumped to 3, so a real v3 state is not silently reinterpreted as v2. Co-authored-by: Isaac

The unsupported-feature error now links to a single fixed documentation page (the state-features section Julia is adding) instead of deriving a per-feature URL from the feature name. There is one docs page for all features, so drop docBaseURL/featureDocURL and use a fixed featuresDocURL constant. Co-authored-by: Isaac

shreyas-goenka mentioned this pull request Jun 19, 2026

bundle/direct: add a feature-flag list to the direct deployment state #5403

Closed

shreyas-goenka temporarily deployed to test-trigger-is June 19, 2026 08:43 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch 2 times, most recently from 663ad08 to a1b0613 Compare June 29, 2026 09:30

shreyas-goenka temporarily deployed to test-trigger-is June 29, 2026 09:30 — with GitHub Actions Inactive

shreyas-goenka commented Jun 29, 2026

View reviewed changes

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from a1b0613 to a550105 Compare June 29, 2026 12:55

shreyas-goenka temporarily deployed to test-trigger-is June 29, 2026 12:56 — with GitHub Actions Inactive

shreyas-goenka marked this pull request as ready for review June 29, 2026 12:57

shreyas-goenka requested review from andrewnester, denik and pietern June 29, 2026 12:58

shreyas-goenka commented Jun 29, 2026

View reviewed changes

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from a550105 to 2d62964 Compare July 6, 2026 23:04

shreyas-goenka temporarily deployed to test-trigger-is July 6, 2026 23:04 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from 2d62964 to e559fca Compare July 7, 2026 00:26

shreyas-goenka temporarily deployed to test-trigger-is July 7, 2026 00:26 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from e559fca to c62c873 Compare July 7, 2026 08:59

shreyas-goenka temporarily deployed to test-trigger-is July 7, 2026 08:59 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from c62c873 to 1a7c392 Compare July 7, 2026 09:01

shreyas-goenka temporarily deployed to test-trigger-is July 7, 2026 09:01 — with GitHub Actions Inactive

shreyas-goenka temporarily deployed to test-trigger-is July 7, 2026 09:05 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from d6fdd85 to 329e0fc Compare July 7, 2026 09:05

shreyas-goenka temporarily deployed to test-trigger-is July 7, 2026 09:06 — with GitHub Actions Inactive

denik reviewed Jul 8, 2026

View reviewed changes

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from 329e0fc to c330241 Compare July 8, 2026 11:16

shreyas-goenka temporarily deployed to test-trigger-is July 8, 2026 11:17 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from c330241 to 4f3a6f8 Compare July 8, 2026 11:20

shreyas-goenka temporarily deployed to test-trigger-is July 8, 2026 11:21 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from 4f3a6f8 to 2df14e3 Compare July 8, 2026 12:27

shreyas-goenka temporarily deployed to test-trigger-is July 8, 2026 12:27 — with GitHub Actions Inactive

shreyas-goenka commented Jul 8, 2026

View reviewed changes

shreyas-goenka temporarily deployed to test-trigger-is July 8, 2026 15:05 — with GitHub Actions Inactive

shreyas-goenka temporarily deployed to test-trigger-is July 8, 2026 15:29 — with GitHub Actions Inactive

bundle/direct/dstate: simplify migrateState version branches

38c6f2c

Collapse the currentStateVersion / featureStateVersion "nothing to migrate" checks into one branch. No behavior change. Co-authored-by: Isaac

shreyas-goenka temporarily deployed to test-trigger-is July 8, 2026 15:30 — with GitHub Actions Inactive

shreyas-goenka added 2 commits July 9, 2026 11:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

bundle/direct: add a feature-flag list to the direct deployment state#5660

bundle/direct: add a feature-flag list to the direct deployment state#5660
shreyas-goenka wants to merge 6 commits into
mainfrom
shreyas-goenka/direct-state-feature-flags

shreyas-goenka commented Jun 19, 2026 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Jun 19, 2026 •

edited

Loading

Uh oh!

shreyas-goenka Jun 29, 2026

Uh oh!

github-actions Bot commented Jun 29, 2026 •

edited

Loading

Uh oh!

shreyas-goenka Jun 29, 2026

Uh oh!

shreyas-goenka Jun 29, 2026

Uh oh!

denik commented Jul 8, 2026

Uh oh!

denik Jul 8, 2026

Uh oh!

shreyas-goenka Jul 8, 2026

Uh oh!

denik Jul 8, 2026

Uh oh!

shreyas-goenka Jul 8, 2026

Uh oh!

shreyas-goenka commented Jul 8, 2026

Uh oh!

shreyas-goenka Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

shreyas-goenka commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

Version handling (the subtle part)

Uh oh!

eng-dev-ecosystem-bot commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration test report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approval status: pending

/acceptance/bundle/ - needs approval

/bundle/ - needs approval

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

denik commented Jul 8, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shreyas-goenka commented Jul 8, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shreyas-goenka commented Jun 19, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Jun 19, 2026 •

edited

Loading

github-actions Bot commented Jun 29, 2026 •

edited

Loading

`/acceptance/bundle/` - needs approval

`/bundle/` - needs approval