Skip to content

direct: Record the CLI that last wrote the deployment state - #6222

Merged
denik merged 2 commits into
mainfrom
denik/state_cli_version
Aug 12, 2026
Merged

denik merged 2 commits into
mainfrom
denik/state_cli_version

Conversation

@denik

@denik denik commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Why

cli_version in resources.json kept the version of the CLI that created the state. Every deploy
wrote the current version into the WAL header, but replay copied only lineage and serial back into
the state, so the field never moved — a state deployed daily by the latest CLI still reported
whatever version first created it.

The terraform engine updates its own cli_version on every deploy, and the field is documented as
the version that last wrote the state, so this brings the direct engine in line with both.

It is refreshed alongside the serial, under the same condition: the state file is only persisted when
the WAL carried entries, so a deploy that commits nothing must not claim to have written one.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 4b2f17d

Run: 31491868526

Env ❌​FAIL 🟨​KNOWN 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
❌​ aws linux 1 1 3 4 294 1130 9:16
❌​ aws windows 1 1 3 4 296 1128 10:08
❌​ azure linux 1 1 3 4 293 1130 8:46
❌​ azure windows 1 1 3 4 295 1128 10:02
❌​ gcp linux 1 1 5 294 1130 8:18
❌​ gcp windows 1 1 5 296 1128 9:16
9 interesting tests: 4 SKIP, 3 RECOVERED, 1 KNOWN, 1 FAIL
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
❌​ TestAccept/pipelines/destroy/cascade-on-destroy-terraform-error ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestFetchRepositoryInfoAPI_FromRepo 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S
💚​ TestFetchRepositoryInfoAPI_FromRepo/root 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/subdir 💚​R 💚​R 💚​R 💚​R

cli_version in resources.json kept the version of the CLI that created the
state: every deploy wrote the current version into the WAL header, but
replay copied only lineage and serial back, so the field never moved.

Refresh it alongside the serial, which is the same condition (the state
file is only persisted when the WAL carried entries). This matches the
terraform engine, which updates its own cli_version on every deploy, and
the field's documented meaning.
@denik
denik force-pushed the denik/state_cli_version branch from 4f78708 to 4b2f17d Compare August 11, 2026 12:33
@denik
denik enabled auto-merge August 11, 2026 12:55
@denik
denik disabled auto-merge August 12, 2026 10:15
@denik
denik merged commit 9d5fd4e into main Aug 12, 2026
9 checks passed
@denik
denik deleted the denik/state_cli_version branch August 12, 2026 10:15
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 9d5fd4e

Run: 31586747155

Env ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
❌​ aws linux 6 1 4 5 2 1157 940 204:35
❌​ aws windows 6 1 2 5 2 1098 959 246:20
🔄​ azure linux 4 6 2 1057 977 196:13
🔄​ azure windows 4 6 2 996 996 234:14
💚​ gcp linux 3 3 1051 981 212:11
🔄​ gcp windows 6 3 3 984 1000 229:53
34 interesting tests: 20 flaky, 6 FAIL, 5 RECOVERED, 2 SKIP, 1 KNOWN
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/deploy/snapshot-comparison ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/deploy/snapshot-comparison/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=schema.yml.tmpl/READPLAN= ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=schema_uppercase_name.yml.tmpl/READPLAN= ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
💚​ TestAccept/bundle/invariant/no_drift 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/resources/dashboards/dataset-catalog-schema ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/dashboards/dataset-catalog-schema/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/grants/registered_models ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/grants/registered_models/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/grants/schemas/all_privileges_coexist ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/grants/schemas/all_privileges_coexist/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/model_serving_endpoints/basic ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/model_serving_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions ❌​F ❌​F 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions ❌​F ❌​F 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F
🔄​ TestAccept/bundle/resources/postgres_projects/update_display_name 🔄​f ✅​p 🙈​s 🙈​s 🙈​s 🙈​s
🔄​ TestAccept/bundle/resources/postgres_projects/update_display_name/DATABRICKS_BUNDLE_ENGINE=direct 🔄​f ✅​p
🔄​ TestAccept/bundle/resources/quality_monitors/change_table_name ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/quality_monitors/change_table_name/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
💚​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestAccept/bundle/templates/default-python/combinations/serverless ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=direct/DLT=no/NBOOK=yes/PY=yes/READPLAN= ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/templates/default-python/integration_classic 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.13 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestFetchRepositoryInfoAPI_FromRepo 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S
💚​ TestFetchRepositoryInfoAPI_FromRepo/root 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/subdir 💚​R 💚​R 💚​R 💚​R
Top 50 slowest tests (at least 2 minutes):
duration env testname
15:13 aws linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
15:04 gcp linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
14:50 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
14:47 gcp windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:57 aws windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:57 gcp linux TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
8:12 gcp linux TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct
8:05 azure linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
8:03 azure linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
8:01 aws windows TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
8:00 gcp windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:57 azure windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:57 aws windows TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
7:47 azure windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:47 aws windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:45 azure linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:38 aws linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:38 azure linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
7:32 gcp linux TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
7:13 azure linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:06 aws linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:02 gcp linux TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct
6:29 gcp linux TestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct
6:29 gcp windows TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct
6:29 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
6:26 aws windows TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
6:22 azure linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
6:21 gcp linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
6:17 aws linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
6:17 azure linux TestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct
6:04 aws linux TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct/DLT=no/NBOOK=no/PY=yes/READPLAN=1
6:04 gcp linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
5:58 gcp windows TestAccept/bundle/resources/apps/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:45 aws windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=job_pydabs_1000_tasks.yml.tmpl/READPLAN=1
5:42 aws linux TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=no/PY=no/READPLAN=
5:41 azure windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=model_serving_endpoint.yml.tmpl/READPLAN=
5:40 aws linux TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:37 azure windows TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:36 aws windows TestAccept/bundle/resources/postgres_databases/update/DATABRICKS_BUNDLE_ENGINE=terraform
5:35 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
5:33 aws windows TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:30 gcp linux TestAccept/bundle/resources/apps/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:28 gcp windows TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:23 azure windows TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=job_apply_policy_default_values_task_cluster.yml.tmpl
5:22 gcp windows TestAccept/bundle/resources/experiments/basic/DATABRICKS_BUNDLE_ENGINE=terraform
5:22 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:20 gcp windows TestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct
5:19 gcp windows TestAccept/bundle/resources/registered_models/basic/DATABRICKS_BUNDLE_ENGINE=terraform
5:10 gcp windows TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=yes/PY=no/READPLAN=
5:09 aws windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1

denik added a commit that referenced this pull request Aug 13, 2026
secrets/basic, secrets/update-value (#6192) and state/permission_level_migration
(#6222) landed on main while this branch was in review, so their goldens still
carried the old "Deploying resources..." output.
denik added a commit that referenced this pull request Aug 13, 2026
secrets/basic, secrets/update-value (#6192) and state/permission_level_migration
(#6222) landed on main while this branch was in review, so their goldens still
carried the old "Deploying resources..." output.
denik added a commit that referenced this pull request Aug 13, 2026
secrets/basic, secrets/update-value (#6192) and state/permission_level_migration
(#6222) landed on main while this branch was in review, so their goldens still
carried the old "Deploying resources..." output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants