Support optional DoDelete in the direct engine - #6647
Merged
Merged
Conversation
Collaborator
Integration test reportCommit: 7b45a97
Top 6 slowest tests (at least 2 minutes):
|
denik
force-pushed
the
denik/optional-delete
branch
4 times, most recently
from
September 14, 2026 11:11
cf7ee33 to
2c44c49
Compare
denik
marked this pull request as ready for review
September 14, 2026 11:13
denik
force-pushed
the
denik/optional-delete
branch
from
September 14, 2026 11:23
2c44c49 to
3a2329b
Compare
denik
requested review from
andrewnester,
janniklasrose,
pietern and
shreyas-goenka
September 14, 2026 11:25
Resources whose deletion has no backend effect (grants, permissions, secret_scope_acls) can now omit DoDelete instead of implementing a no-op. The engine treats a planned Delete for such a resource as a state-only cleanup (new PlanEntry.StateOnly flag): - adapter.go: DoDelete is optional; HasDoDelete() reports it. - Direct planner skips the remote read for the Delete (only used to detect already-gone resources, pointless when there is no delete call) and marks the entry StateOnly; apply removes the state entry without a delete call. - Shared rendering/counts (bundle plan text, deploy summary, destroy list, approval, CountActions) treat StateOnly as non-destructive/unchanged; the Delete stays in the JSON plan. - Terraform plan reader marks the same groups StateOnly so both engines' human output stays consistent (no change to what terraform applies). Co-authored-by: Isaac <no-reply@databricks.com>
janniklasrose
approved these changes
Sep 14, 2026
Member
There was a problem hiding this comment.
Remove permissions and grants from
cli/bundle/direct/dresources/all_test.go
Line 1187 in c04c675
nvm, that test still calls DoDelete which still is no-op after this change
|
|
||
| >>> [CLI] bundle plan | ||
| delete secret_scopes.second | ||
| delete secret_scopes.second.permissions |
Member
There was a problem hiding this comment.
this is actually a nice effect of this change, avoids user confusion regarding "what is going to happen to the permissions" (because deleting the resource does not immediately reset the permissions, at least on V1 /permissions API with eventual consistency)
denik
force-pushed
the
denik/optional-delete
branch
from
September 14, 2026 11:32
3a2329b to
7b45a97
Compare
denik
added a commit
that referenced
this pull request
Sep 14, 2026
#6647 ("optional DoDelete in the direct engine") landed on main and now deletes the orphaned permissions child on the deploy path when its config block is removed. This PR's per-resource reporting prints that as an inline "Deleted <resource>.permissions" line, which the main-generated goldens lacked. Regenerated permissions/jobs/update and permissions/pipelines/update; grants and secret_scopes had no drift. Verified 0 failures across 5 runs each. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
#6647 ("optional DoDelete in the direct engine") landed on main and now deletes the orphaned permissions child on the deploy path when its config block is removed. This PR's per-resource reporting prints that as an inline "Deleted <resource>.permissions" line, which the main-generated goldens lacked. Regenerated permissions/jobs/update and permissions/pipelines/update; grants and secret_scopes had no drift. Verified 0 failures across 5 runs each. Co-authored-by: Isaac <no-reply@databricks.com>
Collaborator
Integration test reportCommit: 928419c
508 interesting tests: 396 FAIL, 102 MISS, 7 KNOWN, 2 flaky, 1 SKIP
Top 50 slowest tests (at least 2 minutes):
|
sunishsheth2009
pushed a commit
to sunishsheth2009/cli
that referenced
this pull request
Sep 15, 2026
## Changes Remove `DoDelete` from the `job_run` resource, so `bundle destroy` and trigger-driven recreations no longer delete (or cancel) the triggered run: it is left on the backend and only dropped from bundle state. With no `DoDelete`, the direct engine treats the delete as a state-only cleanup (databricks#6647). ## Why A job run is a historical record. Deleting it on destroy or recreate is surprising and throws away run history the user may still want to inspect. ## Tests Unit tests and the acceptance goldens under `job_runs/` regenerated. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
## Changes Resources whose delete has no backend effect -`grants`, `permissions`, secret scope ACLs (and job_runs in the future) can now omit `DoDelete` instead of implementing a no-op. A planned delete of such a resource becomes a state-only cleanup: the engine skips the plan-time `DoRead` (its only purpose was detecting an already-gone resource, pointless when there is no delete call), drops the state entry at apply without running the delete path, and omits it from `plan`/`deploy`/`destroy` output and counts. `prevent_destroy` is inert for these resources, and the delete is still recorded in the JSON plan. The terraform plan reader marks the same resources so both engines report identically. ## Why Removing a `grants`/`permissions`/ACL block from a bundle changes nothing on the backend, so reporting it as a deletion — and reading it at plan time — is misleading and wasteful. It also drops a spurious "does not have Manage permissions" warning that destroy printed for these resources. ## Tests Acceptance goldens under `permissions/` and `secret_scopes/` regenerated; unit tests cover the adapter, the direct planner, and the terraform plan reader. _This PR was written by Claude Code._ Co-authored-by: Isaac <no-reply@databricks.com>
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
## Changes Remove `DoDelete` from the `job_run` resource, so `bundle destroy` and trigger-driven recreations no longer delete (or cancel) the triggered run: it is left on the backend and only dropped from bundle state. With no `DoDelete`, the direct engine treats the delete as a state-only cleanup (#6647). ## Why A job run is a historical record. Deleting it on destroy or recreate is surprising and throws away run history the user may still want to inspect. ## Tests Unit tests and the acceptance goldens under `job_runs/` regenerated. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Sep 16, 2026
## Release v1.17.0 ### Notable Changes * Bump the direct deployment state version to 3. Clients older than v1.8.0 will reject bundles deployed with this release. ([#6713](#6713)) ### CLI * Add an `INVALID_REFRESH_TOKEN` error code to `databricks auth token --output json` failures. ([#6684](#6684)) * Add experimental `databricks auth docker configure` to configure Docker credential helper access for Databricks Artifact Registry. ([#6700](#6700)) * Add experimental `databricks auth docker token` to generate Docker credentials for Databricks Artifact Registry. ([#6699](#6699)) * `databricks environments setup-local` now reports the `E_PROVISION_CONFLICT` error code instead of the generic `E_PROVISION` when `uv sync` fails to resolve a dependency conflict. ([#6666](#6666)) * Preserve SSH sessions across temporary tunnel disconnects, with bounded replay and backpressure for large transfers. ([#6650](#6650)) * Allow OAuth U2M logins to override the CLI client ID with `--client-id`, profile `client_id`, or `DATABRICKS_CLIENT_ID`. ([#6594](#6594)) ### Bundles * direct: Store a dashboard's `serialized_dashboard` in state as a content hash instead of its full contents. ([#6105](#6105)) * direct: Fix pipelines recreation when the whole `ingestion_definition` block is added or removed. ([#6589](#6589)) * `bundle plan`, `deploy`, and `destroy` no longer report removing `permissions`, `grants`, or secret scope ACLs from a bundle as a deletion, since it leaves the resource untouched. ([#6647](#6647)) * `bundle plan` and `deploy` no longer list or count a resource that was already deleted remotely as a deletion, matching `bundle destroy`; applying still cleans up its stale state entry. ([#6675](#6675)) * Fix `bundle run` failing with `expected an int, found a string` when an unrelated resource references another resource that is not deployed. `bundle run` now resolves `${resources.*}` references only within the resource being run. ([#6690](#6690)) * Add grants support for the AI Gateway `model_service`, `mcp_service`, and `model_provider_service` resources (direct engine). ([#6635](#6635)) * Add bundle support for the AI Gateway `mcp_service` resource (direct engine). ([#6633](#6633)) * Add bundle support for the AI Gateway `model_provider_service` resource (direct engine). ([#6634](#6634)) * Add bundle support for the AI Gateway `model_service` resource (direct engine). ([#6525](#6525)) * Prevent resource drift on catalogs if `storage_root` contained a trailing slash in the URL. ([#6622](#6622)) * Fixed a "lineage mismatch in state files" error that could occur after destroying a bundle and redeploying it from another machine. `bundle destroy` now removes the local state file so no stale lineage is left behind, and prunes the state directories it leaves empty (such as `.internal/` and `sync-snapshots/`). ([#6210](#6210), [#6685](#6685)) * direct: `bundle plan` no longer reports a permanent update on a cluster that uses a cluster policy: when the cluster spec sets `policy_id`, a field present in the remote but absent from the bundle config is not treated as drift. ([#6531](#6531)) * `bundle deploy` on the direct engine now reports each resource as soon as it is deployed, instead of listing them all after the deployment finishes. A deploy that fails part way through now reports the resources it did apply. ([#6361](#6361)) * Direct-engine bundles no longer flag phantom drift on server-populated nested fields under reused config types (e.g. `external_locations` file-event-queue resource IDs, `database_instances` parent-instance refs, `apps` git credential ID). ([#6618](#6618)) * `databricks bundle generate app` now reproduces a git-backed app's `git_repository` and `git_source` configuration instead of emitting a workspace `source_code_path`, so generating from a Git-deployed app no longer silently converts it to workspace source. ([#6656](#6656)) * Improved configuration load time for bundles with many included files. ([#6195](#6195)) * `bundle destroy` no longer deletes triggered job runs, leaving them untouched on the backend. ([#6672](#6672)) * direct: resources.job\_runs: new lifecycle.triggers.on\_file\_change setting to restart the run when monitored files change. Can be set to a series of paths or globs. ([#6309](#6309)) * Bundle summary now shows a name for Postgres branches, endpoints, databases, and roles instead of a blank Name field. ([#6663](#6663)) * Added PyDABs (Python) support for cluster policies, dashboards, and Genie spaces. ([#6585](#6585)) * CLI commands no longer imply that a resource whose type has no workspace URL is merely not deployed yet. ([#6583](#6583)) * Capture the implicit dependency a vector search index has on a catalog or schema defined in the same bundle, so the catalog and schema are deployed first. ([#6655](#6655)) ### Dependency Updates * Bump dependencies with known vulnerabilities. ([#6695](#6695)) * Bump `github.com/databricks/databricks-sdk-go` from v0.177.0 to v0.178.0. ([#6673](#6673)) * Bump Terraform provider from v1.131.0 to v1.132.0. ([#6671](#6671))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Resources whose delete has no backend effect -
grants,permissions, secret scope ACLs (and job_runs in the future) can now omitDoDeleteinstead of implementing a no-op. A planned delete of such a resource becomes a state-only cleanup: the engine skips the plan-timeDoRead(its only purpose was detecting an already-gone resource, pointless when there is no delete call), drops the state entry at apply without running the delete path, and omits it fromplan/deploy/destroyoutput and counts.prevent_destroyis inert for these resources, and the delete is still recorded in the JSON plan. The terraform plan reader marks the same resources so both engines report identically.Why
Removing a
grants/permissions/ACL block from a bundle changes nothing on the backend, so reporting it as a deletion — and reading it at plan time — is misleading and wasteful. It also drops a spurious "does not have Manage permissions" warning that destroy printed for these resources.Tests
Acceptance goldens under
permissions/andsecret_scopes/regenerated; unit tests cover the adapter, the direct planner, and the terraform plan reader.This PR was written by Claude Code.