Capture UC dependencies for vector search indexes by janniklasrose · Pull Request #6655 · databricks/cli

janniklasrose · 2026-09-11T20:17:27Z

What

Extend the CaptureUCDependencies mutator to handle vector search indexes. A vector search index's name is a three-part catalog.schema.index UC identifier. When the catalog or schema is also defined in the same bundle, the mutator now rewrites those segments to ${resources.catalogs.<key>.name} / ${resources.schemas.<key>.name} references, so the deploy engine captures the deploy-time dependency and creates the catalog/schema before the index.

This follows the existing compound-name handling for quality monitors (OutputSchemaName), just with three segments instead of two.

Why

Without this, a bundle that defines a schema and a vector search index living in that schema has no captured ordering dependency, so a deploy can attempt to create the index before its schema exists.

Tests

Unit tests: wired the index into the combined TestCaptureUCDependencies and added TestCaptureUCDependenciesVectorSearchIndexEdgeCases (match / catalog-only / no-match / empty / two-part / nil), plus the nil-resources panic-safety case.
Acceptance test: bundle/resource_deps/implicit_deps_vector_search_index mirrors implicit_deps_quality_monitor; all EnvMatrix variants (terraform/direct × DMS) produce identical output.

This pull request and its description were written by Isaac.

Co-authored-by: Isaac <no-reply@databricks.com>

eng-dev-ecosystem-bot · 2026-09-11T20:40:47Z

Integration test report

Commit: 043f816

Run: 34643605354

	Env	💚RECOVERED	✅pass	🙈skip	Time
💚	aws linux	1	275	16	6:10
💚	aws windows	1	277	14	5:29
💚	azure linux	1	274	16	6:49
💚	azure windows	1	276	14	4:06
💚	gcp linux	1	275	16	6:30
💚	gcp windows	1	277	14	5:34

	Test Name	aws linux	aws windows	azure linux	azure windows	gcp linux	gcp windows
💚	TestAccept	💚R	💚R	💚R	💚R	💚R	💚R

Top 3 slowest tests (at least 2 minutes):

duration	env	testname
4:03	aws windows	TestAccept
3:32	azure windows	TestAccept
3:22	gcp windows	TestAccept

denik · 2026-09-14T08:55:40Z

+			continue
+		}
+		// Name is a three-part "catalog.schema.index" UC identifier.
+		parts := strings.SplitN(idx.Name, ".", 3)


is SplitN use intentional (vs Split)? It would split a.b.c.d into a, b, c.d

Also, how does it work with references? "${resources.b.a}"

is SplitN use intentional (vs Split)?

Following the existing pattern in this file. Also, we only want the first two components (catalog and schema) to call resolveX on them before merging it back together.

how does it work with references? "${resources.b.a}"

references aren't resolved yet, so

Name: ${references.catalogs.my_catalog.name}.my_schema.my_index_name

gets turned into

"${resources", "catalogs", "my_catalog.name}.my_schema.my_index_name"

which passes through untouched (resolveCatalog and resolveSchema no-op). But that's quite implicit and relying on ${resources not being a valid catalog name. We can do better references handling as follow-up for all resources here?

The example above actually is a bit problematic because my_schema won't be resolved as a UC dependency...

I see, the other resources links also have this issue. I agree it nakes sense to fix all uniformly in a follow up.

Can you at least add // TODO describing this problems.

I'll save the extra CI steps for a TODO commit, already working on this fix

denik · 2026-09-14T09:44:30Z

+			continue
+		}
+		// Name is a three-part "catalog.schema.index" UC identifier.
+		parts := strings.SplitN(idx.Name, ".", 3)


I see, the other resources links also have this issue. I agree it nakes sense to fix all uniformly in a follow up.

Can you at least add // TODO describing this problems.

…ra dot-separated components) (databricks#6667) Follow-up to databricks#6655. `CaptureUCDependencies` parsed compound UC names. We now bail early for every resource if a `${resources.*}` reference is still there - users are expected to either declare every UC component via reference or none. Parsing out the components is also strict, requiring the expected number of dot-separated values. --------- Co-authored-by: Isaac <no-reply@databricks.com>

## What Extend the `CaptureUCDependencies` mutator to handle vector search indexes. A vector search index's `name` is a three-part `catalog.schema.index` UC identifier. When the catalog or schema is also defined in the same bundle, the mutator now rewrites those segments to `${resources.catalogs.<key>.name}` / `${resources.schemas.<key>.name}` references, so the deploy engine captures the deploy-time dependency and creates the catalog/schema before the index. This follows the existing compound-name handling for quality monitors (`OutputSchemaName`), just with three segments instead of two. ## Why Without this, a bundle that defines a schema and a vector search index living in that schema has no captured ordering dependency, so a deploy can attempt to create the index before its schema exists. ## Tests - Unit tests: wired the index into the combined `TestCaptureUCDependencies` and added `TestCaptureUCDependenciesVectorSearchIndexEdgeCases` (match / catalog-only / no-match / empty / two-part / nil), plus the nil-resources panic-safety case. - Acceptance test: `bundle/resource_deps/implicit_deps_vector_search_index` mirrors `implicit_deps_quality_monitor`; all EnvMatrix variants (terraform/direct × DMS) produce identical output. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>

…ra dot-separated components) (#6667) Follow-up to #6655. `CaptureUCDependencies` parsed compound UC names. We now bail early for every resource if a `${resources.*}` reference is still there - users are expected to either declare every UC component via reference or none. Parsing out the components is also strict, requiring the expected number of dot-separated values. --------- Co-authored-by: Isaac <no-reply@databricks.com>

## Release v1.17.0 ### Notable Changes * Bump the direct deployment state version to 3. Clients older than v1.8.0 will reject bundles deployed with this release. ([#6713](#6713)) ### CLI * Add an `INVALID_REFRESH_TOKEN` error code to `databricks auth token --output json` failures. ([#6684](#6684)) * Add experimental `databricks auth docker configure` to configure Docker credential helper access for Databricks Artifact Registry. ([#6700](#6700)) * Add experimental `databricks auth docker token` to generate Docker credentials for Databricks Artifact Registry. ([#6699](#6699)) * `databricks environments setup-local` now reports the `E_PROVISION_CONFLICT` error code instead of the generic `E_PROVISION` when `uv sync` fails to resolve a dependency conflict. ([#6666](#6666)) * Preserve SSH sessions across temporary tunnel disconnects, with bounded replay and backpressure for large transfers. ([#6650](#6650)) * Allow OAuth U2M logins to override the CLI client ID with `--client-id`, profile `client_id`, or `DATABRICKS_CLIENT_ID`. ([#6594](#6594)) ### Bundles * direct: Store a dashboard's `serialized_dashboard` in state as a content hash instead of its full contents. ([#6105](#6105)) * direct: Fix pipelines recreation when the whole `ingestion_definition` block is added or removed. ([#6589](#6589)) * `bundle plan`, `deploy`, and `destroy` no longer report removing `permissions`, `grants`, or secret scope ACLs from a bundle as a deletion, since it leaves the resource untouched. ([#6647](#6647)) * `bundle plan` and `deploy` no longer list or count a resource that was already deleted remotely as a deletion, matching `bundle destroy`; applying still cleans up its stale state entry. ([#6675](#6675)) * Fix `bundle run` failing with `expected an int, found a string` when an unrelated resource references another resource that is not deployed. `bundle run` now resolves `${resources.*}` references only within the resource being run. ([#6690](#6690)) * Add grants support for the AI Gateway `model_service`, `mcp_service`, and `model_provider_service` resources (direct engine). ([#6635](#6635)) * Add bundle support for the AI Gateway `mcp_service` resource (direct engine). ([#6633](#6633)) * Add bundle support for the AI Gateway `model_provider_service` resource (direct engine). ([#6634](#6634)) * Add bundle support for the AI Gateway `model_service` resource (direct engine). ([#6525](#6525)) * Prevent resource drift on catalogs if `storage_root` contained a trailing slash in the URL. ([#6622](#6622)) * Fixed a "lineage mismatch in state files" error that could occur after destroying a bundle and redeploying it from another machine. `bundle destroy` now removes the local state file so no stale lineage is left behind, and prunes the state directories it leaves empty (such as `.internal/` and `sync-snapshots/`). ([#6210](#6210), [#6685](#6685)) * direct: `bundle plan` no longer reports a permanent update on a cluster that uses a cluster policy: when the cluster spec sets `policy_id`, a field present in the remote but absent from the bundle config is not treated as drift. ([#6531](#6531)) * `bundle deploy` on the direct engine now reports each resource as soon as it is deployed, instead of listing them all after the deployment finishes. A deploy that fails part way through now reports the resources it did apply. ([#6361](#6361)) * Direct-engine bundles no longer flag phantom drift on server-populated nested fields under reused config types (e.g. `external_locations` file-event-queue resource IDs, `database_instances` parent-instance refs, `apps` git credential ID). ([#6618](#6618)) * `databricks bundle generate app` now reproduces a git-backed app's `git_repository` and `git_source` configuration instead of emitting a workspace `source_code_path`, so generating from a Git-deployed app no longer silently converts it to workspace source. ([#6656](#6656)) * Improved configuration load time for bundles with many included files. ([#6195](#6195)) * `bundle destroy` no longer deletes triggered job runs, leaving them untouched on the backend. ([#6672](#6672)) * direct: resources.job\_runs: new lifecycle.triggers.on\_file\_change setting to restart the run when monitored files change. Can be set to a series of paths or globs. ([#6309](#6309)) * Bundle summary now shows a name for Postgres branches, endpoints, databases, and roles instead of a blank Name field. ([#6663](#6663)) * Added PyDABs (Python) support for cluster policies, dashboards, and Genie spaces. ([#6585](#6585)) * CLI commands no longer imply that a resource whose type has no workspace URL is merely not deployed yet. ([#6583](#6583)) * Capture the implicit dependency a vector search index has on a catalog or schema defined in the same bundle, so the catalog and schema are deployed first. ([#6655](#6655)) ### Dependency Updates * Bump dependencies with known vulnerabilities. ([#6695](#6695)) * Bump `github.com/databricks/databricks-sdk-go` from v0.177.0 to v0.178.0. ([#6673](#6673)) * Bump Terraform provider from v1.131.0 to v1.132.0. ([#6671](#6671))

janniklasrose and others added 2 commits September 11, 2026 22:13

Capture UC dependencies for vector search indexes

66a706a

Add changelog PR link

043f816

Co-authored-by: Isaac <no-reply@databricks.com>

janniklasrose requested review from andrewnester, denik and pietern September 14, 2026 07:25

denik reviewed Sep 14, 2026

View reviewed changes

andrewnester approved these changes Sep 14, 2026

View reviewed changes

denik approved these changes Sep 14, 2026

View reviewed changes

janniklasrose added this pull request to the merge queue Sep 14, 2026

Merged via the queue into main with commit 48a4b06 Sep 14, 2026
45 checks passed

janniklasrose deleted the janniklasrose/vector-search-uc-dependencies branch September 14, 2026 10:08

janniklasrose mentioned this pull request Sep 14, 2026

Be strict about capturing UC dependencies (disallow references or extra dot-separated components) #6667

Merged

Navigation Menu

Uh oh!

Capture UC dependencies for vector search indexes - #6655

Capture UC dependencies for vector search indexes#6655
janniklasrose merged 2 commits into
mainfrom
janniklasrose/vector-search-uc-dependencies

janniklasrose commented Sep 11, 2026 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Sep 11, 2026

Uh oh!

denik Sep 14, 2026

Uh oh!

denik Sep 14, 2026

Uh oh!

janniklasrose Sep 14, 2026 •

edited

Loading

Uh oh!

janniklasrose Sep 14, 2026

Uh oh!

denik Sep 14, 2026

Uh oh!

janniklasrose Sep 14, 2026

Uh oh!

denik Sep 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

janniklasrose commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Tests

Uh oh!

eng-dev-ecosystem-bot commented Sep 11, 2026

Integration test report

Uh oh!

denik Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

denik Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

janniklasrose Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

janniklasrose Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

denik Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

janniklasrose Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

denik Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

janniklasrose commented Sep 11, 2026 •

edited

Loading

janniklasrose Sep 14, 2026 •

edited

Loading