Add direct-mode bundle support for AI Gateway model_service - #6525
Merged
Merged
Conversation
Collaborator
Integration test reportCommit: 3016db1
Top 14 slowest tests (at least 2 minutes):
|
alex-khakhlyuk
force-pushed
the
aigov-dabs-model-service
branch
5 times, most recently
from
September 10, 2026 11:05
d4a0614 to
da2de5c
Compare
denik
reviewed
Sep 10, 2026
janniklasrose
left a comment
janniklasrose
left a comment
Member
There was a problem hiding this comment.
please add a .nextchanges fragment for this resource
alex-khakhlyuk
force-pushed
the
aigov-dabs-model-service
branch
4 times, most recently
from
September 10, 2026 16:57
fbebe8c to
952d69a
Compare
alex-khakhlyuk
force-pushed
the
aigov-dabs-model-service
branch
from
September 11, 2026 07:51
952d69a to
dcc3c4e
Compare
Adds direct-engine Databricks Asset Bundle support for the Unity Catalog AI Gateway model service securable (model_services), so it can be declared and deployed from databricks.yml. First of a stacked split of #6410 (mcp_services and model_provider_services follow). Direct engine only. - bundle/config/resources/model_service.go: flat ConfigResource exposing the immutable parent + model_service_id identity plus the mutable comment/config body. - bundle/direct/dresources/model_service.go: CRUD onto the AiGateway service; DoRead reconstructs identity from the server-derived name so it participates in drift detection. - resources.yml: parent + model_service_id as immutable id fields; the server-computed output-only config fields (inference_table.table/is_deleted and routing.destinations[*]/fallback.destinations[*].is_deleted) suppressed to avoid phantom drift. - Registration, testserver implementation, regenerated schema, acceptance (basic + remote-delete + bind), and an invariant config. - Wired model_services into the direct-only Terraform-lifecycle ignore list and the statemgmt full-resource-coverage test suite, and fully populated the SDK struct literals so the dresources exhaustruct lint passes. Scope is strictly model_service: no changes to jobs suppression or other resources. Owner is deferred (the API returns effective_owner); permissions use grants; no Catalog Explorer URL is wired up yet. Co-authored-by: Isaac <no-reply@databricks.com>
alex-khakhlyuk
force-pushed
the
aigov-dabs-model-service
branch
from
September 11, 2026 11:37
dcc3c4e to
3016db1
Compare
janniklasrose
approved these changes
Sep 11, 2026
janniklasrose
left a comment
janniklasrose
left a comment
Member
There was a problem hiding this comment.
looking forward to the other AIG resources 
janniklasrose
enabled auto-merge
September 11, 2026 12:41
pranshupand-db
pushed a commit
to pranshupand-db/cli
that referenced
this pull request
Sep 11, 2026
…tabricks#6618) ## What Allow `get_field_behaviors` to exhaustively extract from specs. This discovers `OUTPUT_ONLY` / `INPUT_ONLY` field behaviours. ## Why Follow-up to the discussion on databricks#6525 (databricks#6525 (comment)): the `model_services` resource can't rely on the generator to suppress its output-only config fields, because the destination-config type reused under `config.routing.fallback.destinations[*]` is deduped away. That is a general generator bug, not model_services-specific — it already drops correct suppressions on the three resources above. Fixing it here lets a later `apitypes.yml` pin (`model_services: catalog.ModelService`, riding with databricks#6525) generate the full set instead of hand-classifying it. ## Explanation (There's also inline comments on `resources.generated.yml`) `get_field_behaviors` in `bundle/direct/tools/generate_resources.py` threaded a single global `visited` set through the whole schema walk, deduping by type name across the entire tree. A type reached at two sibling paths was therefore expanded only at the first one, and its nested `OUTPUT_ONLY` / `INPUT_ONLY` annotations were dropped at every later path — surfacing later as phantom drift on those server-populated fields. This scopes `visited` to the current root→node path (each branch gets its own copy via `visited | {ref}`) so a self-referential type still terminates the recursion, but a type reused at sibling paths is expanded at each occurrence. `max_depth` stays as the runaway guard. ## Tests - `go test ./bundle/direct/dresources/` passes, including `TestResourcesYMLNoRedundantRules` (the recovered entries don't collide with hand-written `resources.yml` rules). - Regeneration is idempotent; the script is ruff-clean and formatted with the pinned `ruff@0.15.17`. - No acceptance test references the recovered fields, so there is no golden-output churn. - Simulated databricks#6525 and the `model_services` annotations are picked up This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
sunishsheth2009
pushed a commit
to sunishsheth2009/cli
that referenced
this pull request
Sep 15, 2026
…cks#6654) Follow-up to databricks#6525 after databricks#6618 enabled this. Added `catalog.ModelService` to apitypes.yml, removed the override from resources.yml, and generated the rest
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
Adds direct-engine Databricks Asset Bundle support for the Unity Catalog AI Gateway model service securable (model_services), so it can be declared and deployed from databricks.yml. First of a stacked split of #6410 (mcp_services and model_provider_services follow). Direct engine only. - bundle/config/resources/model_service.go: flat ConfigResource exposing the immutable parent + model_service_id identity plus the mutable comment/config body. - bundle/direct/dresources/model_service.go: CRUD onto the AiGateway service; DoRead reconstructs identity from the server-derived name so it participates in drift detection. - resources.yml: parent + model_service_id as immutable id fields; the server-computed output-only config fields (inference_table.table/is_deleted and routing.destinations[*]/fallback.destinations[*].is_deleted) suppressed to avoid phantom drift. - Registration, testserver implementation, regenerated schema, acceptance (basic + remote-delete + bind), and an invariant config. Scope is strictly model_service: no changes to jobs suppression or other resources. Owner is deferred (the API returns effective_owner); permissions use grants; no Catalog Explorer URL is wired up yet. ## Changes <!-- Brief summary of your changes that is easy to understand --> ## Why <!-- Why are these changes needed? Provide the context that the reviewer might be missing. For example, were there any decisions behind the change that are not reflected in the code itself? --> ## Tests <!-- How have you tested the changes? --> <!-- If your PR needs to be included in the release notes for next release, add a changelog fragment: create .nextchanges/<section>/<name>.md with a one-line description (e.g. .nextchanges/cli/quickstart.md). See .nextchanges/README.md. --> Co-authored-by: Isaac <no-reply@databricks.com>
janniklasrose
added a commit
that referenced
this pull request
Sep 15, 2026
) ## What Allow `get_field_behaviors` to exhaustively extract from specs. This discovers `OUTPUT_ONLY` / `INPUT_ONLY` field behaviours. ## Why Follow-up to the discussion on #6525 (#6525 (comment)): the `model_services` resource can't rely on the generator to suppress its output-only config fields, because the destination-config type reused under `config.routing.fallback.destinations[*]` is deduped away. That is a general generator bug, not model_services-specific — it already drops correct suppressions on the three resources above. Fixing it here lets a later `apitypes.yml` pin (`model_services: catalog.ModelService`, riding with #6525) generate the full set instead of hand-classifying it. ## Explanation (There's also inline comments on `resources.generated.yml`) `get_field_behaviors` in `bundle/direct/tools/generate_resources.py` threaded a single global `visited` set through the whole schema walk, deduping by type name across the entire tree. A type reached at two sibling paths was therefore expanded only at the first one, and its nested `OUTPUT_ONLY` / `INPUT_ONLY` annotations were dropped at every later path — surfacing later as phantom drift on those server-populated fields. This scopes `visited` to the current root→node path (each branch gets its own copy via `visited | {ref}`) so a self-referential type still terminates the recursion, but a type reused at sibling paths is expanded at each occurrence. `max_depth` stays as the runaway guard. ## Tests - `go test ./bundle/direct/dresources/` passes, including `TestResourcesYMLNoRedundantRules` (the recovered entries don't collide with hand-written `resources.yml` rules). - Regeneration is idempotent; the script is ruff-clean and formatted with the pinned `ruff@0.15.17`. - No acceptance test references the recovered fields, so there is no golden-output churn. - Simulated #6525 and the `model_services` annotations are picked up This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
janniklasrose
added a commit
that referenced
this pull request
Sep 15, 2026
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Sep 16, 2026
## Release v1.17.0 ### Notable Changes * Bump the direct deployment state version to 3. Clients older than v1.8.0 will reject bundles deployed with this release. ([#6713](#6713)) ### CLI * Add an `INVALID_REFRESH_TOKEN` error code to `databricks auth token --output json` failures. ([#6684](#6684)) * Add experimental `databricks auth docker configure` to configure Docker credential helper access for Databricks Artifact Registry. ([#6700](#6700)) * Add experimental `databricks auth docker token` to generate Docker credentials for Databricks Artifact Registry. ([#6699](#6699)) * `databricks environments setup-local` now reports the `E_PROVISION_CONFLICT` error code instead of the generic `E_PROVISION` when `uv sync` fails to resolve a dependency conflict. ([#6666](#6666)) * Preserve SSH sessions across temporary tunnel disconnects, with bounded replay and backpressure for large transfers. ([#6650](#6650)) * Allow OAuth U2M logins to override the CLI client ID with `--client-id`, profile `client_id`, or `DATABRICKS_CLIENT_ID`. ([#6594](#6594)) ### Bundles * direct: Store a dashboard's `serialized_dashboard` in state as a content hash instead of its full contents. ([#6105](#6105)) * direct: Fix pipelines recreation when the whole `ingestion_definition` block is added or removed. ([#6589](#6589)) * `bundle plan`, `deploy`, and `destroy` no longer report removing `permissions`, `grants`, or secret scope ACLs from a bundle as a deletion, since it leaves the resource untouched. ([#6647](#6647)) * `bundle plan` and `deploy` no longer list or count a resource that was already deleted remotely as a deletion, matching `bundle destroy`; applying still cleans up its stale state entry. ([#6675](#6675)) * Fix `bundle run` failing with `expected an int, found a string` when an unrelated resource references another resource that is not deployed. `bundle run` now resolves `${resources.*}` references only within the resource being run. ([#6690](#6690)) * Add grants support for the AI Gateway `model_service`, `mcp_service`, and `model_provider_service` resources (direct engine). ([#6635](#6635)) * Add bundle support for the AI Gateway `mcp_service` resource (direct engine). ([#6633](#6633)) * Add bundle support for the AI Gateway `model_provider_service` resource (direct engine). ([#6634](#6634)) * Add bundle support for the AI Gateway `model_service` resource (direct engine). ([#6525](#6525)) * Prevent resource drift on catalogs if `storage_root` contained a trailing slash in the URL. ([#6622](#6622)) * Fixed a "lineage mismatch in state files" error that could occur after destroying a bundle and redeploying it from another machine. `bundle destroy` now removes the local state file so no stale lineage is left behind, and prunes the state directories it leaves empty (such as `.internal/` and `sync-snapshots/`). ([#6210](#6210), [#6685](#6685)) * direct: `bundle plan` no longer reports a permanent update on a cluster that uses a cluster policy: when the cluster spec sets `policy_id`, a field present in the remote but absent from the bundle config is not treated as drift. ([#6531](#6531)) * `bundle deploy` on the direct engine now reports each resource as soon as it is deployed, instead of listing them all after the deployment finishes. A deploy that fails part way through now reports the resources it did apply. ([#6361](#6361)) * Direct-engine bundles no longer flag phantom drift on server-populated nested fields under reused config types (e.g. `external_locations` file-event-queue resource IDs, `database_instances` parent-instance refs, `apps` git credential ID). ([#6618](#6618)) * `databricks bundle generate app` now reproduces a git-backed app's `git_repository` and `git_source` configuration instead of emitting a workspace `source_code_path`, so generating from a Git-deployed app no longer silently converts it to workspace source. ([#6656](#6656)) * Improved configuration load time for bundles with many included files. ([#6195](#6195)) * `bundle destroy` no longer deletes triggered job runs, leaving them untouched on the backend. ([#6672](#6672)) * direct: resources.job\_runs: new lifecycle.triggers.on\_file\_change setting to restart the run when monitored files change. Can be set to a series of paths or globs. ([#6309](#6309)) * Bundle summary now shows a name for Postgres branches, endpoints, databases, and roles instead of a blank Name field. ([#6663](#6663)) * Added PyDABs (Python) support for cluster policies, dashboards, and Genie spaces. ([#6585](#6585)) * CLI commands no longer imply that a resource whose type has no workspace URL is merely not deployed yet. ([#6583](#6583)) * Capture the implicit dependency a vector search index has on a catalog or schema defined in the same bundle, so the catalog and schema are deployed first. ([#6655](#6655)) ### Dependency Updates * Bump dependencies with known vulnerabilities. ([#6695](#6695)) * Bump `github.com/databricks/databricks-sdk-go` from v0.177.0 to v0.178.0. ([#6673](#6673)) * Bump Terraform provider from v1.131.0 to v1.132.0. ([#6671](#6671))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds direct-engine Databricks Asset Bundle support for the Unity Catalog AI Gateway model service securable (model_services), so it can be declared and deployed from databricks.yml. First of a stacked split of #6410 (mcp_services and model_provider_services follow). Direct engine only.
Scope is strictly model_service: no changes to jobs suppression or other resources. Owner is deferred (the API returns effective_owner); permissions use grants; no Catalog Explorer URL is wired up yet.
Changes
Why
Tests