Skip to content

Add direct-mode bundle support for AI Gateway mcp_service - #6633

Queued
alex-khakhlyuk wants to merge 6 commits into
mainfrom
aigov-dabs-mcp-service
Queued

alex-khakhlyuk wants to merge 6 commits into
mainfrom
aigov-dabs-mcp-service

Conversation

@alex-khakhlyuk

Copy link
Copy Markdown
Member

Second of a stacked split of #6410 (databricks/cli), stacked on the model_service PR. Adds the direct-engine bundle resource for the AI Gateway MCP service securable (mcp_services): config/resource + dresources CRUD, testserver, acceptance (basic + remote-delete + bind) and an invariant config, plus registration and regenerated schema. All SDK struct literals set every field to satisfy exhaustruct; mcp_services added to the TF-lifecycle ignore list and the statemgmt full-resource-coverage fixtures. Direct engine only.

Changes

Why

Tests

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 61b4f2c

Run: 34966922933

Env 🟨​KNOWN 🔄​flaky ✅​pass 🙈​skip Time
🟨​ aws linux 3 276 21 6:48
🟨​ aws windows 3 278 19 7:26
🟨​ azure linux 3 275 21 8:33
🟨​ azure windows 3 277 19 6:54
🟨​ gcp linux 3 2 274 21 7:12
🟨​ gcp windows 3 278 19 8:15
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/mcp_services/lifecycle 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/mcp_services/lifecycle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🔄​ TestFilerWorkspaceNotebook ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
🔄​ TestFilerWorkspaceNotebook/pythonJupyterNb.ipynb ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p

Comment thread bundle/direct/dresources/resources.yml Outdated
reason: id_field
- field: mcp_service_id
reason: id_field
ignore_remote_changes:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see #6654

@alex-khakhlyuk
alex-khakhlyuk force-pushed the aigov-dabs-mcp-service branch 3 times, most recently from b0fb58b to 0dbe249 Compare September 14, 2026 19:21

@janniklasrose janniklasrose left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comments can be follow-ups

if ms == nil {
continue
}
ms.Parent = resolveParent(b, ms.Parent)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be small follow-up: resolveAiGatewayParent so the function signature is clear

Comment on lines +42 to +46
>>> [CLI] ai-gateway get-mcp-service mcp-services/main.myschema.myservice
{
"name": "mcp-services/main.myschema.myservice",
"comment": "COMMENT1"
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

printing the whole API response here is going to cause the golden files to drift over time as more fields get added -> let's just assert the fields that you want to ensure the resource got deployed as expected.

I think this is also in model_services so if you could clean that up as a follow-up that'd be great :-)

alex-khakhlyuk and others added 6 commits September 15, 2026 12:01
Second of a stacked split of #6410 (databricks/cli), stacked on the model_service PR. Adds the direct-engine bundle resource for the AI Gateway MCP service securable (mcp_services): config/resource + dresources CRUD, testserver, acceptance (basic + remote-delete + bind) and an invariant config, plus registration and regenerated schema. All SDK struct literals set every field to satisfy exhaustruct; mcp_services added to the TF-lifecycle ignore list and the statemgmt full-resource-coverage fixtures. Direct engine only.

Co-authored-by: Isaac <no-reply@databricks.com>
Adds a Cloud=true lifecycle test for mcp_services, matching model_services
and model_provider_services. An MCP service must point at a pre-existing UC
connection hosting the MCP server (config.source_connection), and there is no
bundle `connections` resource to deploy it as a dependency, so the script
creates a bearer-auth connection out-of-band on cloud (guarded by CLOUD_ENV)
and tears it down via trap. Bearer auth with a placeholder host avoids the
OAuth/DCR reachability probe the backend runs at create time for MCP
connections. The mock testserver doesn't validate source_connection, so the
connection is only created on the real-workspace run.

basic and remote-delete opt out with Cloud=false (they inspect the recorded
request stream / use a fixed out-of-band delete that can't run on cloud).

Co-authored-by: Isaac <no-reply@databricks.com>
Follow-up to the mcp_service support, mirroring #6654 for model_services now
that #6618 enabled it. Added catalog.McpService to apitypes.yml so the
generator reads field behaviors from the response type (config.* paths),
removed the hand-written config.source_connection.is_deleted override from
resources.yml, and regenerated resources.generated.yml. The suppression is
unchanged, just auto-derived from the OpenAPI schema instead of hand-listed.

Co-authored-by: Isaac <no-reply@databricks.com>
…t_tag)

Main upgraded databricks-sdk-go v0.177 -> v0.178, which is the GA-sync SDK that
dropped `owner` (now effective_owner only) and the request_tag rate-limit
feature from the AI Gateway securable types. Adapted the mcp_service resource to
v0.178: removed the stale Owner field from mcpServiceBody, and regenerated the
schema/validation/refschema/pydabs outputs. RATE_LIMIT_KEY_REQUEST_TAG (and the
request_tag_key/value fields) are now absent from mcp_services and model_services
alike, matching the v0.178 spec.

Co-authored-by: Isaac <no-reply@databricks.com>
…test

The earlier cloud test created a metastore-level UC connection out-of-band,
which failed in CI: the deco service principal lacks metastore CREATE
CONNECTION, and under `bash -euo pipefail` the suppressed failure aborted the
script before deploy ("No active deployment found to destroy!").

Switch to a SCHEMA-SCOPED connection (parent schemas/main.default), which needs
only schema-level privileges the deploy SP already has. Schema-level connections
can't set is_mcp_connection (metastore-only), but the mcp_service accepts a
plain HTTP connection as its source_connection (validated end to end against a
staging workspace: create schema-scoped connection -> deploy -> no-drift plan ->
destroy). The reference uses the 3-part form connections/{catalog}.{schema}.{name}.

Also adds temporary debug output around `connections create` (silent on success;
prints the error on failure) so a CI failure surfaces the real cause instead of
being swallowed. Remove once the cloud run is confirmed green.

Co-authored-by: Isaac <no-reply@databricks.com>
…ayout

Main #6679 split the monolithic bundle/direct/dresources/resources.yml and
resources.generated.yml into one file per resource under configs/. Recreate the
mcp_services rules in that layout: configs/mcp_services.yml (provided_id_fields
for the immutable parent + mcp_service_id) and the regenerated
configs/mcp_services.generated.yml (config.source_connection.is_deleted, derived
from catalog.McpService via apitypes.yml). No behavior change; matches how
model_services is expressed on main.

Co-authored-by: Isaac <no-reply@databricks.com>
@alex-khakhlyuk
alex-khakhlyuk added this pull request to the merge queue Sep 15, 2026
Any commits made after this event will not be merged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants