experimental/ssh: improve --help copy and surface serverless flags by TanishqDatabricks · Pull Request #5156 · databricks/cli

TanishqDatabricks · 2026-05-01T13:35:02Z

Summary

databricks ssh: common-workflows examples now lead with serverless (no flags), with the dedicated-cluster path as a follow-up.
databricks ssh connect: Long description groups example invocations by serverless vs dedicated. Unhides --name, --accelerator, --ide, --environment-version so users can discover them via --help.
databricks ssh setup: tightened Short/Long to clarify it's for dedicated clusters; users on serverless should just use connect.

CUJ: https://docs.google.com/document/d/1WnIg2vrnPR98cMay8OTY1zJ4yWcKBAHdZUXi5AhYylU/edit?tab=t.7erhv6e67uw#heading=h.kj99al6d2ch
Tracked at DECO-27026 (under epic DECO-27015 FY27Q2 P0 [IDE-Remote] CLI UX).

Test plan

Run databricks ssh --help and verify serverless examples appear first
Run databricks ssh connect --help and verify --name, --accelerator, --ide, --environment-version are visible (not hidden)
Run databricks ssh setup --help and verify it references dedicated clusters and points users to connect for serverless

This pull request and its description were written by Isaac.

Updates the help text for `databricks ssh`, `databricks ssh connect`, and `databricks ssh setup` based on the May 1 2026 Remote Development CUJ. - `databricks ssh`: common-workflows examples now lead with serverless (no flags), with the dedicated-cluster path as a follow-up. - `databricks ssh connect`: Long description groups example invocations by serverless vs dedicated. Unhides --name, --accelerator, --ide, --environment-version so users can discover them via --help. - `databricks ssh setup`: tightened Short/Long to clarify it's for dedicated clusters; users on serverless should just use connect. Co-authored-by: Isaac

github-actions · 2026-05-01T13:35:30Z

Approval status: pending

`/acceptance/apps/` - needs approval

4 files changed
Suggested: @pkosiec
Also eligible: @MarioCadenas, @arsenyinfo, @fjakobs, @keugenek, @calvarjorge, @jamesbroadhead, @Shridhad, @atilafassina, @igrekun, @pffigueiredo, @ditadi

`/acceptance/auth/` - needs approval

12 files changed
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/acceptance/bundle/` - needs approval

345 files changed
Suggested: @pietern
Also eligible: @denik, @andrewnester, @shreyas-goenka, @anton-107, @janniklasrose, @lennartkats-db

`/acceptance/pipelines/` - needs approval

7 files changed
Suggested: @kanterov
Also eligible: @lennartkats-db, @jefferycheng1

`/bundle/` - needs approval

372 files changed
Suggested: @pietern
Also eligible: @denik, @andrewnester, @shreyas-goenka, @anton-107, @janniklasrose, @lennartkats-db

`/cmd/api/` - needs approval

4 files changed
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/cmd/apps/` - needs approval

11 files changed
Suggested: @pkosiec
Also eligible: @MarioCadenas, @arsenyinfo, @fjakobs, @keugenek, @calvarjorge, @jamesbroadhead, @Shridhad, @atilafassina, @igrekun, @pffigueiredo, @ditadi

`/cmd/auth/` - needs approval

14 files changed
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/cmd/bundle/` - needs approval

9 files changed
Suggested: @pietern
Also eligible: @denik, @andrewnester, @shreyas-goenka, @anton-107, @janniklasrose, @lennartkats-db

`/cmd/configure/` - needs approval

Files: cmd/configure/configure.go
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/cmd/labs/` - needs approval

4 files changed
Suggested: @asnare
Also eligible: @alexott

`/cmd/pipelines/` - needs approval

Files: cmd/pipelines/history_test.go
Suggested: @kanterov
Also eligible: @lennartkats-db, @jefferycheng1

`/cmd/psql/` - needs approval

Files: cmd/psql/psql_autoscaling.go, cmd/psql/psql_provisioned.go
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/cmd/root/` - needs approval

4 files changed
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/cmd/selftest/` - needs approval

7 files changed
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/cmd/sync/` - needs approval

Files: cmd/sync/completion.go
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/cmd/workspace/apps/` - needs approval

Files: cmd/workspace/apps/apps.go
Suggested: @pkosiec
Also eligible: @MarioCadenas, @arsenyinfo, @fjakobs, @keugenek, @calvarjorge, @jamesbroadhead, @Shridhad, @atilafassina, @igrekun, @pffigueiredo, @ditadi

`/experimental/aitools/` - needs approval

11 files changed
Suggested: @pkosiec
Also eligible: @MarioCadenas, @lennartkats-db, @arsenyinfo, @fjakobs, @keugenek, @calvarjorge, @jamesbroadhead, @Shridhad, @atilafassina, @igrekun, @pffigueiredo, @ditadi

`/integration/` - needs approval

14 files changed
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/internal/` - needs approval

7 files changed
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/libs/apps/` - needs approval

8 files changed
Suggested: @pkosiec
Also eligible: @MarioCadenas, @arsenyinfo, @fjakobs, @keugenek, @calvarjorge, @jamesbroadhead, @Shridhad, @atilafassina, @igrekun, @pffigueiredo, @ditadi

`/libs/auth/` - needs approval

10 files changed
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/libs/cmdio/` - needs approval

183 files changed
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/libs/databrickscfg/` - needs approval

12 files changed
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/libs/filer/` - needs approval

Files: libs/filer/workspace_files_client.go, libs/filer/workspace_files_client_test.go
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/libs/flags/` - needs approval

Files: libs/flags/json_flag.go, libs/flags/json_flag_test.go
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/libs/log/` - needs approval

Files: libs/log/handler/colors.go, libs/log/handler/colors_test.go, libs/log/handler/friendly.go
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/libs/sync/` - needs approval

Files: libs/sync/path.go
Suggested: @simonfaltum
Also eligible: @parthban-db, @mihaimitrea-db, @tanmay-db, @renaudhartert-db, @tejaskochar-db, @rauchy, @hectorcast-db, @Divyansh-db, @chrisst

`/libs/template/` - needs approval

8 files changed
Suggested: @pietern
Also eligible: @denik, @andrewnester, @shreyas-goenka, @anton-107, @janniklasrose, @lennartkats-db

General files (require maintainer)

478 files changed
Based on git history:

@pietern -- recent work in bundle/internal/tf/schema/, libs/cmdio/, libs/testserver/

_{Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db) can approve all areas.

See OWNERS for ownership rules.}

…ands Co-authored-by: Isaac

…ting principal (databricks#5151) ## Summary The invariant test config used \`user_name: viewer@example.com\`, which doesn't exist in the cloud workspaces. The Permissions Set API silently drops the unknown user, so a Read after deploy returns an ACL without that entry — the no_drift invariant then sees a phantom update and the test fails on aws-prod-ucws. Pre-existing bug from databricks#4887, not caught earlier because deploy itself was failing on the 50-char endpoint name limit (databricks#5108) before reaching the no_drift check. ### Failure shape (before this fix) \`\`\` "resources.vector_search_endpoints.bar.permissions": { "action": "update", "new_state": { "value": { "__embed__": [ { "level": "CAN_USE", "user_name": "viewer@example.com" }, { "level": "CAN_MANAGE", "service_principal_name": "[USERNAME]" } ] } }, "remote_state": { "__embed__": [ { "level": "CAN_MANAGE", "service_principal_name": "[USERNAME]" } ] }, ... } \`\`\` ### Change Use \`group_name: users\` (always present in every workspace) to match the pattern used by the other \`*_with_permissions\` invariant configs (\`job_with_permissions\`, \`model_with_permissions\`, \`secret_scope_with_permissions\`). ## Test plan - [x] Local: \`go test ./acceptance -run 'TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=vector_search_endpoint'\` passes - [x] Cloud: same target passes on aws-prod-ucws This pull request was AI-assisted by Isaac.

## Changes  When config-remote-sync patches YAML files with remote changes, it now restores variable references rather than always hardcoding values. This prevents configs from losing `${var.X}` references after UI edits. Note: There is a lot of ambiguity in how to restore the variables, because it's not always clear what the user's intent is, and this is hard to describe with a simple heuristic. In this PR, we make a best-effort deterministic attempt as a first step. Supported use cases: 1. If any field that contains a variable was changed, and new values match this variable, we restore it. Safe guard 2. string template `/bundle/${bundle.target}/{var.foo}` case is also supported 3. new list item of the same type is added (job param / job task) -> if variables are used in existing items , and the field value matches the variable, we use this variable ## Why Improve the config-remote-sync experience for customers. We have received feedback that customers usually have their job parameteres defined as variables, and this PR aims to address some gaps there ## Tests  Added acceptance tests + tested integration in the workspace

…atabricks#5168) ## Summary `SelectWarehouse` in `libs/databrickscfg/cfgpickers/warehouses.go` contains: ```go promptui.SearchPrompt = "Search: " ``` This assigns the package-level global to its own default. promptui declares it [here](https://github.com/manifoldco/promptui/blob/v0.9.0/select.go#L184) as `var SearchPrompt = "Search: "` — byte-identical to what we set. The line is the only write to `promptui.SearchPrompt` in the repo. It was introduced in databricks#4170 alongside the template-init warehouse picker. The original warehouse picker (`AskForWarehouse`, added in databricks#956) never had it, which suggests it's copy-paste residue rather than a deliberate override. ## Test plan - [x] `go build ./libs/databrickscfg/cfgpickers/` - [x] No behavior change expected — value matches promptui's default This pull request and its description were written by Isaac.

…databricks#5163) Bumps [github.com/google/jsonschema-go](https://github.com/google/jsonschema-go) from 0.4.2 to 0.4.3. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/google/jsonschema-go/releases">github.com/google/jsonschema-go's releases</a>.</em></p> <blockquote> <h2>v0.4.3</h2> <h2>What's Changed</h2> <ul> <li>improve anyOf errors by <a href="https://github.com/jba"><code>@jba</code></a> in <a href="https://redirect.github.com/google/jsonschema-go/pull/61">google/jsonschema-go#61</a></li> <li>fix: infer - support map with non-string key type by <a href="https://github.com/rafaeljusto"><code>@rafaeljusto</code></a> in <a href="https://redirect.github.com/google/jsonschema-go/pull/70">google/jsonschema-go#70</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/google/jsonschema-go/compare/v0.4.2...0.4.3">https://github.com/google/jsonschema-go/compare/v0.4.2...0.4.3</a></p> <h2>v0.4.3</h2> <h2>What's Changed</h2> <ul> <li>improve anyOf errors by <a href="https://github.com/jba"><code>@jba</code></a> in <a href="https://redirect.github.com/google/jsonschema-go/pull/61">google/jsonschema-go#61</a></li> <li>fix: infer - support map with non-string key type by <a href="https://github.com/rafaeljusto"><code>@rafaeljusto</code></a> in <a href="https://redirect.github.com/google/jsonschema-go/pull/70">google/jsonschema-go#70</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/google/jsonschema-go/compare/v0.4.2...v0.4.3">https://github.com/google/jsonschema-go/compare/v0.4.2...v0.4.3</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/google/jsonschema-go/commit/8c4ab4f02ef64dcea5502e47a6113e8292944087"><code>8c4ab4f</code></a> fix: infer - support map with non-string key type (<a href="https://redirect.github.com/google/jsonschema-go/issues/70">#70</a>)</li> <li><a href="https://github.com/google/jsonschema-go/commit/8bd57428bbbea55d718267fa5b20bbb59b4f9fbd"><code>8bd5742</code></a> improve anyOf errors (<a href="https://redirect.github.com/google/jsonschema-go/issues/61">#61</a>)</li> <li>See full diff in <a href="https://github.com/google/jsonschema-go/compare/v0.4.2...0.4.3">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/google/jsonschema-go&package-manager=go_modules&previous-version=0.4.2&new-version=0.4.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…#5086) Bumps [golang.org/x/text](https://github.com/golang/text) from 0.35.0 to 0.36.0. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/golang/text/commit/8577a70117e110160c45f32af0e0df84eef844f7"><code>8577a70</code></a> go.mod: update golang.org/x dependencies</li> <li>See full diff in <a href="https://github.com/golang/text/compare/v0.35.0...v0.36.0">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…5085) Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.34.0 to 0.35.0. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/golang/mod/commit/03901d351deb5bd95deb90714fb75bf8e232cb22"><code>03901d3</code></a> go.mod: update golang.org/x dependencies</li> <li>See full diff in <a href="https://github.com/golang/mod/compare/v0.34.0...v0.35.0">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…ks#5084) Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.49.0 to 0.50.0. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/golang/crypto/commit/03ca0dcccbd37ba6be80adf74dde8d78a4d72817"><code>03ca0dc</code></a> go.mod: update golang.org/x dependencies</li> <li><a href="https://github.com/golang/crypto/commit/8400f4a938077a7a7817ab7d163d148e371b320b"><code>8400f4a</code></a> ssh: respect signer's algorithm preference in pickSignatureAlgorithm</li> <li><a href="https://github.com/golang/crypto/commit/81c6cb34a8fc386ed53293cd79e3c0c232ee7366"><code>81c6cb3</code></a> ssh: swap cbcMinPaddingSize to cbcMinPacketSize to get encLength</li> <li>See full diff in <a href="https://github.com/golang/crypto/compare/v0.49.0...v0.50.0">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…thub/workflows (databricks#5160) Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 7.0.0 to 7.0.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/upload-artifact/releases">actions/upload-artifact's releases</a>.</em></p> <blockquote> <h2>v7.0.1</h2> <h2>What's Changed</h2> <ul> <li>Update the readme with direct upload details by <a href="https://github.com/danwkennedy"><code>@danwkennedy</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/795">actions/upload-artifact#795</a></li> <li>Readme: bump all the example versions to v7 by <a href="https://github.com/danwkennedy"><code>@danwkennedy</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/796">actions/upload-artifact#796</a></li> <li>Include changes in typespec/ts-http-runtime 0.3.5 by <a href="https://github.com/yacaovsnc"><code>@yacaovsnc</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/797">actions/upload-artifact#797</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/upload-artifact/compare/v7...v7.0.1">https://github.com/actions/upload-artifact/compare/v7...v7.0.1</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/actions/upload-artifact/commit/043fb46d1a93c77aae656e7c1c64a875d1fc6a0a"><code>043fb46</code></a> Merge pull request <a href="https://redirect.github.com/actions/upload-artifact/issues/797">#797</a> from actions/yacaovsnc/update-dependency</li> <li><a href="https://github.com/actions/upload-artifact/commit/634250c1388765ea7ed0f053e636f1f399000b94"><code>634250c</code></a> Include changes in typespec/ts-http-runtime 0.3.5</li> <li><a href="https://github.com/actions/upload-artifact/commit/e454baaac2be505c9450e11b8f3215c6fc023ce8"><code>e454baa</code></a> Readme: bump all the example versions to v7 (<a href="https://redirect.github.com/actions/upload-artifact/issues/796">#796</a>)</li> <li><a href="https://github.com/actions/upload-artifact/commit/74fad66b98a6d799dc004d3353ccd0e6f6b2530e"><code>74fad66</code></a> Update the readme with direct upload details (<a href="https://redirect.github.com/actions/upload-artifact/issues/795">#795</a>)</li> <li>See full diff in <a href="https://github.com/actions/upload-artifact/compare/bbbca2ddaa5d8feaa63e36b76fdaad77386f024f...043fb46d1a93c77aae656e7c1c64a875d1fc6a0a">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/upload-artifact&package-manager=github_actions&previous-version=7.0.0&new-version=7.0.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

## Why `databricks api` always sent the workspace routing identifier (`X-Databricks-Org-Id`) when the profile had one, even when the path was an account API. On unified hosts (one host serving both workspace and account APIs) this misrouted account calls. There was also no way to explicitly route a call to the account API or override the identifier per call. ## Changes Before: routing was decided once from the profile and applied to every call. Now: routing is decided per call from the path being requested. - Paths under `/accounts/{id}/` are auto-detected as account-scope; the routing identifier is dropped. - A small hand-written list in `cmd/api/paths.go` carves out workspace-routed proxy APIs that happen to live under `/accounts/`, so they keep the identifier. - `--account` forces account-scope on a non-`/accounts/` path. - `--workspace-id <id>` overrides the identifier per call. Mutually exclusive with `--account`. - `?o=<id>` on the path (the SPOG URL convention used by the Databricks UI) is recognized as a per-call workspace override, so URLs pasted from the browser route correctly. - The CLI-only `workspace_id = none` sentinel is stripped before the routing decision so the literal "none" never goes on the wire. Routing logic lives in pure functions (`hasAccountSegment`, `extractOrgIDFromQuery`, `resolveOrgID`, `normalizeWorkspaceID`, `isWorkspaceProxyPath`) that take primitives. The cobra `RunE` is a thin adapter that resolves config and calls them. ## Test plan - [x] `go test ./cmd/api` covers the helpers with table-driven cases: deny-list hits and misses, query/fragment edge cases, mutual-exclusion errors, sentinel stripping, `?o=` extraction. - [x] `go test ./acceptance -run TestAccept/cmd/api` exercises seven variants end to end against terraform and direct engines: workspace path, account path, deny-listed proxy under `/accounts/`, `--account`, `--workspace-id`, `?o=` query, `workspace_id = none`. Each test asserts header presence/absence explicitly via `print_requests.py | contains.py`. - [x] `make checks`

## Changes Drop `github.com/nwidger/jsoncolor` and replace `fancyJSON` with a small in-tree colorizer over `json.MarshalIndent` output. Same ANSI palette as before (green strings, cyan numbers, bold-green `true`, red `false`, magenta `null`, bold-blue keys). `defaultRenderer.renderJson` now gates colorization on cmdio TTY/color capabilities; `pretty_json` template helper stays on `!color.NoColor` for parity with the other helpers in `renderFuncMap`. ## Why `fancyJSON` was the last caller of `nwidger/jsoncolor`, and it was the only thing forcing `fatih/color.Color` values across a package boundary. Removing it unblocks a future `fatih/color` migration and replaces the incidental "color is off because fatih's package init saw stdout isn't a TTY" gating with explicit cmdio capability checks. ## Tests - Unit tests in `libs/cmdio/jsoncolor_test.go` cover string/number/literal tokens, escape sequences, key vs value, empty containers, and a round-trip property test (stripping ANSI yields the original bytes). - Manual smoke: `databricks current-user me -o json` on a TTY shows the same colors as before; piped or `NO_COLOR=1` produces plain JSON. _PR description drafted with Claude Code._

## PR Stack 1. **PR 1 (this PR)** — [databricks#5135](databricks#5135) — scaffold + autoscaling targeting + text output 2. [databricks#5136](databricks#5136) — PR 2: provisioned + JSON/CSV streaming + typed values 3. [databricks#5138](databricks#5138) — PR 3: multi-input + multi-statement rejection + error formatting 4. [databricks#5143](databricks#5143) — PR 4: cancellation + timeout + TUI ## Why Talking to Lakebase Postgres from a script today goes through one of two unpleasant paths: 1. **Shell out to `databricks psql -- -c "SQL"`.** Works on macOS / Linux when psql is installed. Fails on Windows 11 by default and on minimal containers / sandboxed CI. No JSON / CSV. 2. **Write Python with `psycopg`.** Forces every consumer to manage OAuth tokens, SSL mode, autocommit, etc. This series adds a third path: a native CLI command that runs SQL against any Lakebase endpoint, returns results in text/JSON/CSV (later PRs), and works without a system psql. `databricks psql` keeps owning the interactive REPL surface; this PR does **not** touch psql. ## Changes **Before:** No CLI command runs SQL against Lakebase from Go. Users either shell out to `psql` (requires the system binary) or write `psycopg` glue. **Now:** `databricks experimental postgres query --target projects/foo/branches/main/endpoints/primary "SELECT 1"` returns a text-rendered result. Provisioned instances and richer output formats land in follow-up PRs. The experimental command is fully contained under `experimental/postgres/cmd/`: - `experimental/postgres/cmd/cmd.go`, `query.go`, `targeting.go`, `connect.go`, `execute.go`, `render.go` — command implementation. - `experimental/postgres/cmd/internal/target/` — Lakebase target resolution helpers (parsing, SDK wrappers, auto-select-when-exactly-one). Internal sub-package so it can't accidentally be imported from outside the experiment. When/if this command graduates from experimental, that's the right time to consider extracting to `libs/`. Single positional SQL, autoscaling targeting only (`--target`, `--project`, `--branch`, `--endpoint`), `--max-retries`, `--connect-timeout`, `--database`. Driver is `github.com/jackc/pgx/v5 v5.9.1` (MIT). Connect retry uses a typed predicate (08xxx SQLSTATE family + `57P03` cannot_connect_now + `net.OpError` with `Op == "dial"`); auth (28xxx) and permission (42501) errors do not retry. Text rendering is buffered (no streaming yet); rows-producing vs command-only is decided at runtime via `FieldDescriptions()`. Outside the experimental tree, this PR only: - Registers the command in `cmd/experimental/experimental.go` (2 lines). - Adds the pgx direct dependency (`go.mod` SPDX annotation, `NOTICE` entry, `NEXT_CHANGELOG.md` dependency-updates entry). `pgx` is already a direct dep of the universe monorepo's Lakebase services; aligning here keeps the SDK surface consistent. ## Test plan - [x] `go test ./experimental/postgres/...` (target parsing, validateTargeting, retry classification, render) - [x] `go test ./internal/build/...` (license + NOTICE completeness) - [x] `go tool ... golangci-lint run ./experimental/postgres/...` (0 issues) - [x] `./task checks` (whitespace, links, deadcode)

…g + types (databricks#5136) ## PR Stack 1. [databricks#5135](databricks#5135) — PR 1: scaffold + autoscaling targeting + text output 2. **PR 2 (this PR)** — [databricks#5136](databricks#5136) — provisioned + JSON/CSV streaming + typed values + `experimental/libs/sqlcli` for output handling 3. [databricks#5138](databricks#5138) — PR 3: multi-input + multi-statement rejection + error formatting 4. [databricks#5143](databricks#5143) — PR 4: cancellation + timeout + TUI Stacked on PR 1. ## Why Two things in this PR. The user-facing one: postgres query learns JSON/CSV streaming and provisioned-instance support. The architectural one: aitools query and postgres query had near-identical output-mode handling (same env var, same flag/env precedence, same threshold). Promote the duplication to a shared `experimental/libs/sqlcli` package before the second consumer ossifies the divergence. ## Changes **Architectural:** `experimental/libs/sqlcli/` is a new package under `experimental/libs/` (not `libs/`) so it inherits the experimental-stability guarantee of its consumers. Exposes: - `sqlcli.EnvOutputFormat`, `sqlcli.StaticTableThreshold` constants. - `sqlcli.Format` typedef + `sqlcli.OutputText/JSON/CSV` consts + `sqlcli.AllFormats`. - `sqlcli.ResolveFormat` — flag > env > default precedence with the explicit-text-on-pipe-is-honoured rule. aitools query migrates to use sqlcli (pure refactor, no behavior change). postgres query was about to add its own copy of all of this; instead it uses sqlcli from day one. **User-facing changes for postgres query:** - `--target my-instance` now resolves a provisioned instance. - `--output json` streams typed values: numbers stay numeric, jsonb stays nested, NaN/Inf/bigints-outside-2^53 become strings. - `--output csv` streams (no buffering). - `DATABRICKS_OUTPUT_FORMAT` honoured. - Auto-fallback to JSON when stdout is piped. - Duplicate column names get deterministic `__N` suffixes with a stderr warning. Also adds `cmdio.IsOutputTTY` (a small public wrapper around the existing private `isTTY`) so commands can ask "is stdout a terminal?" without folding in `NO_COLOR` / `TERM=dumb` (both of which `cmdio.SupportsColor` ANDs in for the colour-rendering decision). ## Test plan - [x] `go test ./experimental/aitools/... ./experimental/postgres/... ./experimental/libs/...` (unit, sinks, value mapping, format selection, aitools tests still pass after migration) - [x] `go tool ... golangci-lint run ./experimental/...` (0 issues)

…atabricks#5138) ## PR Stack 1. [databricks#5135](databricks#5135) — PR 1: scaffold + autoscaling targeting + text output 2. [databricks#5136](databricks#5136) — PR 2: provisioned + JSON/CSV streaming + types + `sqlcli.ResolveFormat` 3. **PR 3 (this PR)** — [databricks#5138](databricks#5138) — multi-input + multi-statement rejection + error formatting + `sqlcli.Collect` 4. [databricks#5143](databricks#5143) — PR 4: cancellation + timeout + TUI Stacked on PR 2. ## Why PR 2 shipped a single-statement, single-input command. Real workflows want multi-input (set-then-query, file-then-stdin), multi-statement rejection with a friendly hint, and rich pg error formatting. This PR also extends `experimental/libs/sqlcli` with input-collection logic shared by aitools and postgres. Same architectural principle as PR 2: instead of postgres growing its own duplicate of aitools' resolveSQLs, both commands now call `sqlcli.Collect`. ## Changes **Architectural:** `experimental/libs/sqlcli/input.go` adds: - `sqlcli.SQLFileExtension` const (.sql). - `sqlcli.Input{SQL, Source}` type — Source is the human-readable origin label ("--file PATH", "argv[N]", "stdin"). - `sqlcli.CollectOptions{Cleaner func(string) string}` — aitools passes its `cleanSQL` (strips comments+quotes); postgres passes the default `TrimSpace` because its multi-statement scanner needs comments preserved. - `sqlcli.Collect` — files-first then positionals, stdin only when neither is present, .sql autodetect on positionals. aitools' resolveSQLs collapses to a thin wrapper around sqlcli.Collect (drops the SQL strings, ignores Source). The "SQL statement #N is empty after removing comments" wording is replaced with sqlcli's `argv[N] is empty`; aitools tests updated. **User-facing changes for postgres query:** - Variadic positionals + repeatable `--file` + stdin fallback. - Multi-statement strings rejected up front with a hint (the hand-written conservative scanner ignores `;` inside string literals, identifiers, line/block comments, and dollar-quoted bodies; tag must be a valid unquoted identifier so `$1` and `$foo-bar$` are correctly NOT treated as tags). - Multi-input output: per-unit blocks for text; canonical-shape JSON array `{"source","sql","kind","elapsed_ms",...}` for json; csv rejected pre-flight when N>1. - Rich pg error formatting (`SEVERITY: message (SQLSTATE XXXXX)` with DETAIL/HINT lines), applied on both single-input and multi-input paths. Single-input keeps streaming. `runUnitBuffered` is a thin wrapper around `executeOne` + a `bufferSink`, so the row-loop and error-wrapping logic stays in one place. ## Test plan - [x] `go test ./experimental/...` (multistatement scanner: 28 cases including dollar-tag punctuation rejection, sqlcli.Collect: 12 cases including a custom-cleaner test, error formatting, multi-input renderers including byte-equal canonical-shape and first-unit-fails framing) - [x] `go tool ... golangci-lint run ./experimental/...` (0 issues)

…abricks#5143) ## PR Stack 1. [databricks#5135](databricks#5135) — PR 1: scaffold + autoscaling targeting + text output 2. [databricks#5136](databricks#5136) — PR 2: provisioned + JSON/CSV streaming + types 3. [databricks#5138](databricks#5138) — PR 3: multi-input + multi-statement rejection + error formatting 4. **PR 4 (this PR)** — [databricks#5143](databricks#5143) — cancellation + timeout + TUI for >30 rows Stacked on PR 3. ## Why PR 3 finished the input ergonomics. The remaining commitments before this command earns the "experimental" label: - A long SELECT shouldn't survive Ctrl+C. Today the pgx default closes the TCP socket but leaves the server-side query running. - CI scripts want `--timeout` so a single statement can't pin a runner. - Interactive users with >30 rows benefit from a scrollable browser instead of a wall of text. ## Changes **Before:** Ctrl+C tears down TCP but the query runs to completion server-side. `--timeout` doesn't exist. >30 rows scroll past in the terminal. **Now:** Ctrl+C cancels the in-flight query at the server. `--timeout 30s` enforces a per-statement deadline. >30 rows on a TTY open the libs/tableview viewer. Specifically: - **Cancellation watcher.** `buildPgxConfig` now installs `CancelRequestContextWatcherHandler` with `CancelRequestDelay=0, DeadlineDelay=5s`. Zero `DeadlineDelay` would race the cancel-request and could leave the connection unusable; 5s gives the cancel round-trip time to land. - **Signal handler.** Per-invocation goroutine watches SIGINT and SIGTERM. Calls cancel; defer'd stop drains the channel. - **--timeout flag.** Per-statement `context.WithTimeout` child of the signal-scoped ctx. `reportCancellation` distinguishes Ctrl+C ("Query cancelled."), timeout ("Query timed out after Xs."), and plain query errors. Returns `(msg, invocationScoped)` so the multi-input loop can drop the source prefix on user-cancel. - **TUI for >30 rows.** `textSink` now has an `interactive` mode; `runQuery` enables it when stdout is a prompt-capable TTY. Static tabwriter table for small results and pipes; libs/tableview for big interactive ones. If `tableview.Run` fails (TUI startup, terminal resize race) the sink falls through to the static tabwriter path so the user still sees the rows. Integration tests aren't included: aitools (the other experimental command) doesn't have them either. Acceptance + unit tests cover argument validation, targeting resolution (SDK-mocked), and output shapes; cancellation/timeout are unit-tested via the seam in `cancel_test.go`. Real-wire integration tests are the right addition when this command graduates from experimental. ## Test plan - [x] `go test ./experimental/postgres/...` (cancel/timeout/signal helpers, race-precedence pinning) - [x] `go test ./acceptance -run TestAccept/cmd/(psql|experimental/postgres)` (no regressions) - [x] `go tool ... golangci-lint run ./experimental/postgres/...` (0 issues) - [x] Manual: Ctrl+C during `SELECT pg_sleep(60)` cancels the server-side query within ~5s. (Smoked on `chatbot-lakebase-dev-simon-faltum` (e2-dogfood): SIGINT to `SELECT pg_sleep(60)` exited the client in 0.53s with `Error: Query cancelled.`; subsequent `pg_stat_activity` query returned zero rows.)

## Summary - Drops `github.com/fatih/color` as a direct dependency by migrating its ~14 call sites (bundle render, bundle run, cfgpickers, logstream, cmd/labs, experimental/aitools, experimental/ssh, python_mutator) to a small ANSI helper set. - Adds `libs/cmdio/color.go` with `cmdio.Red(ctx, msg)`-style helpers and a `RenderFuncMap(ctx)` for templates. The gate matches fatih/color's historical stdout-TTY decision and degrades to plain text when ctx has no cmdIO attached. - Stacks on top of databricks#5170. ANSI constants shared across both colorizers now live in `libs/cmdio/color.go`. No user-visible output changes — the new helpers emit byte-identical SGR sequences. ## Test plan - [ ] Manual smoke: `databricks bundle validate` against a bundle with errors and warnings (colored summary on TTY, uncolored when piped) and `databricks current-user me -o json` (colored on TTY, uncolored when piped through `jq`). This pull request and its description were written by Isaac.

…s#5174) ## Summary - Add `cmdio.SelectOptions`/`RunSelect` and `cmdio.PromptOptions`/`RunPrompt` as a neutral surface around promptui. - Migrate 4 `RunSelect` callers and 6 `Prompt` callers across `cmd/auth`, `cmd/configure`, and `libs/databrickscfg`. After this change, promptui is only imported under `libs/cmdio`, making a future swap of the prompt library a one-package change. - Behavior is unchanged. This pull request and its description were written by Isaac.

…bricks#5106) ## Summary The Workspace files import-file API used to return `Path (<path>) already exists.` for notebook conflicts. The format has changed on some workspaces to `RESOURCE_ALREADY_EXISTS: <path> already exists. ...`. The original regex no longer matched the new format, so `fs.ErrExist` was not returned — breaking `TestImportDirDoesNotOverwrite` and 8 `TestFilerWorkspaceNotebook` subtests on every cloud. The new format might not have been rolled out to all workspaces yet (see [databricks-sdk-go#1639](databricks/databricks-sdk-go#1639)), and the JSON `error_code` is empty in both. Both messages end with `already exists.`, so we anchor on that substring and return the request `absPath` in the error rather than parsing the message. The integration test assertion is updated to expect the path with extension (`tc.name`) instead of without (`tc.nameWithoutExt`), since `absPath` is the request path. ## Test plan - [x] `go test ./libs/filer/...` passes locally - [x] `TestFilerWorkspaceNotebook` passes on aws-prod-ucws (new format) - [ ] Verify a workspace still emitting the old format also passes (best-effort — single substring match handles both) This pull request was AI-assisted by Isaac.

…abricks#5152) ## Summary The `wrapper` and `wrapper_custom_params` tests pin Spark to `12.2.x-scala2.12` to exercise the trampoline workaround for DBR <13.1 ([PR databricks#635](databricks#635)). The AWS test workspaces have disabled legacy access, so 12.2.x is rejected at job submission time: ``` INVALID_PARAMETER_VALUE: Spark version 12.2.x-scala2.12 isn't supported because legacy access is disabled in your workspace. Please use Databricks Runtime 13.3 LTS or above when any legacy features are disabled. ``` Bumping the runtime would defeat the purpose of the test (the trampoline only triggers for DBR <13.1). Restrict the test matrix to Azure where 12.2.x is still bookable. GCP was already excluded for an unrelated DBR release issue. The trampoline itself is covered by unit tests in `bundle/trampoline/` and acceptance tests under `acceptance/bundle/trampoline/`. ## Test plan - [ ] Confirm next nightly: wrapper and wrapper_custom_params no longer in the failures list on aws-prod-is or aws-prod-ucws-is. - [ ] They continue to pass on Azure (azure-prod-is, azure-prod-ucws-is) where they had been passing before. This pull request was AI-assisted by Isaac.

## Summary `AllowEdit` only affects how `promptui` renders a non-empty `Default`: with `AllowEdit:true` the default pre-fills the buffer; with `AllowEdit:false` it appears as a placeholder that's wiped on first keystroke. No caller in the repo sets `Default`, so the field has been a no-op everywhere it was passed. Drop it rather than carry an option that does nothing. ## Test plan - [x] `go build ./...` - [x] `go vet ./cmd/configure/... ./cmd/auth/... ./libs/cmdio/...` - [x] `go test ./libs/cmdio/... ./cmd/configure/... ./cmd/auth/...` This pull request and its description were written by Isaac.

…bricks#5107) ## Summary The Lakebase postgres API now returns `\"default_branch\": \"<project>/branches/production\"` in `ProjectStatus`. Update the testserver mock to populate the field on project create and regenerate the affected acceptance test outputs (basic, recreate, update_display_name) so the tests pass against both the local testserver and a real cloud workspace. This was failing in nightly runs on aws-prod-ucws. ## Test plan - [x] \`go test ./acceptance -run TestAccept/bundle/resources/postgres_projects\` passes locally - [x] Verified all 8 subtests pass on aws-prod-ucws This pull request was AI-assisted by Isaac.

) ## Changes Every time we update a state, we now write resource entry to resources.json.wal. At the end of the deployment, we read those entries, merge them into state and write state file. Internally, the state file can be opened in two modes: read and write. In write mode we only track IDs in memory but not full state. This ensures that when we merge .wal file into state in-memory representation, the .wal file is source of truth, so normal state update is not different from recovery state-update. ## Why If "bundle deploy" is killed, we don't lose state changes. Next time any bundle command runs, we'll recover the updates. ## Tests Modified testserver kill API to be more flexible. Instead of being configured via test.toml, it's a separate endpoint that can add kill middleware on any other endpoint. So you can dynamically insert kill action during script execution. --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Varun Deep Saini <varun.23bcs10048@ms.sst.scaler.com> Signed-off-by: Varun Deep Saini <deepsainivarun@gmail.com> Co-authored-by: Andrew Nester <andrew.nester@databricks.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Varun Deep Saini <varun.23bcs10048@ms.sst.scaler.com> Co-authored-by: Varun Deep Saini <deepsainivarun@gmail.com>

## Why Part of CLI GA. Storing long-lived OAuth refresh tokens for interactive logins (`auth_type = databricks-cli`) in a plain JSON file in the user's home directory is a security weakness: any process with home-directory access can read them. The CLI is increasingly the entry point for local agent workflows, so we want tokens in the OS-native secure store by default. The flip has to handle one big "but": not every system has a usable OS keyring. Linux containers, headless SSH sessions, WSL1, and some CI runners do not have a D-Bus session bus. On those systems the keyring is not merely empty, it is not reachable at all. If we shipped the default flip alone, every command on those systems would error out with a cryptic backend message ("Cannot autolaunch D-Bus without X11 $DISPLAY", etc.) until the user manually set `DATABRICKS_AUTH_STORAGE=plaintext` or re-ran `databricks auth login`. That is a real support burden for the GA window. This PR ships the default flip together with the supporting UX. Three pieces, each scoped narrowly: 1. **Pin-on-success** after a successful login on the default-secure path, so a later transient keyring failure cannot silently demote a working secure-storage user back to plaintext. 2. **Read-path fallback for unavailable keyrings only.** When the keyring backend itself is unreachable (no D-Bus, no Secret Service daemon, etc.), reads fall through to the file cache so pre-upgrade `token-cache.json` entries stay accessible without manual configuration. **This does not fire when the keyring is reachable but empty** — that is the normal post-upgrade case, where we still surface a clear "run `databricks auth login` to sign in" nudge so the user moves their tokens into the secure store rather than silently keeping them in plaintext. 3. **Friendlier `ErrNotFound` messages** that tell the user what to do, with upgrade-specific copy when a legacy `token-cache.json` is present. ## Behavior matrix The distinction between "keyring not available" and "keyring available but empty" drives most of the design: | Scenario | Read-path behavior | | --- | --- | | Keyring reachable, has token for profile | Return token from keyring. | | Keyring reachable, no token for profile | `ErrNotFound` wrapped with "no cached credentials; run `databricks auth login` to sign in" (or upgrade copy if `token-cache.json` exists). User re-authenticates and writes the new token to the keyring. | | Keyring NOT reachable, default-secure mode | Silent fall-back to file cache. Pre-upgrade tokens keep working. **No nudge, no error.** | | Keyring NOT reachable, user explicitly chose secure (env / config) | Return the keyring cache anyway. The actual `Lookup` surfaces the unreachability rather than being silently downgraded against the user's stated intent. | | Keyring probe times out | Stay on the keyring. A locked keyring being unlocked is the common timeout case; misdiagnosing it as "unavailable" would silently route reads to a different backend. | ## Changes Before: tokens were written to `~/.databricks/token-cache.json`. Setting `DATABRICKS_AUTH_STORAGE=secure` opted in to the OS keyring. Now: - Default storage is the OS keyring (Keychain / Credential Manager / Secret Service). Users re-run `databricks auth login` once after upgrade. - `DATABRICKS_AUTH_STORAGE=plaintext` or `[__settings__].auth_storage = plaintext` opts back to the file cache. Env wins over config. - After a successful login on the default-secure path, the CLI writes `auth_storage = secure` to `[__settings__]`. Pins the choice so a later transient probe failure cannot silently demote the user. - Read paths cheap-probe the keyring with a read-only `Get` on a non-existent account. If the backend is unreachable on the default-secure path, the file cache is returned instead. The probe and fall-back are scoped strictly to backend-unavailability, not to empty results. Read-path fallback does NOT pin; pinning stays exclusive to login, which has the stronger write-probe signal and an explicit user action. - `ErrNotFound` from `Lookup` is wrapped with actionable copy: generic case "no cached credentials; run `databricks auth login` to sign in"; upgrade case (mode=secure AND `~/.databricks/token-cache.json` has entries) "stored credentials from older CLI versions are no longer used; run `databricks auth login` to sign in again, or set `DATABRICKS_AUTH_STORAGE=plaintext` to keep using the file cache". - Non-`ErrNotFound` keyring errors get wrapped with the same actionable hint so users on no-keyring systems who somehow bypass the probe (e.g. explicit-secure callers) see "OS keyring unreachable: ... (set `DATABRICKS_AUTH_STORAGE=plaintext` or run `databricks auth login`)" instead of a raw D-Bus message. - Login-time silent fallback (already on `main` as dormant infrastructure) activates and pins. Implementation: - `libs/auth/storage/mode.go`: resolver default flips from `StorageModePlaintext` to `StorageModeSecure`. Constant doc comments updated. - `libs/auth/storage/cache.go`: drops "dormant today" comments. New `PinSecureMode` (login-side pin) and `applyReadFallback` (read-side fallback). `cacheFactories` gains `probeKeyringRead`. `persistPlaintextFallback` now logs internally at debug for shape-consistency with `PinSecureMode`. - `libs/auth/storage/keyring.go`: new `ProbeKeyringRead` (read-only probe). `Lookup` wraps non-`ErrNotFound` errors with the unreachability hint. - `libs/auth/storage/not_found_hint.go` (new): `notFoundHintCache` wraps `ResolveCache` / `ResolveCacheForLogin` so `ErrNotFound` from `Lookup` carries an actionable hint without getting sandwiched between the SDK's `cache:` prefix and `ErrNotFound`'s tail. - `cmd/auth/login.go`, `cmd/auth/token.go`: call `storage.PinSecureMode` after each `persistentAuth.Challenge()`. `login.go` also moves `ResolveCacheForLogin` to run after input validation so trivially-invalid commands no longer probe the keyring. - Unit tests cover all of the above (`PinSecureMode` cases, `applyReadFallback` cases, `ProbeKeyringRead`, `notFoundHintCache`, `legacyCacheHasTokens`). - `acceptance/script.prepare` forces `DATABRICKS_AUTH_STORAGE=plaintext` at the root so existing auth acceptance tests keep exercising the file-backed path. Tests that want the resolver default override it. - `acceptance/cmd/auth/describe/u2m-plaintext-default` renamed to `u2m-secure-default`; its `test.toml` adds a `[[Repls]]` regex normalizing the platform-dependent keyring lookup error. - `acceptance/cmd/auth/describe/u2m-json-output`, `u2m-plaintext-env`, `u2m-plaintext-config`: regenerated to match the new error copy. - `cmd/auth/auth_test.go`: `TestProfileHostCompatibleViaCobra` copies the fixture into a temp directory so the resolver's writes can never dirty the checked-in file. - `NEXT_CHANGELOG.md`: breaking-change entry under Notable Changes covering the flip, the re-login requirement, both opt-out paths, and the read-path fallback for systems without a usable keyring. ## Test plan - [x] `task checks` clean - [x] `task lint-q` clean - [x] `go test ./libs/auth/... ./cmd/auth/... ./libs/databrickscfg/...` passes - [x] `go test ./acceptance -run 'TestAccept/cmd/auth'` passes on macOS - [x] `go test ./acceptance -run 'TestAccept/cmd/configure'` passes (covers a `databricks-cli` auth path outside `cmd/auth`) - [ ] Linux CI is the real test for the `[[Repls]]` regex in `u2m-secure-default/test.toml` (macOS clean miss vs. Linux backend error). - [ ] Manual: with `DATABRICKS_AUTH_STORAGE` unset, `databricks auth login --profile X` writes to the keyring and persists `auth_storage = secure` to `[__settings__]`. - [ ] Manual: `DATABRICKS_AUTH_STORAGE=plaintext databricks auth login --profile X` continues to write to `~/.databricks/token-cache.json` with the host-key dual-write entry; `[__settings__]` is not modified. - [ ] Manual: keyring reachable but empty for the current profile, an auth command produces the "run `databricks auth login` to sign in" nudge (not a silent fall-back). - [ ] Manual: keyring NOT reachable (Linux container, headless SSH), an auth command silently uses the file cache; a populated pre-upgrade `token-cache.json` keeps working. This pull request and its description were written by Isaac.

…cks.com (databricks#5283) ## Why The discovery login flow (`databricks auth login` without `--host`) opens `https://login.databricks.com`. That host is hardcoded in the CLI, so there is no way to point the flow at a non-production login instance during testing or development. The SDK already exposes `u2m.WithDiscoveryHost` (added in databricks-sdk-go databricks#1640, on the CLI's pinned v0.132.0). This PR wires it up. ## Changes **Before:** No way to override the discovery host. `databricks auth login` always opens `https://login.databricks.com`. **Now:** If `DATABRICKS_DISCOVERY_HOST` is set, the CLI passes it through to `u2m.WithDiscoveryHost(...)`. When unset, behavior is identical to before. The "Opening ... in your browser..." log line reflects the override host so it's clear which host is being opened. Intended for testing and development against non-production login instances; unset for normal use. ## Test plan - [x] New unit test `TestDiscoveryLogin_OverridesHostFromEnv` confirms the env var is read and the log message reflects the override host - [x] `go test ./cmd/auth/...` passes - [x] `./task checks` passes - [x] `./task lint-q` passes

) ## Summary Split out from databricks#4917. While that PR keeps responsibility for *moving* the aitools skills-management surface out of `experimental/`, this PR makes the user-facing interface changes that should land at the same moment: - New `--scope=project|global` flag on `install`/`update`/`uninstall`/`list`, with `--scope=both` accepted by `update` and `list`. - `--project` and `--global` are marked deprecated via cobra's `Deprecated` property: hidden from `--help`, emit a stderr deprecation warning when used, continue to function so existing scripts don't break. They're slated for removal in a later release. - `--scope` combined with `--project`/`--global` is rejected up front with an actionable error. - `install`'s `--help` now documents the non-interactive `--agents` auto-detect contract so callers know what gets picked. **Stacked on databricks#4917.** Base will rebase to `main` once that lands. Splitting because (a) databricks#4917 is otherwise a pure file move and reviewers asked to keep it that way, and (b) the interface change has its own product question (boolean pair vs. enum) worth landing as a discrete unit. ## Why land this with the rename aitools is being declared a stable top-level surface in databricks#4917. This is the cheapest moment to fix the two-boolean shape before external scripts depend on it. An enum is also better for agent-driven invocations than two booleans with implicit precedence: `--scope=project|global|both` is one flag with valid values, not two flags with order-dependent semantics. ## Surface ``` databricks aitools install --scope=project|global (--scope=both rejected) databricks aitools uninstall --scope=project|global (--scope=both rejected) databricks aitools update --scope=project|global|both databricks aitools list --scope=project|global|both (default: both) databricks aitools install --project # warns: use --scope=project databricks aitools install --global # warns: use --scope=global ``` ## Test plan - [ ] `databricks aitools install --scope=project` and `--scope=global` go to the right destination - [ ] `databricks aitools install --scope=both` errors with a clear message - [ ] `databricks aitools install --project` still works and prints the deprecation warning to stderr - [ ] `databricks aitools install --scope=global --project` errors with the conflict message - [ ] `databricks aitools list --scope=both` shows both scopes (equivalent to no flag) - [ ] `databricks aitools install --help` no longer shows `--project`/`--global`; `--scope` is documented; `--agents` auto-detect behavior is described - [ ] Unit: `TestParseScopeFlag` (table-driven on the translation), `TestInstallScopeFlag`, `TestListScopeFlag` — all green This pull request was AI-assisted by Isaac. --------- Co-authored-by: simon <4305831+simonfaltum@users.noreply.github.com> Co-authored-by: simon <simon.faltum@databricks.com>

) ## Summary `databricks postgres create-role`'s `--json` flag binds to the inner `Role` object (`CreateRoleRequest.Role`, JSON-tagged `"role"`), so users must supply `spec` / `name` / etc. directly. Without an example this isn't obvious — the auto-generated help leaves the spec fields unflagged (`// TODO: complex arg: spec` in the generator), and the server's error when the body is wrong is vague: ``` Field 'role' is required and must contain at least one subfield with a non-default value ``` That fires whenever the inner `Role` has no recognized fields, which most commonly happens when a user wraps the body in `{"role": ...}` (matching the wire format the SDK marshals to). The CLI strips the unknown outer key with `Warning: unknown field: role` and ships an empty body. Walking out of that loop currently requires reading the SDK source. This adds a curated override (`cmd/workspace/postgres/overrides.go`) that appends a concrete service-principal-role example to `cmd.Long`, plus a short note on the wrapping pitfall. ### Help output (after) ``` Arguments: PARENT: The Branch where this Role is created. Format: projects/{project_id}/branches/{branch_id} Body shape (passed via --json): fields go directly on the Role object. Do not wrap them in '{"role": ...}' — the CLI will strip the unknown outer key and the server will reject the empty body with "Field 'role' is required". Example — create a service-principal-backed role: databricks postgres create-role projects/<PROJECT_ID>/branches/<BRANCH_ID> \ --role-id <SP_CLIENT_ID> \ --json '{"spec": {"identity_type": "SERVICE_PRINCIPAL", "postgres_role": "<SP_CLIENT_ID>", "auth_method": "LAKEBASE_OAUTH_V1", "membership_roles": ["DATABRICKS_SUPERUSER"]}}' ``` ### Scope This PR only touches `create-role`. The same shape gap (`// TODO: complex arg: spec` + opaque error) exists for `create-endpoint`, `create-branch`, `create-project`, and `create-database`. Happy to extend if the approach is right; left them out so reviewers can decide on the pattern first. ## Test plan - [x] `go build ./cmd/workspace/postgres/...` - [x] `databricks postgres create-role --help` shows the new section (output above) - [x] `make fmt` clean - [x] Reproduced the original confusion with a service-principal payload before the change; with this PR the example would have led me straight to the working body shape This pull request and its description were written by Isaac. --------- Co-authored-by: simon <4305831+simonfaltum@users.noreply.github.com> Co-authored-by: simon <simon.faltum@databricks.com>

…tabricks#5287) Follow up to databricks#5149

## Summary The skills manifest in `databricks/databricks-agent-skills` is gaining experimental skills sourced from a new `experimental/` directory in the repo (see paired [d-a-s PR databricks#73](databricks/databricks-agent-skills#73), which imports the ai-dev-kit skill catalog into `experimental/`). This wires the parsing through the aitools installer: - `Manifest.Skills` is a **single map** holding both stable and experimental entries; the per-skill `repo_dir` field ("skills" or "experimental") is the source of truth for whether a skill is experimental. `SkillMeta.IsExperimental()` derives state from `RepoDir`. - Experimental skills get a `-experimental` suffix on their install-side key during `normalizeManifest`; `SourceName` preserves the unsuffixed name for fetch URLs. - The existing `--experimental` flag (already wired in `cmd/skills.go`) now has experimental skills to install; without it, `resolveSkills` filters them out as before. ## UX ``` # default — only stable skills databricks experimental aitools skills install # all experimental skills, plus stable databricks experimental aitools skills install --experimental # one experimental skill by name (--experimental still required by resolveSkills) databricks experimental aitools skills install databricks-iceberg-experimental --experimental ``` ## TODOs / caveats for iteration 1. ~~**`DATABRICKS_SKILLS_REF` pin.**~~ **Partially resolved.** The default ref is still the latest stable release tag (sourced from `experimental/aitools/lib/installer/SKILLS_VERSION`); experimental entries won't exist there until d-a-s cuts a release with [PR databricks#73](databricks/databricks-agent-skills#73) merged. The default ref bump is a follow-up automated by the SKILLS_VERSION file. **UX fix shipped in this PR**: if `--experimental` is passed but the manifest at the resolved ref exposes no experimental skills, a warning is logged pointing users at `DATABRICKS_SKILLS_REF=main`. 2. ~~**Collision handling is naive.**~~ **Resolved.** Every experimental skill gets a `-experimental` suffix on its install-side key during `normalizeManifest`. The manifest key + install dir both carry the suffix; the `SourceName` field on `SkillMeta` preserves the upstream repo dir name for fetch URLs. Users see at a glance which installed skills are experimental. Also handled: **experimental↔stable transitions**. If a skill flips its experimental status upstream (the same logical skill changes manifest key), `install` removes the stale variant on disk + state before installing the new one, and `uninstall` accepts either variant name (and removes both if both are present). Helper: `alternateVariantKey()`. Covered by tests `TestInstallReplacesAlternateVariant`, `TestUninstallByEitherVariantRemovesBoth`, `TestUninstallByAlternateNameWhenOnlyOneVariantInstalled`. 3. ~~**`list` UX.**~~ **Resolved.** `aitools skills list` shows experimental skills with an `[experimental]` tag in the NAME column (driven by `meta.IsExperimental()`). Combined with the TODO databricks#2 resolution (`-experimental` suffix in the manifest key), every experimental row reads e.g. `databricks-iceberg-experimental [experimental]` — slightly redundant but a clear visual anchor. Hide-by-default was considered but rejected: users running `list` are usually looking for what's available, and silently omitting experimental skills makes them un-discoverable. 4. ~~**State tracking.**~~ **Resolved — kept additive semantics.** `InstallState.IncludeExperimental` records what was last requested but is not used to drive retroactive removal. Running `install` without `--experimental` leaves previously-installed experimental skills in place. Rationale: (a) users running `install` are typically adding/updating, not declaring set membership; (b) silently uninstalling things the user previously asked for is surprising; (c) the transition cleanup shipped under TODO databricks#2 handles the actual drift case (skill's experimental status flipping upstream). Removal is what `uninstall` is for. 5. ~~**No acceptance test yet.**~~ **Resolved.** Added acceptance tests under `acceptance/experimental/aitools/skills/install*/` covering the install flow against a mocked manifest server: - Stable-only install (no flag) → 1 skill installed - `--experimental` install adds the experimental skill (with `-experimental` suffix per the install-path model) → 2 skills total - Re-running `--experimental` is idempotent - Specific-skill install (`install --skills <name>`) for both stable and experimental - `--experimental` against a manifest with no experimental entries logs a nudge To make these reachable, exposed a new env-var override `DATABRICKS_SKILLS_BASE_URL` that overrides the hard-coded `raw.githubusercontent.com` base URL used by `GitHubManifestSource.FetchManifest` and `fetchSkillFile`. Defaults to the canonical URL when unset, so no production behavior change. Updated `Taskfile.yml`'s `test-exp-aitools` task to include `acceptance/experimental/aitools/**`. Variants left as follow-up acceptance tests (the structure is now in place): - Variant transition cleanup (stable → experimental, experimental → stable) - Uninstall flow (with both variants installed) 6. ~~**`--experimental` flag scope.**~~ **Resolved — kept current scope.** Each command has internally consistent behavior: - `install --experimental` → explicit opt-in (required to install experimental skills). - `update` → state-driven (honors `InstallState.IncludeExperimental` from the last `install`). If you opted in once, future updates refresh experimentals; otherwise they're skipped. - `list` → shows all skills with an `[experimental]` tag (no filtering — discovery first, opt-in to install). Adding `--experimental` / `--no-experimental` to `update` for one-off overrides was considered but rejected: the natural workflow is to re-run `install --experimental` (or just `install`), which already sets the desired state. Follow-up if real users hit a use case for the override. 7. ~~**Manifest shape.**~~ **Resolved.** Replaced the original two-map design (`skills` + `experimental_skills` + a per-skill `experimental` bool) with a single `skills` map where each entry's `repo_dir` (`"skills"` or `"experimental"`) is the source of truth. The cli derives experimental state from `RepoDir` via `SkillMeta.IsExperimental()`. Collisions between stable and experimental skills with the same repo dir name must be resolved upstream in d-a-s (which they already are — d-a-s PR databricks#73's TODO #1a merged the only known collision into stable). The d-a-s manifest generator should be updated to emit `repo_dir` per skill; until then `normalizeManifest` defaults a missing `RepoDir` to `"skills"` so older manifests still parse. ## Test plan - [x] `go build ./...` passes. - [x] `go test ./experimental/aitools/...` passes (`source_test.go` covers the normalize/IsExperimental cases). - [x] `go test ./acceptance -run TestAccept/experimental/aitools` passes (a pre-existing flake intermittently surfaces an `lstat` warning during copyDir, ~10% of multi-test runs; unrelated to this refactor). - [ ] Run `./task lint` and `./task fmt` before merge. - [ ] Manual: against a d-a-s ref containing experimental entries with `repo_dir`, verify the four UX cases above behave correctly. This pull request and its description were written by Claude. --------- Co-authored-by: simon <4305831+simonfaltum@users.noreply.github.com> Co-authored-by: simon <simon.faltum@databricks.com>

…databricks#5118) ## Summary `databricks workspace import-dir` walks the source tree and copies every entry into the workspace verbatim — it has no awareness of `.gitignore` or default exclusions. This change adds a name-based skip for `.git`, `.databricks`, and `node_modules` directories during the walk. `.gitignore` and other dotfiles at the root remain copied. If a user explicitly passes `.git` (or any of the others) as the source root, that root is still copied — the skip rule applies to entries encountered during recursion. ## Motivation: align `import-dir` with `sync`'s existing defaults `databricks sync` already hard-codes skips for the same two directories that cause the most trouble: - `libs/git/repository.go` — `// Always ignore root .git directory.` adds `.git` to the default ignore rules unconditionally. - `libs/git/view.go` (`SetupDefaults`) — `// Hard code .databricks ignore pattern so that we never sync it (irrespective of .gitignore patterns)`. So `sync` and `import-dir` currently produce different workspace contents for the same source tree: `sync` skips `.git/` and `.databricks/`, `import-dir` copies them. This PR closes that gap for `import-dir` so the two commands behave consistently. `node_modules` is the one entry that goes beyond what `sync` does by default. For any project with a typical `.gitignore`, `sync` would already skip it via gitignore rules; `import-dir` ignores `.gitignore` entirely, so adding it to the name-based skip list keeps the behavior aligned with what users get from `sync`. ## Why this matters in practice `databricks workspace import-dir` is commonly reached for as the inverse of `databricks workspace export-dir`. Without these defaults, the imported tree carries: 1. The local repo's `.git/` directory, including its config and history. 2. The local `.databricks/` bundle cache, which can clobber state that bundle commands maintain remotely. 3. `node_modules/` for JS/TS projects — large, slow to upload, and recreated by the runtime install step anyway. The canonical answer is `databricks sync`, which respects `.gitignore` and already excludes the first two by default. This PR is not a substitute for `sync` — it just brings `import-dir`'s defaults into line for users who reach for it anyway. ## Test plan - [x] Unit tests covering: root `.git/` skipped, nested `.git/` skipped, `.databricks/` skipped, `node_modules/` skipped, `.gitignore` file kept, explicit `.git` root copied (escape hatch). - [x] `go test ./cmd/workspace/workspace/` — pass - [x] `golangci-lint run ./cmd/workspace/workspace/` — clean - [ ] Existing integration `TestImportDir` — unchanged, no `.git` in its testdata so behavior is identical. This pull request and its description were written by Isaac. --------- Co-authored-by: simon <4305831+simonfaltum@users.noreply.github.com> Co-authored-by: simon <simon.faltum@databricks.com>

) > Replaces databricks#4820 — re-opened from an upstream branch (was previously from a fork, which blocked CI from running properly). ## Summary - Replace the opaque `accepts 1 arg(s), received 0` error with a clear message explaining that `APP_NAME` is required - Show usage, mention the `databricks.yml` auto-detect alternative, and suggest an app name inferred from the current directory name ## Test plan - [x] `go test ./cmd/apps/ -run TestMakeArgsOptional` passes - [x] `go build ./cmd/apps/` succeeds This pull request was AI-assisted by Isaac. --------- Signed-off-by: James Broadhead <jamesbroadhead@gmail.com> Co-authored-by: simon <4305831+simonfaltum@users.noreply.github.com> Co-authored-by: simon <simon.faltum@databricks.com>

## Summary `databricks aitools list` learns `--output json`, emitting a structured document so coding agents and CI can consume the skill/version/installation matrix without scraping the tabwriter text output. Text rendering is unchanged. **Stacked on databricks#4917** (uses `--scope` and the moved-to-top-level `aitools/` package). Base will rebase to `main` once databricks#4917 merges. ## JSON shape ```json { "release": "0.1.0", "skills": [ { "name": "databricks-jobs", "latest_version": "1.0.0", "experimental": false, "installed": { "global": "1.0.0", "project": "0.9.0" } } ], "summary": { "global": { "installed": 5, "total": 10 }, "project": { "installed": 3, "total": 10 } } } ``` - `installed` is keyed by scope; absent key = not installed in that scope; empty map = not installed anywhere. - `summary` only includes scopes that were queried, so `--scope=global` narrows it to one key. - `release` is the version string without the `v` prefix. This is the documented public contract — field names and types should not change without a major version bump. ## Why `aitools list` is one of the surfaces an agent reaches for first ("what's installed, what's available, what's stale"). Scraping tabwriter columns from stderr is fragile; a stable JSON contract makes the command declarative for non-human callers. Matches the convention used by other CLI commands that already honor `--output json` (`bundle validate`, `pipelines run`, etc.). ## Test plan - [ ] `databricks aitools list --output json` against a workspace with a mix of installed/uninstalled skills, both scopes — JSON validates against the shape above. - [ ] `databricks aitools list --output json --scope=global` — `summary` only contains `global`. - [ ] `databricks aitools list` (no `--output`) — output is byte-for-byte unchanged from main. - [ ] Unit: `TestRenderListJSON`, `TestRenderListJSONScopeFiltersSummary`, `TestInstalledStatusFromEntry` cover the rendering paths. This pull request was AI-assisted by Isaac.

## Changes New `postgres_synced_tables` resource that syncs a Unity Catalog Delta table into a Postgres table on a Lakebase Autoscaling branch. Supported on both direct and terraform deployment engines. ## Tests Acceptance coverage: `basic` and `recreate` exercise each engine, plus the existing `no_drift` and `migrate` invariants pick up the new resource. Both engines produce identical human-readable output and identical wire bodies. Verified end to end on a live workspace: the bundle deploys a project, lakebase catalog, pipeline-storage schema, and synced table; the pipeline materializes in under a minute; `SELECT` against the destination through the UC federated view returns the rows from the source Delta table; and `bundle destroy` cleans up the full chain. This pull request and its description were written by Isaac.

Adds a Stability Policy section to README covering feature stability, SemVer versioning rules, and security patches; documents extended support for the 0.299.x line in SECURITY.md; and updates NEXT_CHANGELOG.md for the v1.0.0 release.

…icks#5280) ## Why `databricks auth login --profile DEFAULT --host ...` followed by a no-flag `databricks auth describe` (or any other command that needs the U2M token) fails when secure storage is in use: ``` Unable to authenticate: A new access token could not be retrieved because the refresh token is invalid. ``` `databricks auth describe --profile DEFAULT` works. Running the same flow under `DATABRICKS_AUTH_STORAGE=plaintext` also works. So the bug is specific to secure storage + the implicit DEFAULT fallback. Root cause is a cache-key mismatch between login and read: - `cmd/auth/login.go:222` hardcodes `profileName = "DEFAULT"` when no `--profile` is given, so the OAuthArgument's cache key is the literal string `"DEFAULT"`. The token lands in the keyring under account `"DEFAULT"`. - On the read path, `cfg.Profile` starts empty, `resolveDefaultProfile` only consults `[__settings__].default_profile` (so it stays empty), and the SDK's `configFileLoader.Configure` (`config_file.go:103-105`) loads `[DEFAULT]`'s values but **deliberately leaves `cfg.Profile` empty** when it falls back (`isFallback=true`). `CLICredentials.Configure` then builds an OAuthArgument with `profile=""`, so `GetCacheKey()` falls back to `GetHostCacheKey()` and the lookup goes to the host URL, not `"DEFAULT"`. Miss. plaintext mode masks the same mismatch with `DualWritingTokenCache`, which mirrors every write under the host key — so reads via host URL still find the token. secure mode does not dual-write, so the bug surfaces. This is a pre-existing bug independent of toggling secure-storage by default, but doing so turns a corner case into the default experience. The fix here is targeted enough to land standalone. A defense-in-depth followup in `databricks-sdk-go` will drop the SDK-side `if !isFallback` gate so all SDK consumers benefit from the same self-consistency. The CLI fix lands first so secure-storage users are unblocked without waiting on an SDK release cycle. ## Changes - `cmd/root/auth.go`: `resolveDefaultProfile` swaps `databrickscfg.ResolveDefaultProfile` (settings-only) for `databrickscfg.GetDefaultProfile`, which already does the full 4-step resolution: `[__settings__].default_profile` → the only profile in the file → `[DEFAULT]` → empty. The SDK then sees a non-empty `cfg.Profile`, takes the `isFallback=false` branch, and the name flows through to `CLICredentials.Configure`. OAuthArgument's cache key now matches what login wrote. - `cmd/root/bundle.go` is intentionally NOT touched: bundles deliberately limit their fallback to `[__settings__].default_profile` so a hostless bundle does not get silently routed at a `[DEFAULT]` profile pointing at the wrong workspace. That comment in `bundle.go:74-80` stays load-bearing. - `cmd/root/auth_test.go`: - `TestMustWorkspaceClientWithoutConfiguredDefaultFallsBackToDefaultSection` now asserts `cfg.Profile == "DEFAULT"` (was `""`). The previous assertion documented the bug; the new one documents the contract. - New table-driven `TestResolveDefaultProfile` covers the full resolution order: preset `cfg.Profile`, `DATABRICKS_CONFIG_PROFILE` env, `[__settings__].default_profile`, single profile, `[DEFAULT]` section among many, no fallback, missing file. - `NEXT_CHANGELOG.md`: one-line entry describing the fix and the mismatch it removes. ## Test plan - [x] `task checks` clean - [x] `task lint-q` clean - [x] `go test ./cmd/root/... ./cmd/auth/... ./libs/databrickscfg/...` passes - [x] `go test ./acceptance -run 'TestAccept/cmd/auth'` passes - [ ] Manual repro of Pieter's case (`auth login --profile DEFAULT --host ...` then `auth describe` with no flag under secure storage) succeeds after this PR; the same flow on `main` fails. - [ ] Verify bundle resolution is unaffected: a bundle without `workspace.host` and no `--profile` still uses `[__settings__].default_profile` only (no silent DEFAULT routing). This pull request and its description were written by Isaac.

## Release v1.0.0 ### Notable Changes * The Databricks CLI is now generally available with version v1.0.0 as the first major release 🚀. From this version on, the CLI follows semantic versioning (see [README](README.md)). This change does not impact DABs or other existing commands beyond the changes listed below. * The 0.299.x line continues to receive security-critical patches through May 20, 2027; see [SECURITY](SECURITY.md) for the support policy. * Starting with v1.0.0, the CLI will use [immutable release tags](https://docs.github.com/en/code-security/concepts/supply-chain-security/immutable-releases) to increase security against supply chain attacks. * Breaking change: OAuth tokens for interactive logins (`auth_type = databricks-cli`) are now stored in the OS-native secure store by default (Keychain on macOS, Credential Manager on Windows, Secret Service on Linux) instead of `~/.databricks/token-cache.json`. After upgrading, run `databricks auth login` once per profile to re-authenticate; cached tokens from older versions are not migrated. To keep the previous file-backed storage, set `DATABRICKS_AUTH_STORAGE=plaintext` or add `auth_storage = plaintext` under `[__settings__]` in `~/.databrickscfg` (the env var takes precedence over the config setting), then re-run `databricks auth login`. On systems where the OS keyring is not reachable (e.g. Linux containers without a D-Bus session bus), the CLI transparently falls back to the file cache when reading tokens so legacy `token-cache.json` entries remain accessible without manual configuration. ### CLI * Added `databricks aitools` command group for installing Databricks skills into your coding agents (Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity). Skills are fetched from [github.com/databricks/databricks-agent-skills](https://github.com/databricks/databricks-agent-skills) and either symlinked into each agent's skills directory or copied into the current project. Use `databricks aitools install` to set up, `update` to pull newer versions, `list` to see what's available, and `uninstall` to remove them. Pick where they go with `--scope=project|global` (`--scope=both` is accepted on `update` and `list`). * `[__settings__].default_profile` is now consulted as a fallback by `databricks api`, `databricks auth token`, and bundle commands when neither `--profile` nor `DATABRICKS_CONFIG_PROFILE` is set. `databricks auth token` continues to give precedence to `DATABRICKS_HOST` over `default_profile`. For bundle commands, `default_profile` only applies when the bundle does not pin its own `workspace.host`. * Fixed bug where auth commands did not load the DEFAULT profile properly during auth where type is `databricks-cli`. * `databricks workspace import-dir` now skips `.git`, `.databricks`, and `node_modules` directories during recursive imports. To import one of these directories deliberately, pass it as `SOURCE_PATH` ([databricks#5118](databricks#5118)). * `databricks postgres create-role --help` now documents the `--json` body shape and rejects the common mistake of wrapping the body in `{"role": ...}` client-side with a hint pointing at the correct shape ([databricks#5111](databricks#5111)). * `databricks aitools list` honors `--output json`, emitting a structured `{release, skills[...], summary{}}` document so coding agents and CI can consume the skill/version/installation matrix without scraping the tabular text output ([databricks#5233](databricks#5233)). ### Bundles * Make sure warnings asking for approval are understood by agents ([databricks#5239](databricks#5239)) * Support `replace_existing: true` on `postgres_branches` and `postgres_endpoints` so bundles can manage the implicitly-created production branch and primary read-write endpoint of a Lakebase project. * Add `postgres_catalogs` resource to bind a Unity Catalog catalog to a Postgres database on a Lakebase Autoscaling branch ([databricks#5265](databricks#5265)). * Add `postgres_synced_tables` resource to sync a Unity Catalog Delta table into a Postgres table on a Lakebase Autoscaling branch ([databricks#5268](databricks#5268)). * engine/direct: Changes to state file now persisted to .wal file right away instead of being saved in the end ([databricks#5149](databricks#5149))

Enables no-git-provider test on local. Follow up to databricks#5287

…ricks#5294) The `acceptance/bundle/templates/pydabs/test.toml` replacement regex only matched `databricks-bundles==0.x.y`, so after the v1.0.0 release the wheel version `1.0.0` slipped through unredacted and the recorded output (`databricks-bundles==x.y.z`) no longer matched. Broaden the regex to `\d+\.\d+\.\d+` (and escape the dots) so any future version is normalized.

@andrewnester

## Changes Extend `linguist-generated=true` coverage so generated files don't pollute language stats or show up as hand-written code in review. Follows the existing convention of one local `.gitattributes` per generated folder (alongside `bundle/schema/`, `bundle/internal/tf/`, `bundle/internal/validation/generated/`, `acceptance/`, `integration/`). New / updated local files: | Generator | `.gitattributes` location | Pattern | |---|---|---| | `./task generate-direct` | `bundle/direct/dresources/` | `*.generated.yml` | | `./task generate-schema-docs` | `bundle/schema/` (extended) | `jsonschema_for_docs.json` | | `./task generate-schema` | `bundle/internal/schema/` | `annotations_openapi.yml` | | `./task generate-docs` | `bundle/docsgen/output/` | `*.md` | | `mockery` | `internal/mocks/` | `*.go` | | `./task pydabs-codegen` | `python/databricks/bundles/` | `{catalogs,jobs,pipelines,schemas,volumes}/**` | Excluded by request from review: - `internal/genkit/tagging.py{,.lock}` and `.github/workflows/tagging.yml` — kept visible because changes there can affect the release process (@andrewnester). - `acceptance/**/out*` and `acceptance/**/output/**` — kept visible because acceptance-output diffs are how reviewers detect behavior changes (@pietern, @andrewnester). ## Why `bundle/direct/dresources/apitypes.generated.yml` (and `resources.generated.yml`) were showing up as human-authored code on GitHub. The genkit-managed section of the root `.gitattributes` only covers `cmd/account/**` and `cmd/workspace/**`, so the rest of the auto-generated tree was unmarked. This brings linguist's view in line with `.agent/rules/auto-generated-files.md`. ## Tests Verified each pattern with `git check-attr linguist-generated <path>`: - Intended generated files resolve to `true`. - Adjacent manually-maintained files stay `unspecified`: `bundle/direct/dresources/apitypes.yml`, `bundle/internal/schema/annotations.yml`, `bundle/internal/schema/annotations_openapi_overrides.yml`, `bundle/docsgen/{main.go,README.md}`, `bundle/schema/schema.go`, `internal/mocks/README.md`, `python/databricks/bundles/{__init__.py,build.py,core/**}`. - Pre-existing local `.gitattributes` files (TF schema, validation generated, `bundle/schema/jsonschema.json`, acceptance `out.test.toml`) still resolve correctly. `./task ws` passes. _This PR was written by Claude Code._

…atabricks#5293) The backend returns `enable_pg_native_login: false` by default on project creation, causing integration test divergence in `postgres_projects/*`. Match the backend in testserver: set the default to `false` and add it to `ForceSendFields` so the field is emitted explicitly (the real API includes it in the response rather than omitting the zero value). Regenerated acceptance fixtures accordingly. This pull request and its description were written by Isaac.

The Postgres GET API now echoes the leaf id on the Status payload (`BranchStatus.BranchId`, `EndpointStatus.EndpointId`, `ProjectStatus.ProjectId`). Drop the regex-based `ParsePostgresName` recovery in the three `make*Remote` helpers and read the id directly from Status. The component helper and its unit test go with it; the one remaining call site in `all_test.go` now uses the project id literal it just created. Fields added in SDK v0.129.0 (databricks/databricks-sdk-go#1644); available in the CLI since databricks#5237 bumped the SDK to v0.132.0. The testserver was taught to populate these fields in the companion PR databricks#5246, so local acceptance fixtures are unchanged. Stacked on databricks#5273 — base will retarget to `main` once that merges. This pull request and its description were written by Isaac.

…ricks#5270) ## Summary Follow-up to databricks#5214 (per [pietern's review comment](databricks#5214 (comment))). Replace the inline heredocs in `acceptance/auth/bundle_default_profile/script` with three committed templates: - `databricks.yml.no-host.tmpl` — bundle with neither `workspace.host` nor `workspace.profile`. Used for the "default_profile is honored" and "`--profile` overrides" phases. - `databricks.yml.with-host.tmpl` — pins `workspace.host: $DATABRICKS_HOST`. `envsubst` fills in the test server URL before each invocation. - `databricks.yml.with-profile.tmpl` — pins `workspace.profile: other`. Before each phase the script overwrites the active `databricks.yml` from the relevant template. The per-scenario subdirectories (`./bundle-with-host`, `./bundle-with-profile`) and the `cd` calls are gone — the active bundle is always at the test root. Stacked on databricks#5266 (sethome-absolute fix). The base will auto-retarget to `main` once databricks#5266 merges. `output.txt` is unchanged across the refactor: both engine variants (`terraform` and `direct`) produce byte-identical output to the pre-refactor run. ## Test plan - [x] `go test ./acceptance -run TestAccept/auth/bundle_default_profile -v` passes for both `DATABRICKS_BUNDLE_ENGINE=terraform` and `=direct`. - [x] `./task fmt-q` and `./task lint-q` clean. This pull request and its description were written by Isaac.

databricks#5295) ## Changes Update the error emitted by `ValidateDirectOnlyResources` (raised when a direct-only resource type — catalogs, external locations, vector search endpoints — is declared while running with the terraform engine) to mention both ways of switching to the direct engine: > Please set the `DATABRICKS_BUNDLE_ENGINE` environment variable to 'direct' **or set `bundle.engine: direct` in your databricks.yml** to use ... resources. ## Why The previous wording only pointed users at the env var. `bundle.engine` is the equivalent (and usually more durable) configuration knob, so mentioning it makes the error actionable for users who prefer to declare engine choice in `databricks.yml` instead of plumbing an env var through their workflow. ## Tests - `go test ./bundle/config/mutator/ -run TestValidateDirectOnlyResources` — unit tests updated to assert the new message. - `go test ./acceptance -run TestAccept/bundle/validate/catalog_requires_direct_mode` — acceptance snapshot updated. - `./task fmt`, `./task checks`, `./task lint` — clean. _This PR was written by Claude Code._

Fail any acceptance test whose `test.toml` parses to an empty table (zero-byte or comment-only). The check lives in the acceptance test runner's config loader. Deletes 20 pre-existing offenders -- 4 zero-byte placeholders and 16 comment-only files. None of the comments in those files are relevant for the test. This codifies a review comment that shows up from time to time and thereby removes ambiguity/overhead. This pull request and its description were written by Isaac.

## Why The CLI is GA, so the "This project is in Public Preview" line at the top of the README is no longer accurate. ## Changes Before: README opens with a "This project is in Public Preview." line. Now: That line is gone; the README jumps from the build badge straight to the documentation link. ## Test plan - [x] `./task checks` passes

## Changes  ## Why  ## Tests   Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com> Co-authored-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

Bumps `go` to `1.26.0` and `toolchain` to `go1.26.3` across all four modules (root, `tools/`, `tools/task/`, `bundle/internal/tf/codegen/`), and folds in the `golangci-lint --fix` output that the new minor version requires. Bumping the `go` directive to 1.26 unlocks two `modernize` analyzers (already enabled in `.golangci.yaml`) that were silent on 1.25: - `stditerators` — prefer `reflect.Type.Fields()`/`Methods()` and `reflect.Value.Fields()`/`Methods()` over the `NumField()`/`Field(i)` loop pattern. - `newexpr` — replace local `*T` helpers like `func intPtr(v int) *int { return &v }` (and their callers) with Go 1.26's `new(expr)`. These fixes are in the same PR so CI doesn't fail the moment the bump lands. A manual fixup commit removes the redundant `field := field` shadows and the now-dead `*Ptr` helpers the auto-fix left behind (including rewriting 20 `int64Ptr(N)` callers to `new(int64(N))`). Release notes: https://go.dev/doc/go1.26 ## Test plan - `go build ./...` and `go vet ./...` clean on all four modules - `go tool -modfile=tools/go.mod golangci-lint run ./...` — 0 issues - Unit tests pass on all packages touched by the `--fix` and cleanup This pull request and its description were written by Isaac.

Updates `github.com/golangci/golangci-lint/v2` from v2.11.4 to v2.12.2 in `tools/go.mod`. Release notes: https://github.com/golangci/golangci-lint/releases/tag/v2.12.2 Replaces the deprecated `reflect.Ptr` with `reflect.Pointer` across 12 files (auto-fixed by the new govet `inline` analyzer). This pull request and its description were written by Isaac.

## Why Pulls in four releases of the Go SDK (v0.133 through v0.136). ## Changes **Before:** CLI pinned `databricks-sdk-go` at `v0.132.0` and OpenAPI spec at SHA `a499dda0`. **Now:** Pinned at `v0.136.0` / OpenAPI SHA `0555d6a5`. Most of the diff is regenerated output from `./task generate` (CLI command stubs, JSON schema, bundle docs, Python bindings, acceptance goldens). Notable in this batch: - **New `cmd/workspace/bundle/`** — the auto-generated `databricks bundle` workspace commands fronting the DMS service (`CreateDeployment`, `CreateVersion`, `CreateResource`, `CreateOperation`, `Heartbeat`, `CompleteVersion`, etc.). The interface is currently marked `Deprecated` in the SDK; this is a generator artifact, not a real deprecation. - Refreshed pydabs models for new SDK types (`pipelines.Transformer*`, `pipelines.JsonTransformerOptions`, `pipelines.KafkaOptions`, `jobs.PythonOperatorTask*`). Hand-written changes: - **`CurrentUser.Me` signature:** v0.136 changed it from `Me(ctx)` to `Me(ctx, iam.MeRequest{})`. Updated 14 production callsites (`cmd/auth`, `cmd/apps`, `cmd/psql`, `cmd/sync`, `bundle/config/mutator`, `libs/template`, `libs/sync`, `libs/databrickscfg/cfgpickers`, `experimental/postgres`, `experimental/ssh/internal/{client,keys,workspace}`, `integration/internal/acc`, `acceptance/internal/prepare_server.go`, `acceptance/dbr_test.go`) plus 3 mocked test expectations in `cmd/auth/describe_test.go` (`Me(mock.Anything)` -> `Me(mock.Anything, mock.Anything)`). <details><summary>SDK v0.132 -> v0.136 changelog summary</summary> - **v0.133:** WorkspaceAssignmentDetails methods on `a.AccountIamV2` / `w.WorkspaceIamV2`. `jobs.PythonOperatorTask` on `RunTask`/`SubmitTask`/`Task`. `pipelines.ConnectorOptions.KafkaOptions`. Several Postgres / settings additions. [Breaking] `ml.ListFeaturesRequest` argument order changes; `postgres.RequestedResource.UnspecifiedResourceName` removed. - **v0.134:** Jobs `PipelineParams` / `PipelineTask` gained `FullRefreshSelection`, `RefreshFlowSelection`, `RefreshSelection`, `ResetCheckpointSelection`. `settingsv2.Setting` gained `OperationalEmailCustomRecipient` fields. - **v0.135:** **Added `bundle` package** and `w.Bundle` workspace-level service. `ml.AuthConfig.MtlsConfig`. - **v0.136:** Postgres `UndeleteBranch` + branch delete/purge plumbing. `iam.MeRequest.{Attributes,ExcludedAttributes}` (the `CurrentUser.Me` signature change). [Breaking] `bundle.Operation.{ActionType,ResourceId}` and `bundle.Version.CliVersion` made required. [Breaking] `marketplace.ListListingsRequest.Tags` retyped to `ListingTag`. [Breaking] `ClustersAPI.Events` pagination shape changed (genkit absorbed this). </details> ## Test plan - [x] `go build ./...` - [x] `go vet ./...` - [x] `go test ./internal/build ./bundle/internal/schema ./bundle/direct/dresources ./bundle/config/resources ./libs/template ./bundle/config/mutator` - [x] `TestConsistentDatabricksSdkVersion` confirms SDK version and OpenAPI SHA match - [x] Pydabs codegen tests (8 passed) - [ ] CI on this PR --- Replaces databricks#5298 -- moved branch from my fork to upstream so the JFrog/OIDC token exchange in the test setup works (`id-token: write` is denied to fork PRs by GitHub Actions).

## Summary - Update `cli-compat.json` to point CLI version `1.0.0` at AppKit `0.38.0` (from `0.37.0`) ## Test plan - [ ] CI passes (`go test ./libs/clicompat/...`) This pull request and its description were written by Isaac.

…MENTAL] tag Co-authored-by: Isaac

github-actions · 2026-05-22T15:29:51Z

An authorized user can trigger integration tests manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

PR number: 5156
Commit SHA: f1115ee99ff4259c5bf7740943c97a959c1f4b85

Checks will be approved automatically on success.

TanishqDatabricks and others added 28 commits May 22, 2026 17:10

experimental/ssh: refine --help copy for connect, setup, and ssh comm…

9a8a962

…ands Co-authored-by: Isaac

Update connect.go

36d3c86

Update ssh.go

64b4eea

denik and others added 29 commits May 22, 2026 17:27

Update output of TestAccept/bundle/resources/jobs/no-git-provider (da…

fc2640c

…tabricks#5287) Follow up to databricks#5149

Update testserver to validate git_provider (databricks#5288)

0f5afed

Enables no-git-provider test on local. Follow up to databricks#5287

Bump cli-compat to appkit 0.38.0 (databricks#5312)

d12bfd8

## Summary - Update `cli-compat.json` to point CLI version `1.0.0` at AppKit `0.38.0` (from `0.37.0`) ## Test plan - [ ] CI passes (`go test ./libs/clicompat/...`) This pull request and its description were written by Isaac.

experimental/ssh: replace verbose disclaimer with lightweight [EXPERI…

f1115ee

…MENTAL] tag Co-authored-by: Isaac

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

experimental/ssh: improve --help copy and surface serverless flags#5156

experimental/ssh: improve --help copy and surface serverless flags#5156
TanishqDatabricks wants to merge 145 commits into
databricks:mainfrom
TanishqDatabricks:ssh-help-copy

TanishqDatabricks commented May 1, 2026

Uh oh!

github-actions Bot commented May 1, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

19 participants

Conversation

TanishqDatabricks commented May 1, 2026

Summary

Test plan

Uh oh!

github-actions Bot commented May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approval status: pending

/acceptance/apps/ - needs approval

/acceptance/auth/ - needs approval

/acceptance/bundle/ - needs approval

/acceptance/pipelines/ - needs approval

/bundle/ - needs approval

/cmd/api/ - needs approval

/cmd/apps/ - needs approval

/cmd/auth/ - needs approval

/cmd/bundle/ - needs approval

/cmd/configure/ - needs approval

/cmd/labs/ - needs approval

/cmd/pipelines/ - needs approval

/cmd/psql/ - needs approval

/cmd/root/ - needs approval

/cmd/selftest/ - needs approval

/cmd/sync/ - needs approval

/cmd/workspace/apps/ - needs approval

/experimental/aitools/ - needs approval

/integration/ - needs approval

/internal/ - needs approval

/libs/apps/ - needs approval

/libs/auth/ - needs approval

/libs/cmdio/ - needs approval

/libs/databrickscfg/ - needs approval

/libs/filer/ - needs approval

/libs/flags/ - needs approval

/libs/log/ - needs approval

/libs/sync/ - needs approval

/libs/template/ - needs approval

General files (require maintainer)

Uh oh!

github-actions Bot commented May 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

19 participants

github-actions Bot commented May 1, 2026 •

edited

Loading

`/acceptance/apps/` - needs approval

`/acceptance/auth/` - needs approval

`/acceptance/bundle/` - needs approval

`/acceptance/pipelines/` - needs approval

`/bundle/` - needs approval

`/cmd/api/` - needs approval

`/cmd/apps/` - needs approval

`/cmd/auth/` - needs approval

`/cmd/bundle/` - needs approval

`/cmd/configure/` - needs approval

`/cmd/labs/` - needs approval

`/cmd/pipelines/` - needs approval

`/cmd/psql/` - needs approval

`/cmd/root/` - needs approval

`/cmd/selftest/` - needs approval

`/cmd/sync/` - needs approval

`/cmd/workspace/apps/` - needs approval

`/experimental/aitools/` - needs approval

`/integration/` - needs approval

`/internal/` - needs approval

`/libs/apps/` - needs approval

`/libs/auth/` - needs approval

`/libs/cmdio/` - needs approval

`/libs/databrickscfg/` - needs approval

`/libs/filer/` - needs approval

`/libs/flags/` - needs approval

`/libs/log/` - needs approval

`/libs/sync/` - needs approval

`/libs/template/` - needs approval