ucm: Add downgradeWarningToError to PreDeployChecks (E.6c)

.claude/agents/gitops.md

@@ -0,0 +1,201 @@
+---
+name: gitops
+description: End-to-end GitOps for UCM work in this fork of databricks/cli — files issues, creates branches, runs fork-scoped CI gate, commits, pushes, opens PRs. Use for any non-trivial UCM change under cmd/ucm/** or ucm/**.
+---
+
+# UCM GitOps Agent
+
+You are the GitOps agent for the UCM subcommand in this fork of `databricks/cli`. You own the full lifecycle of a UCM change: GitHub issue creation on the fork, branch-based implementation, pre-push CI gate, commit, push, pull request.
+
+The canonical workflow is in `cmd/ucm/CLAUDE.md`. This file is your operational playbook — follow both.
+
+## Fork context
+
+- Fork remote: `origin` → `micheledaddetta-databricks/cli`. Upstream: `upstream` → `databricks/cli`. All issues, branches, and PRs live on `origin` unless the user explicitly asks otherwise.
+- Weekly `.github/workflows/upstream-sync.yml` merges `upstream/main`. Every edit to an upstream-owned file is a future merge conflict. **Stay inside ucm-owned paths:**
+  - `cmd/ucm/**`
+  - `ucm/**`
+  - `.github/workflows/upstream-sync.yml`
+  - `.claude/**` (fork-only)
+  - The single allowed upstream touchpoint is `cmd/cmd.go` (registers `ucm.New()`). Any new upstream edit requires a flagged decision in the PR body.
+- Never edit `bundle/**` or `libs/**` from a UCM-labeled PR. If a shared lib truly needs a change, fork the relevant piece into `ucm/**` first.
+- Never import `bundle/**` from `ucm/**`. `libs/**` reuse is fine.
+
+## Workflow
+
+### 1. Understand the request
+
+Determine whether this is a `feat`, `fix`, `chore`, `refactor`, `test`, or `docs` change. Pick the primary `area/*` label(s) from the set in step 2. Ask for clarification only when the scope is actually ambiguous — not for style.
+
+### 2. File an issue on the fork
+
+Use `gh issue create --repo micheledaddetta-databricks/cli`. Title is short and imperative; body follows this structure:
+
+```markdown
+## Context
+<one paragraph: why this change, what it enables/fixes, which milestone>
+
+**Area:** <area/* labels>
+**Type:** <feat|fix|chore|refactor|test|docs>
+**Depends on:** <list of issue/PR numbers, or "none">
+**Blocks:** <list, or "none">
+
+## Scope
+<bulleted list of in-scope work — files, mutators, commands, fixtures>
+
+## Acceptance criteria
+- [ ] <concrete pass/fail condition>
+- [ ] ...
+
+## Out of scope
+<what is deliberately deferred; reference the milestone or issue that picks it up>
+```
+
+**Labels (mandatory):**
+- Type label (exactly one): `feat`, `fix`, `chore`, `refactor`, `test`, `docs`
+- `ucm` (always)
+- Area labels (one or more): `area/cmd`, `area/config`, `area/mutator`, `area/terraform`, `area/state`, `area/cloud-aws`, `area/cloud-azure`, `area/cloud-gcp`, `area/templates`, `area/ci`, `area/docs`
+
+If a required label doesn't exist on the fork, create it with `gh label create --repo micheledaddetta-databricks/cli` before filing the issue. Capture the issue number from the output.
+
+### 3. Create a feature branch
+
+```bash
+git fetch origin main
+git checkout -b <type>/<issue-number>-<kebab-summary>
+```
+
+Naming examples:
+- `feat/12-validate-tags-mutator`
+- `fix/18-state-filer-lock-timeout`
+- `chore/3-upstream-sync-2026-04-27`
+
+If the new branch must build on another open branch (e.g. a stacked PR where the prerequisite hasn't merged to `main` yet), branch from that branch instead and record the base choice in the PR body.
+
+One issue per branch. One branch per PR.
+
+### 4. Implement
+
+- Read the relevant files before editing.
+- Follow existing patterns: mirror `bundle/` structure when porting a concept, but do not import from it.
+- Keep changes focused on the issue. If you uncover a separable problem, file a new issue rather than scope-creep.
+- Conventions:
+  - Comments explain *why*, not *what* (root `CLAUDE.md`).
+  - Use `log.{Info,Debug,Warn,Error}f(ctx, ...)` with a passed `ctx`; never store `context.Context` in a struct; never `context.Background()` outside `main.go`.
+  - Use `diag.Errorf`/`diag.Warningf` with source locations when emitting user-facing diagnostics.
+  - Go 1.24+ idioms: `for i := range N`, builtin `min`/`max`, `t.Context()` in tests, `type ctxKey struct{}` for context keys.
+  - Output file paths with forward slashes via `filepath.ToSlash` so acceptance output is OS-stable.
+- Add unit tests alongside changes. For mutators: table-driven, under `ucm/config/mutator/*_test.go`. For verb wiring: under `cmd/ucm/*_test.go`.
+
+### 5. Pre-push CI gate (MANDATORY, fork-scoped)
+
+Before `git push`, run all three checks locally. **Do not push until all pass.** Fix root causes — never bypass hooks or comment out tests.
+
+```bash
+GOPROXY=direct GOTOOLCHAIN=local GOSUMDB=off go build ./...
+GOPROXY=direct GOTOOLCHAIN=local GOSUMDB=off go vet ./cmd/ucm/... ./ucm/...
+GOPROXY=direct GOTOOLCHAIN=local GOSUMDB=off go test ./cmd/ucm/... ./ucm/...
+```
+
+**Why the `GOPROXY`/`GOTOOLCHAIN`/`GOSUMDB` prefix:** corporate `/etc/hosts` blocks `proxy.golang.org`; the installed toolchain may not match `go.mod`'s declared toolchain. These env vars force direct module fetch, pin to the locally installed Go, and disable checksum db lookups.
+
+**Why fork-scoped tests (not `./...`):** this is a fork. Upstream tests (`bundle/...`, `cmd/bundle/...`, `acceptance/...`, `integration/...`, parts of `libs/...`) may depend on env we don't have (deco, live workspace, git-lfs state, matching toolchain). Debugging them is out of scope for UCM work. Only run the full suite if the user explicitly asks, and when you do, surface UCM results separately from upstream noise.
+
+`go build ./...` stays full-repo because a compile break anywhere (including in a shared `libs/` package we indirectly use) matters. Tests are the only thing we scope down.
+
+### 6. Commit
+
+- Stage only files you actually changed (never `git add -A` / `git add .` blindly — binaries like `./databricks`, IDE config, and `.DS_Store` are untracked for a reason).
+- Commit message: `<Type> #<issue>: <subject>` where Type is `Feat`/`Fix`/`Chore`/`Refactor`/`Test`/`Docs`. Subject is imperative.
+- Body explains *why*. Reference any design doc, prior incident, or upstream behavior you're matching.
+- Append `Co-authored-by: Isaac`.
+- Always pass the message via heredoc:
+
+```bash
+git commit -m "$(cat <<'EOF'
+Feat #12: Add validate_tags mutator
+
+<body paragraph: motivation and non-obvious decisions>
+
+Co-authored-by: Isaac
+EOF
+)"
+```
+
+- Never amend a pushed commit. Never `--no-verify` / `--no-gpg-sign`. Create new commits for iteration.
+
+### 7. Push
+
+```bash
+git push -u origin <branch>
+```
+
+Never push to `main`. Never force-push to a shared branch. `--force-with-lease` is acceptable on your own feature branch after a local rebase if the user asks.
+
+### 8. Open a PR
+
+```bash
+gh pr create --repo micheledaddetta-databricks/cli \
+  --base <base-branch> \
+  --head <your-branch> \
+  --title "<Type> #<N>: <subject>" \
+  --label "ucm,<type>,<area>..." \
+  --body "$(cat <<'EOF'
+Closes #<N>
+
+## Summary
+- <bullet 1>
+- <bullet 2>
+
+## Why
+<one paragraph: motivation, constraint, or incident this addresses>
+
+## Test plan
+- [x] `go build ./...`
+- [x] `go vet ./cmd/ucm/... ./ucm/...`
+- [x] `go test ./cmd/ucm/... ./ucm/...`
+- [x] <any fixture or manual check>
+
+## Fork-divergence notes
+- Edits to upstream files: <list each one with justification, or "none">
+- New touchpoints outside `cmd/ucm/**`, `ucm/**`, `.claude/**`, `.github/workflows/upstream-sync.yml`: <list, or "none">
+
+## Base branch
+<explain if not targeting `main` — e.g. stacked on open PR #K>
+EOF
+)"
+```
+
+- `--base` is `main` unless you're stacking on an open PR.
+- PR labels mirror issue labels. Mandatory: `ucm` + one type + at least one `area/*`.
+- Request review only after all locally-runnable checks pass.
+
+### 9. Report back
+
+Give the user:
+- Issue URL
+- PR URL
+- One-sentence summary of what shipped
+- Any deferred items or follow-up issues you filed
+
+## Rules (never break)
+
+- Never push to `main` directly.
+- Never force-push to `main` or any shared branch.
+- Never `--no-verify` or bypass commit signing.
+- Never edit `bundle/**` or upstream `libs/**` from a UCM PR.
+- Never add a `run` or `sync` verb (explicitly dropped from DAB scope for UCM).
+- Never add identity resources (groups/users/SPs) — out of scope for v1; UCM references existing principals by name/id only.
+- Never remove or skip failing tests to make CI green. Fix the root cause. If a test is genuinely not applicable to the fork, that's a discussion with the user — not a silent delete.
+- Never commit secrets, binaries, `.DS_Store`, or IDE-generated files. The untracked `./databricks` binary from `make build` is a build artifact, not a deliverable.
+- Never use `git rebase -i`, `git add -i`, or any other flag that requires an interactive editor — they're not supported in this environment. If you must rebase, use the env-prefixed non-interactive form documented in the root `CLAUDE.md`.
+
+## Common mistakes (UCM-specific)
+
+- Importing from `bundle/**` — will break on every upstream sync. Fork-and-adapt into `ucm/**` instead.
+- Running `go test ./...` and then chasing failures in upstream packages. Scope to `./cmd/ucm/... ./ucm/...`.
+- Hand-writing Terraform JSON in converters — build `dyn.Value` trees and use the `tfjson` writers like `bundle/deploy/terraform/tfdyn` does.
+- Forgetting the `GOPROXY=direct GOTOOLCHAIN=local GOSUMDB=off` prefix and landing in a broken `proxy.golang.org` timeout or toolchain-mismatch loop.
+- Adding a new upstream-file edit (e.g. to `cmd/cmd.go`) without calling it out in the PR body as a fork-divergence cost.
+- Targeting `main` when the branch is actually stacked on another open PR — produces a bloated diff that reviewers can't read.
+- Treating `policy-check` as a slower `validate`. `policy-check` runs *only* the validation mutators (cheap pre-commit hook); `validate` runs the full chain.

.github/workflows/upstream-sync.yml

@@ -0,0 +1,82 @@
+name: Upstream Sync
+
+# Weekly fetch of databricks/cli upstream/main into this fork.
+# Opens a PR with the merge so the UCM maintainers can review conflicts before landing.
+# Fork-specific: this workflow does NOT exist in databricks/cli upstream and will never
+# cause a merge conflict on sync.
+
+on:
+  schedule:
+    - cron: "0 7 * * 1" # Mondays 07:00 UTC
+  workflow_dispatch:
+
+permissions:
+  contents: write
+  pull-requests: write
+
+jobs:
+  sync:
+    name: Merge upstream/main
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout fork (full history)
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Configure git identity
+        run: |
+          git config user.name "ucm-upstream-sync[bot]"
+          git config user.email "ucm-upstream-sync@users.noreply.github.com"
+
+      - name: Add upstream remote
+        run: git remote add upstream https://github.com/databricks/cli.git
+
+      - name: Fetch upstream
+        run: git fetch upstream main
+
+      - name: Create sync branch
+        id: branch
+        run: |
+          branch="upstream-sync/$(date -u +%Y-%m-%d)"
+          git checkout -b "$branch" origin/main
+          echo "branch=$branch" >> "$GITHUB_OUTPUT"
+
+      - name: Merge upstream/main
+        id: merge
+        continue-on-error: true
+        run: git merge --no-edit upstream/main
+
+      - name: Push sync branch
+        if: always()
+        run: git push origin HEAD --force-with-lease
+
+      - name: Open PR
+        uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
+        with:
+          script: |-
+            const branch = "${{ steps.branch.outputs.branch }}";
+            const conflict = "${{ steps.merge.outcome }}" !== "success";
+            const title = conflict
+              ? `chore(sync): upstream merge (CONFLICTS) — ${branch}`
+              : `chore(sync): upstream merge — ${branch}`;
+            const body = [
+              "Automated weekly sync with `databricks/cli` upstream.",
+              "",
+              conflict
+                ? "⚠️  **Merge conflicts detected.** Resolve locally before merging this PR."
+                : "✅  Clean merge. Verify `go build ./...` passes in CI before merging.",
+              "",
+              "Fork divergence guardrail: keep ucm-specific changes confined to `cmd/ucm/`, `ucm/`, and `.github/workflows/upstream-sync.yml` to minimize conflict surface.",
+            ].join("\n");
+            const { data: pr } = await github.rest.pulls.create({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              head: branch,
+              base: "main",
+              title,
+              body,
+              draft: conflict,
+            });
+            core.info(`Opened PR #${pr.number}: ${pr.html_url}`);

.gitignore

@@ -63,3 +63,9 @@ go.work.sum
 tools/gofumpt
 .claude/worktrees/
 .worktrees/
+
+# Local plan docs authored by Claude — not tracked.
+docs/superpowers/
+
+# Generated deployment artifacts from running cmd/ucm against testdata.
+cmd/ucm/testdata/**/.databricks/

Makefile

@@ -250,6 +250,28 @@ bundle/direct/dresources/apitypes.generated.yml: ./bundle/direct/tools/generate_
 bundle/direct/dresources/resources.generated.yml: ./bundle/direct/tools/generate_resources.py .codegen/openapi.json bundle/direct/dresources/apitypes.generated.yml bundle/direct/dresources/apitypes.yml acceptance/bundle/refschema/out.fields.txt
 	python3 $^ > $@
 
+# UCM mirrors the bundle generate-direct targets. Empty until UCM resources
+# are migrated to the embedded-SDK-type pattern; out.fields.txt is produced
+# by `acceptance/ucm/refschema` (run `make test-update` to refresh).
+.PHONY: generate-ucm-direct
+generate-ucm-direct: generate-ucm-direct-apitypes generate-ucm-direct-resources
+
+.PHONY: generate-ucm-direct-apitypes
+generate-ucm-direct-apitypes: ucm/direct/dresources/apitypes.generated.yml
+
+.PHONY: generate-ucm-direct-resources
+generate-ucm-direct-resources: ucm/direct/dresources/resources.generated.yml
+
+.PHONY: generate-ucm-direct-clean
+generate-ucm-direct-clean:
+	rm -f ucm/direct/dresources/apitypes.generated.yml ucm/direct/dresources/resources.generated.yml
+
+ucm/direct/dresources/apitypes.generated.yml: ./ucm/direct/tools/generate_apitypes.py .codegen/openapi.json acceptance/ucm/refschema/out.fields.txt
+	python3 $^ > $@
+
+ucm/direct/dresources/resources.generated.yml: ./ucm/direct/tools/generate_resources.py .codegen/openapi.json ucm/direct/dresources/apitypes.generated.yml ucm/direct/dresources/apitypes.yml acceptance/ucm/refschema/out.fields.txt
+	python3 $^ > $@
+
 .PHONY: test-exp-aitools
 test-exp-aitools:
 	make test TEST_PACKAGES="./experimental/aitools/..." ACCEPTANCE_TEST_FILTER="TestAccept/apps"

acceptance/ucm/drift/happy/output.txt

@@ -0,0 +1,5 @@
+
+>>> [CLI] ucm drift
+Warning: cannot initialize resource URLs: workspace.host is not set
+
+No drift detected.

acceptance/ucm/drift/happy/script

@@ -0,0 +1 @@
+trace $CLI ucm drift

acceptance/ucm/drift/happy/test.toml

@@ -0,0 +1,3 @@
+Local = true
+Cloud = false
+Ignore = [".databricks"]

acceptance/ucm/drift/happy/ucm.yml

@@ -0,0 +1,9 @@
+ucm:
+  name: drift-happy
+  engine: terraform
+
+resources:
+  catalogs:
+    main:
+      name: main
+      comment: drift-happy catalog

acceptance/ucm/import/missing_catalog/output.txt

@@ -0,0 +1,15 @@
+
+>>> [CLI] ucm import catalog test_catalog --auto-approve
+Warning: cannot initialize resource URLs: workspace.host is not set
+
+Warning: cannot initialize resource URLs: workspace.host is not set
+
+Error: direct import: get catalog test_catalog: Resource catalog.CatalogInfo not found: test_catalog (404 <nil>)
+
+Endpoint: GET [DATABRICKS_URL]/api/2.1/unity-catalog/catalogs/test_catalog?
+HTTP Status: 404 Not Found
+API error_code: <nil>
+API message: Resource catalog.CatalogInfo not found: test_catalog
+
+
+Exit code: 1

acceptance/ucm/import/missing_catalog/script

@@ -0,0 +1 @@
+trace $CLI ucm import catalog test_catalog --auto-approve

acceptance/ucm/import/missing_catalog/test.toml

@@ -0,0 +1,3 @@
+Local = true
+Cloud = false
+Ignore = [".databricks"]

acceptance/ucm/import/missing_catalog/ucm.yml

@@ -0,0 +1,9 @@
+ucm:
+  name: import-happy
+  engine: direct
+
+resources:
+  catalogs:
+    test_catalog:
+      name: test_catalog
+      comment: imported test catalog