bundle: Add postgres_roles resource#5467
Merged
Merged
Conversation
Adds DAB support for Lakebase Postgres roles, mirroring the existing postgres_databases resource. The state holds role_id and parent separately (so bundle variable references resolve), and RemapState recovers role_id from remote.Name via a local strings.TrimPrefix — no shared parser helper. recreate_on_changes fires on either field since both are part of the immutable hierarchical name. Also fixes collectUpdatePathsWithPrefix to drop a parent path when a more specific child path is present; the real Postgres API rejects an update_mask that contains both (e.g. spec.attributes plus spec.attributes.createdb), expecting all sibling fields when the parent is named. Tested end-to-end against AWS prod (basic, recreate, update, bind) as well as the invariant suite. Co-authored-by: Isaac
Two follow-ups to the postgres_roles resource: - Regenerate required-field validation so role_id is required alongside parent, matching the JSON schema (jsonschema.json already lists both under required). Without this, bundle validate accepted a role config missing role_id and the failure only surfaced during deploy. - In PostgresRole.Exists, recognize 404 via apierr.IsMissing and return (false, nil) so bundle deployment bind reports the user-friendly "postgres_role ... is not found" path instead of a generic fetch error. Co-authored-by: Isaac
Missed alongside required_fields in the previous commit. Same generator run, just the second output file. Co-authored-by: Isaac
Previously logged "does not exist" for any GetRole error, including transient failures, before checking apierr.IsMissing. Flip the order so the debug message only fires when the role is genuinely absent. Co-authored-by: Isaac
The SDK's RoleRoleStatus already carries role_id; use it directly instead of stripping the "<parent>/roles/" prefix from remote.Name. Matches the catalog convention (Status.CatalogId) and avoids a local string parse. Co-authored-by: Isaac
# Conflicts: # NEXT_CHANGELOG.md # acceptance/bundle/invariant/continue_293/out.test.toml # acceptance/bundle/invariant/migrate/out.test.toml # acceptance/bundle/invariant/no_drift/out.test.toml # acceptance/bundle/invariant/test.toml # acceptance/bundle/refschema/out.fields.txt # bundle/config/mutator/resourcemutator/apply_bundle_permissions_test.go # bundle/config/mutator/resourcemutator/apply_target_mode_test.go # bundle/config/mutator/resourcemutator/run_as_test.go # bundle/config/resources.go # bundle/config/resources_test.go # bundle/deploy/terraform/interpolate.go # bundle/deploy/terraform/pkg.go # bundle/deploy/terraform/util.go # bundle/direct/dresources/all.go # bundle/direct/dresources/apitypes.generated.yml # bundle/direct/dresources/apitypes.yml # bundle/direct/dresources/resources.generated.yml # bundle/direct/dresources/util.go # bundle/internal/schema/annotations.yml # bundle/internal/validation/generated/enum_fields.go # bundle/internal/validation/generated/required_fields.go # bundle/schema/jsonschema.json # bundle/schema/jsonschema_for_docs.json # bundle/statemgmt/state_load_test.go # libs/testserver/fake_workspace.go # libs/testserver/handlers.go # libs/testserver/postgres.go
…ate-only Live testing showed the PATCH update_mask only accepts spec.attributes and spec.membership_roles; the backend rejects spec.postgres_role, spec.auth_method, and spec.identity_type with 400 INVALID_PARAMETER_VALUE "Unknown field path in update_mask". Without declaring these as recreate_on_changes: - direct engine: deploy fails on PATCH and re-plan loops on the same "1 to change" forever - terraform engine: silently no-ops the change (state records new value, remote keeps old, GET returns no spec → invisible divergence) These spec fields aren't marked immutable in the OpenAPI definition, so the generator can't pick them up — declare them in the manual resources.yml until upstream is fixed. Adds an acceptance test that toggles postgres_role and confirms the plan recreates instead of patching. Restricted to the direct engine because the terraform provider still treats the field as updateable and would silently diverge from the bundle. Co-authored-by: Isaac
Co-authored-by: Isaac
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
Collaborator
Integration test reportCommit: 935a4cf
22 interesting tests: 15 SKIP, 7 KNOWN
Top 24 slowest tests (at least 2 minutes):
|
janniklasrose
commented
Jun 9, 2026
…d helper
Revert collectUpdatePathsWithPrefix to its original form so the widely-used
helper (also called by postgres branches/endpoints/projects) is unchanged, and
add a dedicated collectLeafUpdatePathsWithPrefix used only by postgres roles.
The role PATCH endpoint rejects an update_mask listing both a struct and one of
its sub-fields, so {attributes, attributes.createdb} must collapse to the leaf.
The new helper also sorts its output so the generated update_mask is stable
across map-iteration order. Add unit tests covering the collapse, sorting, and
non-update filtering.
Co-authored-by: Isaac
GET returns the role configuration under Status, not Spec (Spec is nil on read). The previous RemapState zeroed the spec fields and claimed remote drift could not be detected, which is inaccurate: RoleRoleStatus echoes postgres_role, auth_method, identity_type, attributes, membership_roles and role_id. Map those back onto the flattened state so the snapshot reflects the live role. The spec fields stay ignore_remote_changes (spec:input_only), so the plan remains a no-op; the regenerated direct plan output just records the now-visible remote values instead of dropping them. Co-authored-by: Isaac
The fake role update applied the whole request spec regardless of update_mask,
so it could not model partial updates. Read the update_mask query parameter and
apply only the named spec fields onto the existing status, preserving the rest;
an empty mask updates everything, matching the API ("if unspecified, all fields
will be updated when possible").
Add a unit test asserting a field absent from update_mask is preserved even when
the request body carries a new value for it.
Co-authored-by: Isaac
janniklasrose
commented
Jun 9, 2026
janniklasrose
changed the title
Add a local-only acceptance test that reproduces what happens when a bundle declares a Postgres role that already exists on the branch — the situation a child branch lands in, since it inherits the parent's roles at creation. The testserver does not model branch role inheritance, so the precondition is staged with deploy + 'bundle deployment unbind': the role then exists on the branch but is untracked, and the next deploy plans to create it and fails. The Badness note records the real-Lakebase behavior verified on dogfood (HTTP 400 'role with that name already exists') and the two testserver fidelity gaps (no role inheritance on branch create; duplicate create returns 409 instead of 400). The supported fix is 'bundle deployment bind'. Co-authored-by: Isaac
The real Lakebase API rejects creating a role that already exists with HTTP 400
BAD_REQUEST ('role with that name already exists'), not 409 — verified on
dogfood. Match it so the conflict a bundle hits on an inherited/pre-existing
role looks identical locally and on cloud. Add a testserver unit test for it.
Reframe the inherited-role-conflict Badness around the product decision: the
4xx-on-existing-role is the intended interim behavior; until a replace_existing
flag lets a bundle take explicit ownership, 'bundle deployment bind' is the
supported escape hatch.
Co-authored-by: Isaac
Companion to inherited-role-conflict. Stages the same pre-existing-role state (deploy + unbind), then uses 'bundle deployment bind' to take explicit ownership; the subsequent deploy manages the role instead of trying to create it and succeeds. This is the supported way to adopt an inherited role until a replace_existing flag lands. Co-authored-by: Isaac
pietern
reviewed
Jun 15, 2026
| "auth_method", | ||
| "identity_type", | ||
| "membership_roles", | ||
| "postgres_role", |
Contributor
There was a problem hiding this comment.
Why is this not needed for the other postgres resources?
Contributor
Author
There was a problem hiding this comment.
other resources don't return spec, updated comment
Contributor
There was a problem hiding this comment.
This also doesn't return the spec.
The difference is a custom remote type that embeds the spec type in the remote type to ensure the shapes match. The spec is not returned today but will be returned in a couple of weeks. The spec is still annotated input_only, so they're all ignored for drift detection right now. When they become I/O, the annotation will be dropped, they'll be removed from resources.generated.yml and drift detection will work as usual. You can check out makePostgresBranchRemote for ex.
Resolve conflicts: - NEXT_CHANGELOG.md: keep main's v1.4.0 Bundles entries, re-add the Postgres role line, drop the already-released #5452 entry. - bundle/schema/jsonschema_for_docs.json: regenerated from the merged sources. Regenerate schema artifacts (jsonschema.json, annotations_openapi*.yml) and register postgres_roles in the workspaceurls noURL allowlist that main's new TestBundleResourcePluralNamesResolveInWorkspaceURLs requires. Co-authored-by: Isaac
Address review feedback: the old comment cited spec.attributes.createdb as if the logic masked that leaf, but it collapses nested paths to their top-level spec field and applies the whole field. State that plainly, and record the verified real-backend behavior (dogfood 2026-06-16: update_mask=spec.attributes.createdb masks at the leaf and preserves sibling attributes; identical to the collapse for the direct engine's full-spec request bodies). Co-authored-by: Isaac
Revert the status->spec mapping: GET returns the role config under Status, which rolls up server-side and parent-inherited defaults mixed with the spec, so mapping it back produces drift that is often unresolvable. Leave the spec empty and rely on ignore_remote_changes, matching how the other postgres resources handle it; revisit once the backend echoes the spec intent back on GET. Regenerate the update plan outputs accordingly. Also explain in type_test why postgres_roles needs knownMissingInRemoteType entries that the sibling resources don't: roles' DoRead returns the raw postgres.Role (spec nested under .Spec/.Status) rather than a custom *Remote type that embeds the spec, which is the deliberate consequence of not synthesizing a spec from status. Co-authored-by: Isaac
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
pietern
reviewed
Jun 16, 2026
| "auth_method", | ||
| "identity_type", | ||
| "membership_roles", | ||
| "postgres_role", |
Contributor
There was a problem hiding this comment.
This also doesn't return the spec.
The difference is a custom remote type that embeds the spec type in the remote type to ensure the shapes match. The spec is not returned today but will be returned in a couple of weeks. The spec is still annotated input_only, so they're all ignored for drift detection right now. When they become I/O, the annotation will be dropped, they'll be removed from resources.generated.yml and drift detection will work as usual. You can check out makePostgresBranchRemote for ex.
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
commented
Jun 16, 2026
| Successfully bound postgres_role with an id 'projects/test-project/branches/main/roles/test-role' | ||
| Run 'bundle deploy' to deploy changes to your workspace | ||
|
|
||
| >>> [CLI] bundle summary |
Contributor
Author
There was a problem hiding this comment.
this doesn't really show bind was successful, because it looks the same as after unbind
| title "Deploy project, branch, and role" | ||
| trace $CLI bundle deploy | ||
|
|
||
| title "Unbind the role: it now exists on the branch but is no longer tracked by the bundle" |
Contributor
Author
There was a problem hiding this comment.
better: teach test server inheritance, then
- add branch A with role "test-role" on A
- deploy
- add branch B off branch A, with role "test-role" on B
- deploy - conflict
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
pietern
approved these changes
Jun 16, 2026
Collaborator
Integration test reportCommit: 6a7a04a
424 interesting tests: 340 MISS, 42 FAIL, 29 RECOVERED, 8 KNOWN, 3 PANIC, 2 SKIP
Top 50 slowest tests (at least 2 minutes):
|
pietern
added a commit
that referenced
this pull request
Jun 17, 2026
Conflicts came from two origin/main changes landing on top of the annotation-storage overhaul: - #5611 (launch stage as the single preview source of truth) refactored the same codegen the branch rewrote. Resolved parser.go to keep the branch's in-memory annotation.File (SetSelf/SetField, no overrides file I/O) while adopting #5611's typed clijson.LaunchStage, ParseLaunchStage validation, and stageErr accumulation; dropped the now-removed Preview field and previewFromLaunchStage. annotations.go, descriptor.go, preview.go and extension.go auto-merged into the same shape. - #5467 (postgres_roles resource) added entries to the old multi-file annotations. Reproduced its manual descriptions for the PostgresRole fields in the new tree-format annotations.yml (the embedded RoleRoleSpec is flattened inline, so these can't be derived from cli.json); the RoleAttributes/RoleAuthMethod/etc. openapi docs regenerate in memory. The deleted annotations_openapi*.yml stay deleted. Verified: jsonschema.json and jsonschema_for_docs.json regenerate byte-identical to origin/main, annotations.yml is byte-idempotent, and go test ./bundle/internal/schema/... ./internal/clijson/... plus the refschema acceptance test pass. Co-authored-by: Isaac
pietern
added a commit
that referenced
this pull request
Jun 17, 2026
Resolve conflicts from main's #5467 (postgres_roles resource) landing while this branch developed postgres_databases on the earlier role pattern: - postgres_role.go: take main's #5467 version (PostgresRoleRemote drift detection, collectLeafUpdatePathsWithPrefix); drop the branch's superseded PostgresRoleState marshaler and its unit test. - Remove duplicate postgres_roles registrations from the add/add merge across resources.go, all.go, pkg.go, interpolate.go, fake_workspace.go, the testserver role handlers, and several test fixtures. - Apply main's role refinements to postgres_databases: leaf update-mask paths, testserver update_mask honoring (applyDatabaseSpecMask), and 400-on-duplicate create, with parallel testserver tests. - Regenerate apitypes/resources/refschema/schema artifacts; merge NEXT_CHANGELOG. Co-authored-by: Isaac
pietern
added a commit
that referenced
this pull request
Jun 17, 2026
…ermissions fixture Cross-checking against the postgres_roles PR (#5467) surfaced two misses for postgres_databases: - deployment/bind: ~20 resources (incl. postgres_role) have a deployment/bind acceptance test, but postgres_database had none. Add one mirroring the role test (bind -> summary -> unbind -> summary, both engines). - apply_bundle_permissions_test.go: the unsupportedResources slice listed "postgres_roles" twice — a slice-element add/add artifact from the earlier merge that compiles cleanly so build/test didn't catch it. Deduplicated. Other role-PR changes were already present for databases (config methods, the tfdyn converter, generated schema/validation/apitypes/resources, testserver handlers, mutator fixtures). Role-only items correctly have no database analog: enum_fields/annotation type-entries (the database spec has only scalar fields), and the inherited-role / recreate-postgres-role tests (role-specific semantics). Co-authored-by: Isaac
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Jun 17, 2026
## Release v1.4.0 ### CLI * Improved error messages for `ssh connect`: when an SSH connection attempt fails, the client now fetches and prints the server's recent error logs ([#5555](#5555)). * Increase the SSH server startup timeout from 10 to 45 minutes when a GPU accelerator is requested via `databricks ssh connect --accelerator` ([#5569](#5569)). * Fix authentication falling back to the default profile in `.databrickscfg` when a host is already configured via the environment (e.g. `DATABRICKS_HOST` with `DATABRICKS_TOKEN`) ([#5616](#5616)). * ssh: fix opening remote environment in Cursor, which previously hung on default-extension install and never opened the editor ([#5619](#5619)). * Improve the error shown when `databricks labs install` cannot find a project's `labs.yml`: the message now explains that either the requested version does not exist or the project is not installable with the CLI, and links to the repository ([#5559](#5559)). ### Bundles * Remove API enum values and types that are still in development from the `databricks-bundles` Python package; these were never accepted by the backend ([#5484](#5484)). * direct: Fix resolving a resource reference that is used more than once within the same field ([#5558](#5558)). * Bundle variable references now accept Unicode letters in path segments (e.g. `${var.变量}`). ([#5532](#5532)) * Ignore remote changes for vector search direct_access_index_spec.schema_json to prevent drift when the backend normalizes the schema ([#5481](#5481)). * Remove hidden, never-functional `--existing-dashboard-id`, `--existing-dashboard-path`, `--existing-alert-id`, and `--existing-genie-space-id` alias flags from `bundle generate`; use the documented `--existing-id` / `--existing-path` flags instead ([#5591](#5591)). * engine/direct: Fix WAL corruption after two consecutive failed deploys ([#5606](#5606)). * engine/direct: Don't open the deployment state WAL when a deploy's plan fails ([#5607](#5607)). * Ignore unity catalog managed schema property defaults to avoid unnecessary drift ([#5195](#5195)). * Add `postgres_roles` and `postgres_databases` resources to create Postgres roles and databases on a Lakebase branch ([#5467](#5467), [#5627](#5627)). * direct: Stop spurious recreate/rename on redeploy when the backend normalizes a resource's name-based ID (e.g. Unity Catalog lowercasing a schema or volume name) ([#5599](#5599)). * Fix the generated pipeline README to suggest `databricks bundle run <pipeline> --refresh <table>` for running a single transformation; the previously documented `--select` flag is not supported by `bundle run` ([#5252](#5252)).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Add
postgres_roles(Lakebase Postgres Roles) as a bundle resource for both the direct and Terraform engines: config type, direct-engine CRUD, Terraform converter, JSON schema, test-server handlers, and tests.parent,role_id, and the spec fieldspostgres_role/auth_method/identity_typeare recreate-only;attributes/membership_rolesupdate in place.RemapStatekeeps the spec empty (GET returns onlystatus, which rolls up server/parent defaults) and relies onignore_remote_changes, matching the sibling Postgres resources.Tests
update_maskhandling, the update-mask helper, state-load, bind-support, and field coverage (type_test).basic,update,recreate,recreate-postgres-role, plusinherited-role-conflict/inherited-role-binddocumenting inherited-role behavior.aws-prod-ucwsagainst real Lakebase (both engines).This pull request and its description were written by Isaac, an AI coding agent.