bundle: Add postgres_databases resource#5627
Merged
Merged
Conversation
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Add exhaustruct ForceSendFields zero on CreateDatabaseRequest and align the resources.yml comment with sibling phrasing. Co-authored-by: Isaac
Co-authored-by: Isaac
Adds DAB support for Lakebase Postgres roles, mirroring the existing postgres_databases resource. The state holds role_id and parent separately (so bundle variable references resolve), and RemapState recovers role_id from remote.Name via a local strings.TrimPrefix — no shared parser helper. recreate_on_changes fires on either field since both are part of the immutable hierarchical name. Also fixes collectUpdatePathsWithPrefix to drop a parent path when a more specific child path is present; the real Postgres API rejects an update_mask that contains both (e.g. spec.attributes plus spec.attributes.createdb), expecting all sibling fields when the parent is named. Tested end-to-end against AWS prod (basic, recreate, update, bind) as well as the invariant suite. Co-authored-by: Isaac
Two follow-ups to the postgres_roles resource: - Regenerate required-field validation so role_id is required alongside parent, matching the JSON schema (jsonschema.json already lists both under required). Without this, bundle validate accepted a role config missing role_id and the failure only surfaced during deploy. - In PostgresRole.Exists, recognize 404 via apierr.IsMissing and return (false, nil) so bundle deployment bind reports the user-friendly "postgres_role ... is not found" path instead of a generic fetch error. Co-authored-by: Isaac
Missed alongside required_fields in the previous commit. Same generator run, just the second output file. Co-authored-by: Isaac
Previously logged "does not exist" for any GetRole error, including transient failures, before checking apierr.IsMissing. Flip the order so the debug message only fires when the role is genuinely absent. Co-authored-by: Isaac
The live Lakebase API rejects POST without spec.role despite the SDK's
omitempty tag. Interpolate ${workspace.current_user.domain_friendly_name}
into the existing basic/recreate/update fixtures so they run unchanged on
both the local testserver and cloud. Regenerate recorded outputs.
Co-authored-by: Isaac
Add live_errors/missing_role to lock in the live API behavior observed during the 2026-05-19 dogfood2 smoke: POST .../databases without spec.role returns 400 with "Field 'spec.role' cannot be empty". A local Server stub overrides the default testserver auto-fill so the test runs identically on local and cloud. direct engine only; terraform's rollback-on-failure semantics diverge on error paths (same reason jobs/create-error is direct-only). Co-authored-by: Isaac
The real Lakebase API includes database_id in the database status, echoing the create-time query parameter. The testserver omitted it, so local and cloud acceptance outputs diverged after the role parameterization landed. Capture it on create so GET responses match. Co-authored-by: Isaac
Replace ${workspace.current_user.domain_friendly_name} with an
explicit postgres_roles.owner resource so the bundle works for any
caller. The previous approach worked for human users on dogfood2 but
failed on the SP-authenticated aws-prod-ucws because the auto-created
project-owner role's id is not consistently derivable from the
workspace user identity. Declaring the role explicitly sidesteps that
naming variance and exercises postgres_roles as a side effect.
All three tests (basic, recreate, update) pass against the live API on
aws-prod-ucws for both direct and terraform engines.
Co-authored-by: Isaac
Two additional live-validated error paths, both observed against
aws-prod-ucws on 2026-05-19:
- bad_role_ref: referencing a role that does not exist yields
404 NOT_FOUND with message "role not found; role_id:..." and a
trailing [TraceId: <hex>] suffix. Stub mirrors the response shape;
a regex Repls normalizes the TraceId so output is deterministic.
- bad_database_id: an underscore in database_id violates the pattern
^[a-z]([a-z0-9-]{0,61}[a-z0-9])?$. The API echoes the failing value
and the pattern in the message; locking that in catches regressions
on either side.
Findings during probing (not turned into tests): "postgres" as a
database_id is accepted, a 3-char database_id is accepted, and a
postgres_database starting with a digit is accepted — the live API
does not currently validate these inputs despite SDK doc-comments
implying otherwise.
direct engine only; matches missing_role rationale.
Co-authored-by: Isaac
The SDK's RoleRoleStatus already carries role_id; use it directly instead of stripping the "<parent>/roles/" prefix from remote.Name. Matches the catalog convention (Status.CatalogId) and avoids a local string parse. Co-authored-by: Isaac
# Conflicts: # NEXT_CHANGELOG.md # acceptance/bundle/invariant/continue_293/out.test.toml # acceptance/bundle/invariant/migrate/out.test.toml # acceptance/bundle/invariant/no_drift/out.test.toml # acceptance/bundle/invariant/test.toml # acceptance/bundle/refschema/out.fields.txt # bundle/config/mutator/resourcemutator/apply_bundle_permissions_test.go # bundle/config/mutator/resourcemutator/apply_target_mode_test.go # bundle/config/mutator/resourcemutator/run_as_test.go # bundle/config/resources.go # bundle/config/resources_test.go # bundle/deploy/terraform/interpolate.go # bundle/deploy/terraform/pkg.go # bundle/deploy/terraform/util.go # bundle/direct/dresources/all.go # bundle/direct/dresources/apitypes.generated.yml # bundle/direct/dresources/apitypes.yml # bundle/direct/dresources/resources.generated.yml # bundle/direct/dresources/util.go # bundle/internal/schema/annotations.yml # bundle/internal/validation/generated/enum_fields.go # bundle/internal/validation/generated/required_fields.go # bundle/schema/jsonschema.json # bundle/schema/jsonschema_for_docs.json # bundle/statemgmt/state_load_test.go # libs/testserver/fake_workspace.go # libs/testserver/handlers.go # libs/testserver/postgres.go
Only NEXT_CHANGELOG.md conflicted (kept the postgres_databases entry alongside main's #5599 entry). main's #5599 added provided_id_fields/updatable_id_fields for name-based IDs to schemas/volumes only — postgres resources keep recreate_on_changes, so no realignment was needed. Generated schema artifacts (#5611 codegen change) are consistent; regeneration produced no drift. Co-authored-by: Isaac
…ermissions fixture Cross-checking against the postgres_roles PR (#5467) surfaced two misses for postgres_databases: - deployment/bind: ~20 resources (incl. postgres_role) have a deployment/bind acceptance test, but postgres_database had none. Add one mirroring the role test (bind -> summary -> unbind -> summary, both engines). - apply_bundle_permissions_test.go: the unsupportedResources slice listed "postgres_roles" twice — a slice-element add/add artifact from the earlier merge that compiles cleanly so build/test didn't catch it. Deduplicated. Other role-PR changes were already present for databases (config methods, the tfdyn converter, generated schema/validation/apitypes/resources, testserver handlers, mutator fixtures). Role-only items correctly have no database analog: enum_fields/annotation type-entries (the database spec has only scalar fields), and the inherited-role / recreate-postgres-role tests (role-specific semantics). Co-authored-by: Isaac
…_role
postgres_role.GetName returns "" ("roles don't have a user-visible name
field"); postgres_database returned DatabaseId, so bundle summary showed a Name
for databases but not roles. Return "" for consistency. Regenerated the affected
summary outputs (bind test and basic).
Co-authored-by: Isaac
Co-authored-by: Isaac
postgres_databases was absent from the deploy/destroy approval groups, so a recreate (database_id/parent change) or delete silently dropped the Postgres database with no prompt and no abort in non-interactive mode. Add it to both groups with data-loss messages matching the sibling postgres_projects/branches entries, and cover the non-interactive abort in the recreate acceptance test. Co-authored-by: Isaac
Co-authored-by: Isaac
pietern
changed the title
Conflict in bundle/internal/schema/annotations.yml: #5574 consolidated the annotations into a single tree-structured generated file. Resolved to main's structure and regenerated so postgres_databases is re-added in the new form. Also reordered the postgres entries in acceptance/bundle/invariant/test.toml to match main (project, branch, endpoint, catalog, synced_table, role) with postgres_databases appended after postgres_roles. Co-authored-by: Isaac
…diff Match the sibling Lakebase resources (projects/branches/endpoints/catalogs/ synced_tables): declare parent and database_id via provided_id_fields so a local change recreates while a backend-normalized remote value is skipped, instead of recreate_on_changes. Also keep apitypes.yml to a single added line. Co-authored-by: Isaac
andrewnester
approved these changes
Jun 17, 2026
| } | ||
|
|
||
| // Wait for the database to be ready (long-running operation) | ||
| result, err := waiter.Wait(ctx) |
Contributor
There was a problem hiding this comment.
Is it intentional not to do the wait in WaitAfterCreate?
Contributor
Author
There was a problem hiding this comment.
Yes, we need the "operation ID" to resume waiting in a future call, if we have to.
All postgres resources do the same at the moment.
Addressing this likely means persisting the operation ID, because we can't use the database until this long running operation returns. A future update might work or might not work, undefined.
#5452 shipped and was moved to CHANGELOG.md upstream; it was carried over here through the changelog merge conflicts. Co-authored-by: Isaac
| trace $CLI bundle plan | ||
|
|
||
| title "Recreate requires approval: non-interactive deploy must abort, not silently delete the database" | ||
| trace errcode $CLI bundle deploy |
Contributor
There was a problem hiding this comment.
nit: musterr (I'm working on addressing this repo wide as part of a separate follow-up PR)
janniklasrose
approved these changes
Jun 17, 2026
The recreate test relied on errcode to record the exit code of the approval-gated deploy; musterr asserts it must abort, so a regression that silently recreates the database (losing data) fails the test. Co-authored-by: Isaac
Collaborator
Integration test reportCommit: 717d4c1
25 interesting tests: 15 SKIP, 7 KNOWN, 3 flaky
Top 24 slowest tests (at least 2 minutes):
|
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Jun 17, 2026
## Release v1.4.0 ### CLI * Improved error messages for `ssh connect`: when an SSH connection attempt fails, the client now fetches and prints the server's recent error logs ([#5555](#5555)). * Increase the SSH server startup timeout from 10 to 45 minutes when a GPU accelerator is requested via `databricks ssh connect --accelerator` ([#5569](#5569)). * Fix authentication falling back to the default profile in `.databrickscfg` when a host is already configured via the environment (e.g. `DATABRICKS_HOST` with `DATABRICKS_TOKEN`) ([#5616](#5616)). * ssh: fix opening remote environment in Cursor, which previously hung on default-extension install and never opened the editor ([#5619](#5619)). * Improve the error shown when `databricks labs install` cannot find a project's `labs.yml`: the message now explains that either the requested version does not exist or the project is not installable with the CLI, and links to the repository ([#5559](#5559)). ### Bundles * Remove API enum values and types that are still in development from the `databricks-bundles` Python package; these were never accepted by the backend ([#5484](#5484)). * direct: Fix resolving a resource reference that is used more than once within the same field ([#5558](#5558)). * Bundle variable references now accept Unicode letters in path segments (e.g. `${var.变量}`). ([#5532](#5532)) * Ignore remote changes for vector search direct_access_index_spec.schema_json to prevent drift when the backend normalizes the schema ([#5481](#5481)). * Remove hidden, never-functional `--existing-dashboard-id`, `--existing-dashboard-path`, `--existing-alert-id`, and `--existing-genie-space-id` alias flags from `bundle generate`; use the documented `--existing-id` / `--existing-path` flags instead ([#5591](#5591)). * engine/direct: Fix WAL corruption after two consecutive failed deploys ([#5606](#5606)). * engine/direct: Don't open the deployment state WAL when a deploy's plan fails ([#5607](#5607)). * Ignore unity catalog managed schema property defaults to avoid unnecessary drift ([#5195](#5195)). * Add `postgres_roles` and `postgres_databases` resources to create Postgres roles and databases on a Lakebase branch ([#5467](#5467), [#5627](#5627)). * direct: Stop spurious recreate/rename on redeploy when the backend normalizes a resource's name-based ID (e.g. Unity Catalog lowercasing a schema or volume name) ([#5599](#5599)). * Fix the generated pipeline README to suggest `databricks bundle run <pipeline> --refresh <table>` for running a single transformation; the previously documented `--select` flag is not supported by `bundle run` ([#5252](#5252)).
Collaborator
Integration test reportCommit: ede4566
470 interesting tests: 406 MISS, 43 FAIL, 10 KNOWN, 6 RECOVERED, 3 PANIC, 2 SKIP
Top 50 slowest tests (at least 2 minutes):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Add
postgres_databases(Lakebase Postgres databases) as a bundle resource for both the direct and Terraform engines: config type, direct-engine CRUD, Terraform converter, JSON schema, test-server handlers, and tests. Pairs withpostgres_roles(#5467): a database is owned by a Postgres role, so the create API requiresspec.role.parentanddatabase_idare recreate-only; the spec fieldspostgres_database/roleupdate in place.specon GET (it currently returns onlystatus).Tests
basic,update,recreate,bind,live_errors).role, duplicate create) pass.This pull request and its description were written by Isaac.