feat(do): infra.spaces_key driver (engine-routing pattern, workflow v0.27.0)#84
Conversation
…pes (PR4b prep) This is the Task 15 Step 1 go.mod bump landed AHEAD of Tasks 11-14 so that their failing-test commits compile against the new workflow surface area (`interfaces.EnumeratorAll`, `cmd/wfctl.RotationResult`, `bootstrapSecrets` storage-filter behavior). Without this prep commit the rest of the PR4b stack would fail to compile against the prior v0.25.0 pin. Per plan section "PR4b — DO plugin infra.spaces_key driver + EnumeratorAll": > Sequencing: PR4b — but Task 15 go.mod bump must run at the START of > PR4b before any code in Tasks 11-14 references workflow types from > the new tag (won't compile against old workflow tag). `go test ./...` and `go build ./...` both green against v0.26.0 (no existing call sites broke; the bump is purely additive on the workflow side). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
iac-codemod refactor-apply reportMode: dry-run Skipped (// wfctl:skip-iac-codemod)
Full report (90-day retention) attached as workflow artifact. |
Adds TestProvider_EnumerateAll_SpacesKeys + supporting newDOProviderForTest
helper to internal/provider_test.go, per Task 14 of the spaces-key-iac-resource
plan (docs/plans/2026-05-08-spaces-key-iac-resource.md, commit 316559f7).
The test fakes a paginated DO API response (page 1 → 2 keys + next link;
page 2 → 1 key) and asserts that *DOProvider implements interfaces.EnumeratorAll
(added in workflow v0.26.0), that EnumerateAll(ctx, "infra.spaces_key") returns
3 *ResourceOutput entries (pagination handled inside the provider, not punted
to caller), and that each output has Type, ProviderID (= access_key), and
Outputs.{access_key, created_at} populated so audit-keys / prune CLIs can
filter without re-reading.
Currently FAILS with `Provider must implement EnumeratorAll` — the failing-side
signal Task 14 is supposed to produce. Task 15's EnumerateAll implementation
will make it PASS.
Stacks on top of Task 11's failing test for SpacesKeyDriver.Create (commit
21250db); both share the workflow v0.26.0 pin from commit 69718ef.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR kicks off PR4b by (1) pinning the plugin’s dependency on github.com/GoCodeAlone/workflow to v0.26.0 so upcoming code can use newly-shipped workflow interfaces, and (2) adding a contract-style test that defines the expected split-storage behavior for the upcoming infra.spaces_key driver’s Create method.
Changes:
- Bump
github.com/GoCodeAlone/workflowdependency from v0.25.0 → v0.26.0. - Add a new Spaces key driver “happy path” contract test plus test-only helpers (godo BaseURL rewrite + in-memory secrets provider).
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| internal/drivers/spaces_key_test.go | Adds the new contract test and test helpers for the future SpacesKeyDriver.Create implementation. |
| go.mod | Pins workflow dependency to v0.26.0. |
| go.sum | Updates checksums for the workflow v0.26.0 bump. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| client := godoClientForTest(t, srv.URL) | ||
| fakeGHSecrets := &fakeSecretsProvider{stored: map[string]string{}} | ||
|
|
||
| driver := NewSpacesKeyDriver(client, fakeGHSecrets) |
| @@ -0,0 +1,164 @@ | |||
| package drivers | |||
| srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | ||
| if r.URL.Path == "/v2/spaces/keys" && r.Method == http.MethodPost { | ||
| w.WriteHeader(http.StatusCreated) | ||
| _ = json.NewEncoder(w).Encode(map[string]any{ | ||
| "key": map[string]any{ | ||
| "name": "test-key", | ||
| "access_key": "AKIATEST123", | ||
| "secret_key": "SK_secret_data_here", | ||
| "created_at": "2026-05-08T11:00:00Z", | ||
| "grants": []any{map[string]any{"permission": "fullaccess"}}, | ||
| }, | ||
| }) |
| }) | ||
| return | ||
| } | ||
| t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path) |
Implements Task 15 of the spaces-key-iac-resource plan
(docs/plans/2026-05-08-spaces-key-iac-resource.md, commit 316559f7).
Adds three things to internal/provider.go:
1. (p *DOProvider) EnumerateAll(ctx, resourceType) — implements the
interfaces.EnumeratorAll opt-in interface (added in workflow v0.26.0;
see iac_provider.go). Switches on resourceType; spaces_key is the only
supported type today (other DO resources have native tag enumeration
via EnumerateByTag and don't need the all-resources path). Unknown
resourceType returns a clear error so audit-keys/prune CLIs surface a
precise diagnostic.
2. (p *DOProvider) enumerateAllSpacesKeys(ctx) — paginates GET
/v2/spaces/keys via godo SpacesKeys.List using ListOptions{Page,
PerPage:200}; loop terminates when godo signals last page (Links.Pages
== nil || Pages.Next == ""). Each *ResourceOutput carries name,
access_key, created_at, grants so the audit/prune CLIs can filter by
age or grant scope without re-reading every key. ProviderID is set to
AccessKey to match SpacesKeyDriver.Create's contract (sister driver in
internal/drivers/spaces_key.go).
3. grantsToMaps([]*godo.Grant) helper — converts the typed grants slice
into []map[string]any so it can live in ResourceOutput.Outputs (which
is map[string]any, not a typed slice). Returns nil for empty/nil input.
Verification:
- GOWORK=off go test ./internal -run "TestProvider_EnumerateAll" -v →
TestProvider_EnumerateAll_SpacesKeys (Task 14's failing test) now PASS
- GOWORK=off go test ./internal -count=1 → all internal-package tests PASS
- GOWORK=off go build ./cmd/plugin → produces a Mach-O binary (994KB).
Full `wfctl plugin list | grep infra.spaces_key` smoke depends on Task
12's driver registration in DOProvider.drivers + Capabilities() entry —
that lands separately in implementer-3's Task 12 commit.
Adaptations from plan:
- Plan code uses `(p *Provider)` receiver; actual type is `*DOProvider`.
- Added p.client nil guard at top of EnumerateAll (mirrors EnumerateByTag's
uninitialised-provider check, which is the in-tree pattern).
- Added k != nil and g != nil defensive checks in the loops.
Per Task 15 Step 6, after PR4b merges: tag DO plugin v0.14.0 and push.
That step happens post-merge, not in this commit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Switches `godoClientForTest` from `http.DefaultClient` to `srv.Client()` and takes `*httptest.Server` directly instead of just the URL string. This matches the sister-code pattern in `internal/provider_enumerator_test.go::newProviderForEnumeratorTest` (line 149: `godo.NewClient(srv.Client())`) — every godo request rides the test server's own transport rather than the package-global default client. Why it matters: another test could mutate `http.DefaultClient.Transport` via `defer`-restore around an httptest swap (the workflow secrets package does exactly this in `generators_test.go`). If that other test panicked mid-swap, our DO plugin tests would inherit a leaked transport. Using `srv.Client()` keeps each test self-contained. Closes Copilot 3212656317 carry-over from Task 11 quality review. No behavior change beyond test-isolation hardening — the `undefined: NewSpacesKeyDriver` failure mode is preserved (Task 12 still makes it pass). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
| // grantsToMaps converts a []*godo.Grant to []map[string]any so it can live in | ||
| // ResourceOutput.Outputs (which is map[string]any, not a typed grants slice). | ||
| // Returns nil for empty/nil input so the Outputs entry is omitted-rather-than-empty. | ||
| func grantsToMaps(grants []*godo.Grant) []map[string]any { |
| // Until Task 15 lands the EnumerateAll method on *DOProvider, this test | ||
| // fails with `Provider must implement EnumeratorAll` — which is the | ||
| // failing-side signal Task 14 is supposed to produce. |
| client := godoClientForTest(t, srv) | ||
| fakeGHSecrets := &fakeSecretsProvider{stored: map[string]string{}} | ||
|
|
||
| driver := NewSpacesKeyDriver(client, fakeGHSecrets) | ||
| spec := interfaces.ResourceSpec{ |
| // EnumerateAll lists every resource of resourceType in the DO account, | ||
| // regardless of tag. Used for resource types that don't support tags | ||
| // (e.g. spaces_key, where the DO API has no tag surface). Paginates | ||
| // transparently using godo.Response.Links so callers don't have to. | ||
| // | ||
| // Returns []*ResourceOutput per the workflow EnumeratorAll contract | ||
| // (interfaces/iac_provider.go) — full metadata is populated so | ||
| // downstream callers (wfctl infra audit-keys, prune, rotate-and-prune) | ||
| // can filter without re-reading each resource individually. | ||
| // | ||
| // Implements interfaces.EnumeratorAll (workflow v0.26.0+; opt-in | ||
| // interface, type-asserted by the host's IaCProvider proxy). | ||
| func (p *DOProvider) EnumerateAll(ctx context.Context, resourceType string) ([]*interfaces.ResourceOutput, error) { | ||
| if p.client == nil { | ||
| return nil, fmt.Errorf("digitalocean: EnumerateAll called on provider that is not initialized — call Initialize first") | ||
| } | ||
| switch resourceType { | ||
| case "infra.spaces_key": | ||
| return p.enumerateAllSpacesKeys(ctx) | ||
| default: | ||
| return nil, fmt.Errorf("digitalocean: EnumerateAll: resource type %q not supported", resourceType) | ||
| } | ||
| } |
…agate pagination error
Bundles three code-reviewer follow-ups on Tasks 14+15 (PR4b) into a
single commit:
1. Task 15 (Important / security) — `internal/provider.go`,
`enumerateAllSpacesKeys`: replace `break` after a `resp.Links.CurrentPage()`
parse error with `return nil, fmt.Errorf(...)`. Silent break would
cause prune/audit-keys to miss orphan keys past the failed page
boundary, leaving valid credentials live against DO. Sister
paginators in `EnumerateByTag` (droplets, volumes, databases) all
return-on-error; this matches that contract.
2. Task 14 (Minor / hermeticity) — `internal/provider_test.go`,
`newDOProviderForTest`: refactor signature from
`(t, serverURL string)` to `(t, srv *httptest.Server)` so the helper
can use `srv.Client()` instead of `http.DefaultClient`. Mirrors the
sister helper at `provider_enumerator_test.go:149` and
`internal/drivers/spaces_key_test.go` (post-f203b15). Hermetic — the
server's client trusts its own TLS cert if/when tests move to TLS
and never mutates the global http.DefaultClient state. Updated the
`TestProvider_EnumerateAll_SpacesKeys` call site accordingly.
3. Task 14 (Minor / structpb safety) — same test, `created_at` and
`access_key` assertions: change from `o.Outputs[k] == nil` (which
passes for any non-nil interface{} value, including time.Time) to
`if got, _ := o.Outputs[k].(string); got == ""`. ResourceOutput.Outputs
is map[string]any but the gRPC-side proto roundtrip (structpb) only
accepts string/number/bool/list/map values; a regression that stored
time.Time would pass a bare nil-check but fail proto marshal. Locking
the structpb-safe string shape at the test boundary catches that.
Verification:
- GOWORK=off go test ./internal -run "TestProvider_EnumerateAll" -v → PASS
- GOWORK=off go test ./internal -count=1 → all internal/ tests PASS
- GOWORK=off go build ./... → clean
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
workflow v0.27.0 ships the iac/sensitive package which routes ResourceOutput.Sensitive[k]==true fields through the engine's configured secrets.Provider. This unblocks the post-pivot SpacesKeyDriver landing — drivers no longer take a secrets.Provider parameter; the engine handles routing per-call based on the Sensitive map. Per ADR 0015/0020 and workflow PR #588 (commit ffae1409).
Pre-pivot: driver took a secrets.Provider parameter and called Set() itself; test asserted secret_key was NOT in Outputs and that fakeSecretsProvider.stored received both sub-keys. Post-pivot (workflow v0.27.0): driver returns Outputs containing every field including secret_key, with Sensitive[access_key|secret_key]=true. The engine's iac/sensitive.Route routes flagged fields through the configured secrets.Provider before persisting state — the driver itself is platform-agnostic. Test changes: - Switch package drivers -> drivers_test (black-box, addresses Copilot #2) - Drop fakeSecretsProvider + secrets import (no longer used) - NewSpacesKeyDriver(client) — single-arg constructor - Validate request payload reaching DO (name + grants), addresses Copilot #4 - Return 500 on unexpected request path (deterministic failure surface), addresses Copilot #5 - Assert Outputs.secret_key == "SK_secret_data_here" (driver returns it, engine routes after) - Assert Sensitive[access_key]=true, Sensitive[secret_key]=true, Sensitive[created_at] unset - Assert SensitiveKeys() includes both access_key + secret_key - Updated header comment to reflect engine-routing per ADR 0015/0020
internal/drivers/spaces_key.go — full ResourceDriver implementation per the engine-routing contract (workflow v0.27.0): - Create: builds godo.SpacesKeyCreateRequest from spec.Config, returns ResourceOutput with all fields populated (name, access_key, secret_key, created_at, grants) and Sensitive[access_key|secret_key]=true so the engine's iac/sensitive.Route handles routing through secrets.Provider per ADR 0015. - Read: refreshes via godo.SpacesKeys.Get(accessKey); falls back to enumerate-and-match-by-name when ProviderID is empty (upsert recovery path). secret_key is omitted — DO API only returns it on Create; the cached value lives in the engine's secrets.Provider. - Update: in-place rename + grant updates via godo.SpacesKeys.Update. - Delete: idempotent (404 -> success); matches RevokeProviderCredential stance for sister Spaces credentials. - Diff: compares mutable fields (name + grants); name divergence is an in-place Update for Spaces keys (no NeedsReplace). - HealthCheck: report healthy iff Get succeeds. - Scale: not supported (access keys have no replica concept). - SupportsUpsert: true (Read can locate by name when ProviderID empty). - ProviderIDFormat: IDFormatFreeform (DO access keys are not UUIDs). - SensitiveKeys: returns both access_key + secret_key for plan/diff display masking. internal/provider.go: - Register infra.spaces_key driver with NewSpacesKeyDriver(p.client) (single-arg, plugin-agnostic — the engine handles secrets routing). - Add infra.spaces_key to Capabilities (Tier 1, noScale ops). - enumerateAllSpacesKeys: only set Outputs["grants"] when non-empty so the docstring matches reality (Copilot #7).
The "Until Task 15 lands ... this test fails" preamble is stale once the EnumerateAll method already lands on *DOProvider in this branch. Replace with a forward-looking pin description (Copilot #9).
Post-pivot rewrite landed (HEAD
|
| Commit | Change |
|---|---|
c620f65 |
Bump workflow to v0.27.0 (engine-side sensitive routing) |
fd5a1b0 |
Pivot test fixture to engine-routing pattern (drop secrets.Provider param + fake) |
108108f |
Implement SpacesKeyDriver Create/Read/Update/Diff/Delete + register in provider |
ee28967 |
Update stale Task 14/15 comment in EnumerateAll test |
Driver responsibility under engine-routing
Createreturns*ResourceOutputwith all fields populated includingsecret_key, plusSensitive[access_key|secret_key] = true.- The engine's
iac/sensitive.Routepulls flagged values out, callssecrets.Provider.Set(ctx, "<resource>_<key>", value), replaces them withsecret_ref://...placeholders before persisting state. - The driver is fully plugin-agnostic — no
secrets.Providerparameter, noSet()calls. SensitiveKeys()(display masking) returns[access_key, secret_key]; this is independent of the per-callSensitivemap (routing trigger).
Copilot comments addressed in this push
- feat: bump workflow SDK to v0.15.2, release v0.4.0 #2 (line 1): switched test to
package drivers_test(black-box). - feat(plugin): full IaCProvider bulk-method dispatch — Plan/Apply/Destroy/Status/DetectDrift/Import/ResolveSizing #4 (line 46): test now validates the request payload (
name+grants) reaching the fake DO server. - chore: bump version to 0.5.0 #5 (line 49): unexpected paths now return HTTP 500 so godo surfaces deterministically.
- chore: bump version to 0.5.1 #7 (provider.go:762):
enumerateAllSpacesKeysnow omitsOutputs["grants"]entirely when empty so the docstring matches reality. - fix(health-check): recognize in-progress deployments in AppPlatform/APIGateway #9 (provider_test.go:1146): updated stale "Task 14/15 hasn't landed" comment.
- feat: map godo ErrorResponse to interfaces sentinels #10 (provider.go:704, scope mismatch): the PR title/description still describe the pre-pivot Task 11 + v0.26.0 pin. Title and description need a rewrite for the new scope — this branch now ships the full
infra.spaces_keydriver under engine-routing per ADR 0015/0020 + workflow v0.27.0. Will update the PR title/description after CI passes. - feat: expose iac.provider module type via SDK ModuleProvider + ServiceInvoker #1, feat(plugin): full ResourceDriver dispatch — Create/Read/Delete/Diff/Scale/SensitiveKeys #3 (compile-fail signals on Task 11 commit): no longer applicable —
NewSpacesKeyDriverlands in this push and the test compiles + passes. - fix(app-platform): translate flat image string to DO AppSpec nested image object #6 (pagination silent break): already addressed in
7ad1f02(returns the parse error rather than break-with-partial-result). - feat(app-platform): lookup app by name when ProviderID is empty #8 (
newDOProviderForTestusinghttp.DefaultClient): already addressed in7ad1f02(usessrv.Client()).
Verification
go build ./...— cleango test ./...— all pass (full driver suite + provider tests)go vet ./...— cleanwfctl plugin validate --strict-contracts—OK workflow-plugin-digitalocean v0.12.0iac-codemod refactor-apply -dry-run .— 0 errors, 1 deliberate skip
| require ( | ||
| github.com/GoCodeAlone/workflow v0.25.0 | ||
| github.com/GoCodeAlone/workflow v0.27.0 | ||
| github.com/aws/aws-sdk-go-v2 v1.41.6 |
Five Copilot inline findings on PR #84: #1 (Critical) spaces_key.go package doc claimed name changes force NeedsReplace=true, but godo's SpacesKeysService.Update accepts both Name and Grants — the impl was correct (in-place rename via Update), the doc was wrong. Updated doc to match the impl. #2 (Critical) provider_test.go TestProvider_EnumerateAll_SpacesKeys unexpected-request branch only called t.Errorf; godo could observe an empty 200 body and proceed. Added http.Error(...,500) so godo surfaces the failure as an HTTP error. Mirrors the same fix already in internal/drivers/spaces_key_test.go. #3 (Important) enumerateAllSpacesKeys returned Status="running" while SpacesKeyDriver.Create returns Status="active". Spaces keys have no lifecycle status in the godo API; "active" matches Driver.Create AND the sister bucket driver (internal/drivers/spaces.go). Switched enumerateAllSpacesKeys to "active" and added a comment explaining the canonical choice. #4 (Important) spacesKeyOutput unconditionally set Outputs["grants"] to grantsToMaps result (nil when empty). enumerateAllSpacesKeys omits the key entirely when grants is empty. Aligned spacesKeyOutput to the same omit-when-empty convention so both code paths produce identical state-record shapes. #5 (Minor) Tightened package doc claims for Read/Update/Delete: - Read AND Update return Outputs without secret_key (was previously only stating Read; Update also routes through includeSecret=false). - Delete idempotency on 404 stated explicitly in the package doc. go build ./... + go test ./... pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…overage Closes the regression-detection gap flagged by code-reviewer on PR #84: the existing TestSpacesKeyDriver_Create_HappyPath only pinned Create. For a destructive credential-lifecycle driver, the rest of the ResourceDriver methods needed coverage before merge. 15 new tests, all using stdlib testing + httptest (no testify): Read (4): - Read_HappyPath: GET /v2/spaces/keys/{access_key}; secret_key NOT in Outputs; Sensitive[secret_key] unset (DO API never re-emits secret). - Read_NotFound: 404 → errors.Is(err, ErrResourceNotFound) via WrapGodoError mapping. - Read_FindByName: empty ProviderID drives findByName fallback; paginated List response exercised. - Read_FindByName_NotFound: name-fallback exhaustion → ErrResourceNotFound. Update (2): - Update_RenameInPlace: PUT carries Name+Grants; secret_key absent in response Outputs (Update never re-issues secret). - Update_GrantsChange: grants-only update; request payload + Outputs shape pinned for read/readwrite permissions across two buckets. Delete (3): - Delete_HappyPath: 204 No Content → nil error. - Delete_Idempotent_404: 404 → nil (matches RevokeProviderCredential). - Delete_OtherError_Propagates: 500 → errors.Is(err, ErrTransient); guards against silent swallow of transient API failures. Diff (5): - Diff_NilCurrent_NeedsUpdate: nil current → NeedsUpdate=true; doesn't panic. - Diff_NoChanges: desired==current → 0 changes, no update, no replace. - Diff_NameChange: name divergence → NeedsUpdate=true (NOT NeedsReplace); godo Update accepts Name → in-place rename per ADR 0015. - Diff_GrantsChange: grants divergence → FieldChange{Path: grants} with canonical []map[string]any Old/New shapes (post-normalize). - Diff_StructpbRoundtripResilience: current.Outputs[grants] as []any{map[string]any} (post-structpb shape) MUST NOT false-positive drift; pins the gRPC plugin boundary contract per workspace memory feedback_workflow_plugin_structpb_boundary. HealthCheck (1): - HealthCheck: existence == healthy; spaces keys have no provider-side health concept. Verified: GOWORK=off go test ./internal/drivers ./internal -count=1 → ok. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Test-coverage gap closedCommit: 4e3f0ed — 15 new tests in
Critical correctness checks pinned:
Local verification (HEAD 4e3f0ed): No driver-impl changes were necessary — all tests pass against the existing implementation, confirming the v0.27.0 engine-routing contract is correctly implemented. |
| "bucket": g.Bucket, | ||
| "permission": string(g.Permission), | ||
| }) | ||
| } |
Summary
Lands the
infra.spaces_keyresource driver for the DO plugin under the engine-routing pattern (workflow v0.27.0 + ADR 0015/0020). The driver returnsResourceOutputwithSensitiveflags; the engine'siac/sensitivepackage routes flagged fields through the configuredsecrets.Providerper-call. The driver itself is plugin-agnostic — nosecrets.Providerparameter, noSet()calls.Tasks
SpacesKeyDriver.Create(pre-pivot, replaced this push)SpacesKeyDriver.Create+ spec→godo translationSpacesKeyDriver.Read/Update/Diff/Delete+ driver registrationEnumerateAll(infra.spaces_key)EnumerateAllimpl with godo paginationPivot context
This branch was originally on the pre-pivot design (driver received a
secrets.Providerparameter and stored sub-keys directly). Per the user's decision, the design pivoted to engine-routing: drivers returnOutputswithSensitiveflags, the engine handles routing.The pivot lands as 4 commits forward of
7ad1f02:c620f65— bump workflow to v0.27.0fd5a1b0— pivot test fixture108108f— implement driver + registeree28967— fix stale commentDriver contract
Createname,access_key,secret_key,created_at,grants)access_key,secret_keyReadsecret_key(DO API only returns it on Create)access_keyUpdatesecret_keyaccess_keyDeleteDiffHealthCheckScaleerrors.New("not supported")SensitiveKeys()returns[access_key, secret_key](display masking, independent from per-call routing).ProviderIDFormat()returnsIDFormatFreeform(DO access keys are not UUIDs).SupportsUpsert()returnstrue(Read can locate by Name when ProviderID is empty).Test plan
go test ./internal/drivers -run TestSpacesKeyPASSESgo test ./...all passgo build ./...cleango vet ./...cleanwfctl plugin validate --strict-contracts— OKiac-codemod refactor-apply -dry-run— 0 errors🤖 Generated with Claude Code