Skip to content

Record dev store shop id in command analytics - #8535

Draft
amcaplan wants to merge 1 commit into
mainfrom
record-shop-id
Draft

amcaplan wants to merge 1 commit into
mainfrom
record-shop-id

Conversation

@amcaplan

@amcaplan amcaplan commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

WHY are these changes introduced?

Dev store creations can't be attributed to the CLI vs. the dev dashboard in our metrics. Both surfaces call the same Business Platform mutation, and the store_creations row it writes keeps no caller — so there is nothing to group by, and shop_signup_requests tags both paths identically.

Today the only way to identify CLI-created dev stores is a regex on the shop signup request's user agent, which works but breaks if the user agent format ever changes.

Context: #dev-platform thread.

WHAT is this pull request doing?

Request shopifyShopId from createAppDevelopmentStore and record it, with the store domain, as command metadata — giving the warehouse a store_id to join on instead of inferring the surface from a user agent.

One note for anyone writing the query: cli3_commands also has a shop_id column, from monorail schemas 1.23 and 1.27, but no released CLI ever populated it — the field was added and withdrawn between 3.94.3 and 4.0.0, then renamed to store_id in 1.28 before being adopted. store_id is the live field.

Preview stores already record this, so recordStoreFqdnMetadata moves into organizations and is shared rather than duplicating the metadata shape. The dependency already runs storeorganizations, so the helper moves down a layer instead of the call moving up, and packages/store/src/cli/services/store/attribution.ts becomes a one-line re-export — the same shim pattern store already uses for createDevStore and businessPlatformTokenRefreshHandler, which keeps every existing import site in store unchanged. app reaches createDevStore through the same barrel (app/src/cli/services/dev/select-store.ts:40), so one call site inside createDevStore covers both store create dev and app dev's inline store creation.

While moving it, the helper now takes an options object. validated was a positional boolean at nine call sites — recordStoreFqdnMetadata(store, false) told a reader nothing, and with the optional id, argument order was the only thing separating the store domain from the store id. This is what puts the store call sites back in the diff; they're mechanical, and no behavior changes.

Four decisions worth a reviewer's eye:

  • Recorded before polling, not after. A store can exist server-side but never reach COMPLETE, and those are exactly the cases worth seeing during rollout. A poll timeout or failure still attributes.
  • The shop id is normalized. Business Platform returns it as a global id (gid://shopify/Shop/123) — the resolver passes the same value as shopify_shop_gid — and the metadata field is numeric, so an un-normalized value would be silently dropped. Bare-numeric values pass through, matching the existing numberFromGid handling in app-management-client.ts.
  • The helper lives in organizations, not cli-kit. Only store and organizations need it today, and cli-kit's ./* export would make it published public API. Promoting it later is cheap; demoting it isn't.
  • validated stayed required. Seven call sites pass true and two pass false, so neither default is safe: defaulting to true would silently claim user-supplied input had been checked against the platform.

This makes the CLI vs. dashboard split visible, but it is a floor rather than exact attribution: CLI analytics are opt-out and best-effort, so anything missing reads as a dashboard creation, and failed creations produce no shop id at all. Persisting a source on store_creations in shop/world is still the complete fix.

No changeset — this is telemetry only, with no user-visible behavior change.

How to manually test your changes?

shopify store create dev --name my-test-store --plan basic

The store id and domain should appear in the command's analytics payload:

SHOPIFY_CLI_ENV=development DEBUG=1 shopify store create dev --name my-test-store --plan basic

Look for store_id and store_domain in the reported metadata. shopify store create preview should report the same fields as before.

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows)
  • I've considered possible documentation changes
  • I've considered analytics changes to measure impact

🤖 Generated with Claude Code

@github-actions github-actions Bot added the no-changelog This PR doesn't include a changeset entry. Is an internal only change not relevant to end users. label Sep 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

packages/cli-kit/dist/public/node/store-attribution.d.ts
/**
 * Records the store a command acted on, so command analytics can be grouped by store.
 *
 * @param storeFqdn - The store's fully qualified domain name.
 * @param validated - Whether the fqdn was confirmed against the platform, rather than taken from user input as-is.
 * @param storeId - The store's numeric id as a string. Ignored when it isn't numeric.
 */
export declare function recordStoreFqdnMetadata(storeFqdn: string, validated: boolean, storeId?: string): Promise<void>;

Existing type declarations

We found no diffs with existing type declarations

@amcaplan
amcaplan force-pushed the record-shop-id branch 2 times, most recently from 556a31a to 2d8783b Compare September 14, 2026 18:03
Dev store creations can't be attributed to the CLI today. Both the CLI and the
dev dashboard call the same Business Platform mutation, and the `store_creations`
row it writes keeps no caller, so the only way to identify CLI-created dev stores
is a regex on the shop signup request's user agent.

Request `shopifyShopId` from the create mutation and record it, along with the
store domain, as command metadata. Preview stores already do this, so move
`recordStoreFqdnMetadata` into `organizations` and share it rather than
duplicating the metadata shape in two packages. `store` already depends on
`organizations`, so its own attribution module becomes a one-line re-export,
matching the shims it already uses for `createDevStore` and
`businessPlatformTokenRefreshHandler`.

Recorded before polling, so a store that is created but never reaches COMPLETE is
still attributed.

Also take the helper's arguments as an options object. `validated` is a
positional boolean at nine call sites, where `recordStoreFqdnMetadata(store,
false)` gives a reader nothing to go on, and the optional id made argument order
the only thing distinguishing the two.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Assisted-By: devx/47074c59-ea41-4708-aac2-7fccae638d6f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog This PR doesn't include a changeset entry. Is an internal only change not relevant to end users.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant