Skip to content

client: name the three host facts that make a consumer's tests pass locally - #40

Open
iceteaSA wants to merge 1 commit into
cortexkit:masterfrom
legion-works:docs/consumer-testing-facts
Open

client: name the three host facts that make a consumer's tests pass locally#40
iceteaSA wants to merge 1 commit into
cortexkit:masterfrom
legion-works:docs/consumer-testing-facts

Conversation

@iceteaSA

@iceteaSA iceteaSA commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Docs only, packages/client/README.md, +23 lines. Gate green.

Three ways a consumer's suite passes on the machine that runs the vault and proves nothing. Each was found the same way — someone shipped green and failed elsewhere — so this is a record of observed traps, not anticipated ones.

The connection file at the default path. If the daemon runs on the box, detectClaustrumConnection finds its socket whether or not a test arranged one. Two of anthropic-auth's suites passed on my daemon's real connection file and went red on CI, which has none. The daemon leaving that file is a host fact; a test that does not force it absent is asserting my machine's state.

Suites defaulting into ~/.config/opencode. Same PR, same day: tests resolving the real config path wrote lock files beside the operator's live credentials. Locally invisible, because the writes succeed.

A bare bun run as a resolution oracle. This one I got wrong myself in #39 before testing it. The Bun CLI auto-installs a public dependency it cannot resolve — it needs only a package.json in scope, no node_modules — so a bare-CLI repro silently supplies the dependency whose absence is under test. A compiled binary has no auto-install:

compiled loader -> disk module, package.json present, no node_modules
    ERR_MODULE_NOT_FOUND

bare `bun -e`   -> same module, same directory, same absent node_modules
    resolved

Same file, only the loader differs. Consumers of a credential client are daemons and plugins, so the compiled case is the one that matters and the convenient case is the one that gets tested.

I stated the shape rather than only the three instances, because the instances are not the useful part: the producing machine supplies the thing under test, so a passing check is evidence only once the ambient supply is removed. Naming that is what lets the next consumer recognise a fourth one.

Related to #39 but independent of it — this lands regardless of how the packaging question resolves.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Adds a docs-only section to packages/client/README.md cataloguing three host-environment traps that make a consumer's tests pass locally and fail elsewhere.

  • A connection file left at the default path by the vault daemon satisfies detectClaustrumConnection without a test arranging one.
  • Suites that resolve ~/.config/opencode without an override write lock files beside the operator's live credentials.
  • A bare bun run auto-installs unresolvable public dependencies, while compiled binaries do not — so bun -e is not a valid oracle for module resolution.

Written for commit 8a8512c. Summary will update on new commits.

Review in cubic

…ocally

Each was found by a consumer shipping green and failing elsewhere: a
connection file the daemon leaves at the default path, a suite defaulting
into the operator's live config, and a bare `bun` run auto-installing the
dependency whose absence the test exists to catch. The last one is
measured rather than asserted -- compiled binaries have no auto-install,
which is every consumer this client has.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

@ckcred-alfonso

ckcred-alfonso Bot commented Sep 8, 2026

Copy link
Copy Markdown

The three facts are right and this is worth having in the README. I reproduced the one that is checkable from here, and it does not behave the way the text says.

The asymmetry is real; the named symptom is not

Measured on bun 1.4.0, empty temp dir, an uninstalled public package:

bun -e            "import x from 'left-pad'"   ->  resolved: function
bun run m.mjs     (NO package.json in scope)   ->  resolved: function
bun build --compile m.mjs                      ->  error: Could not resolve: "left-pad".
                                                   Maybe you need to "bun install"?
                                                   NO BINARY IS PRODUCED

So your conclusion holds and your recommendation is sound — a bare bun run is not an oracle for module resolution, and the compiled path is the honest check. Two corrections to the text:

It fails at BUILD, not at run. The README says the compiled loader gives ERR_MODULE_NOT_FOUND. Here there is no loader to reach: bun build --compile refuses and emits nothing. That difference matters to the reader you are writing for — someone following this will run the build, see a build error, and reasonably wonder whether they have mis-set something up, when in fact they have just seen the check working. Naming the actual message (Could not resolve: "X". Maybe you need to "bun install"?) makes a correct result recognisable as one.

It also strengthens the point rather than weakening it: a build-time refusal is louder and earlier than a runtime throw, so the compiled path is a better oracle than your text claims.

The package.json qualifier does not hold. You write that the auto-resolution needs "only a package.json in scope". With no package.json anywhere in the temp dir, bun run m.mjs still resolved. So the ambient supply is broader than the text says, which makes the hazard slightly worse and the advice unchanged.

Both may be version-dependent, which is the argument for pinning the reading: I would add measured on bun 1.4.0 beside the numbers. A behavioural claim about a toolchain with no version attached reads as permanent, and this one is a moving target by construction.

On the other two

The connection-file one I can confirm from the other side. detectClaustrumConnection walks the real host paths, and on this machine the daemon writes to ~/.local/share/cortexkit/run/subc-connection.json — so a suite that does not force it absent is testing against a live vault whether or not it means to. Your CLAUSTRUM_SUBC_CONNECTION=/nonexistent/x.json plus clearing XDG_RUNTIME_DIR is the right pair; the second half is easy to omit, and omitting it leaves one rung of the ladder still live.

I shipped this exact defect in a Rust fixture earlier this week — a temp-dir override that took on macOS and Linux and silently did not on Windows, so the suite passed on two platforms while scanning the runner's real temp directory on the third. A fixture that redirects on some hosts and not others is worse than one that redirects on none, because it passes everywhere you can run it.

The closing line is the transferable part

The producing machine supplies the thing under test. A passing check on it is evidence only when the ambient supply is removed first.

That generalises past this client and past testing. It is the same reason a green gate on my machine says nothing about a fork PR's CI, and the same reason a .db file copied without its WAL opens cleanly and answers from an earlier moment.

@ckcred-alfonso

ckcred-alfonso Bot commented Sep 8, 2026

Copy link
Copy Markdown

Correcting my own review above. I said the compiled loader "fails at BUILD, not at run". That is true of the shape I tested and false of the shape you meant, and the difference is the whole point.

Both reproduce here, on bun 1.4.0:

STATIC import in the compiled source
    bun build --compile         error: Could not resolve: "left-pad"
                                no binary produced                    <- what I measured

DYNAMIC import of a disk module whose bare dep the build never sees
    bun build --compile         ok, 64 MB binary
    ./host ./mod/lib.mjs        error: Cannot find package 'left-pad'  <- what you measured
                                from .../mod/lib.mjs

Same toolchain, opposite failure points, and yours is the one that matches a plugin or a path-imported client under a shipped host. My correction was scoped to my test rather than to your claim, which is the error I have been naming in other people's work all week.

So the README should say which shape it means. I would name both, because the pair is more useful than either: a bare dep the compiler can see is refused early and loudly; one it cannot see survives the build and fails at import time in the consumer's process, which is exactly where a credential client cannot afford to fail.

An instrument note, because it nearly manufactured a false confirmation

My first run of your shape returned this:

rc=137, stdout 0 bytes, stderr 0 bytes
Killed: 9

Read casually that is "the compiled binary failed", which agrees with your conclusion — and it is not your mechanism at all. macOS SIGKILLs a freshly linked unsigned 64 MB binary; codesign --force --sign - and the same run gives the real error. An empty stderr with rc=137 is a signature problem wearing a resolution problem's clothes, and had I stopped there I would have confirmed your claim for a reason that has nothing to do with module resolution.

Worth adding to the README beside the third fact, since anyone reproducing this on macOS will hit it: sign the compiled probe, or you are measuring Gatekeeper.

On #39

Your follow-up there changes the shape of the packaging question rather than just supporting it, and I think you are right: documenting bun install && bun run build is not sufficient while tsc emits per-file output with the bare import intact, because the consumer then needs node_modules reproduced beside the dist. Either the build gains a bundling step or the package is published. I have that decision parked with my operator and will answer there rather than deciding it in a comment thread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant