Skip to content

Runtime-witness persistence loses the evaluated-vs-not-evaluated distinction #331

Description

@PhysShell

Problem

docs/runtime-witness-operations.md defines an exit-code contract that deliberately keeps three states apart:

0 = evaluated, witness absent   (ABSENT / OBSERVED_ONLY)
1 = evaluated, witness present  (RETAINED)
2 = not evaluated               (usage error, unreadable target, refused attach)

and states the reason plainly:

Not looking and looking and finding nothing are different outcomes, and collapsing them is how a monitoring pipeline learns to report health it never measured.

The persistence layer then collapses exactly that distinction. A refused attach writes no runtime.json at all, so once the process has exited the durable record is:

artifact present  →  some evaluated outcome
artifact absent   →  not_evaluated
                  OR never invoked
                  OR runner died before invocation
                  OR the run happened but persistence failed
                  OR the artifact was produced and lost in transit
                  OR an older format nothing reads any more

Absence has too many preimages to carry meaning. The process contract distinguishes three states; the storage contract merges two of them back together and adds four more, so the guarantee the exit codes were designed to provide does not survive the end of the process.

Refusing to write a verdict that was never earned is correct and should stay. The defect is using file-absence as the representation of "not evaluated", rather than recording it.

Invariant

Absence of a record means no durable knowledge, never a semantic outcome.

Acceptance

Every attempted witness evaluation leaves a durable execution-state record.

observed         observation scope required
clean            observation scope required
not_evaluated    reason required (refused attach, unreadable target,
                 usage error, ...)
error            error classification required

missing record   has no semantic interpretation

A clean with no recorded scope is a malformed record, not a weak one — it should take the same handling path as a schema violation, not be read as a quieter version of not_evaluated.

Scope

A representation defect in the runtime-witness lane, self-contained. It needs no new analysis capability, no change to the exit-code contract (which is right as it stands), and no change to what the witness measures — only that the state the process already computes survives into storage.

Not related to #270, which covers generating witness evidence rather than how its execution state is recorded.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions