Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion docs/notes/metamorphic.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,24 @@ Statfier line: testing the analyzer, not the program.
invariant under N classes of meaning-preserving transformation across the
corpus." The research-landscape note files this as a separable contribution.
4. **Free corpus amplification** — each real case spawns many variants that must
all behave identically.
all behave identically. Amplification of *robustness* coverage; see the
counting rule below before quoting it as corpus size.

### Counting rule

Metamorphic variants amplify perturbation/robustness coverage, **not** semantic
case count. Report these dimensions separately:

- **semantic cases** — independently distinct behaviour/invariant scenarios;
- **project/environment cases** — independent source/project contexts;
- **metamorphic variants** — meaning-preserving perturbations of a parent case;
- **executions** — total generated/checker runs.

A parent case and N metamorphic variants count as **one** semantic case and N
metamorphic variants. Metamorphic variants MUST NOT be used to satisfy a minimum
semantic-corpus size or an independence-based evaluation threshold. A harness
that renames `x` to `y` 376 times has not produced 376 observations, and an
evaluation that reports it as one number is decoratively green.

It is *also* the conformance check a future LLM fix-loop needs (RLVR reward =
checker-green **and** behavior-preserved, see `research-landscape-2026.md`) — but
Expand Down
21 changes: 18 additions & 3 deletions docs/proposals/P-025-obligation-protocols.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,24 @@ universal protocols (dispose your `IDisposable`, unsubscribe your event);
NDepend/CodeQL can query structure but have no barrier-sensitive obligation
model; typestate research languages don't speak legacy C#. The niche is real:
**barrier-sensitive, project-specific obligation checking for code review** —
and the OwnAudit STS corpus already shows the shape in the wild (17k
INPC findings, 8 recorded `IsLoaded` findings, `BrokerDataClasses` as the
subscription-leak epicenter).
and the OwnAudit STS material already shows the shape in the wild: real
project-specific notification/lifecycle patterns, including `IsLoaded`-style
consistency state, and a visible concentration of related findings in
`BrokerDataClasses`.

**Quantitative claims are intentionally omitted here** until they are derived
reproducibly from a provenance-bound audit artifact. An earlier revision of this
paragraph cited "17k INPC findings" and "8 recorded `IsLoaded` findings"; neither
survives checking. `OwnAudit/sts_audit/` is not one canonical snapshot —
`health-report.md` carries an empty commit field and `generated: ?`,
`health-report.json` beside it is a different two-tool run bound to `59e284ba`,
`PLAN.md` quotes a third set again, and the current STS-210 audit lives in
`artifacts/`, which is not committed. The raw per-tool SARIF under `sts_audit/`
is a fine measurement substrate, but a grep over it yields search hits across
repeated runs and `_wpftmp` shadow projects, not findings. A count belongs here
only once it names its source artifact, its selector version, and its result —
and this proposal does not need one: the `IsLoaded` example above already
establishes the semantic gap on its own.

The same three verbs cover the whole family:

Expand Down
Loading