Skip to content

formal: end-to-end stream correctness of the incremental reduce model - #777

Merged
frankmcsherry merged 1 commit into
TimelyDataflow:master-nextfrom
frankmcsherry:reduce-model-lean
Jul 2, 2026
Merged

formal: end-to-end stream correctness of the incremental reduce model#777
frankmcsherry merged 1 commit into
TimelyDataflow:master-nextfrom
frankmcsherry:reduce-model-lean

Conversation

@frankmcsherry

Copy link
Copy Markdown
Member

Adds formal/TARGETS2.md (the incremental model, in prose) and formal/Differential/Model.lean (the model in Lean, with its goal theorem proved).

The model (TARGETS2.md, following on from #776's per-round Coverage.lean): rounds delimited by antichain frontiers; a truncated join-closure enumeration (joins beyond the round's frontier are deferred to a pending set and closed no further); and logical compaction modeled adversarially — between rounds the stored input and output may be re-represented arbitrarily (times advanced, merged, cancelled; support added or dropped), subject only to preserving accumulations at or beyond the frontier. advance_by plus consolidation is one instance (acc_mapDomain, with #776's Compaction.advance_le_iff).

The theorem (Model.Run.stream_correct, quantified as Model.streamCorrectness_holds): every run — every choice of rounds, frontiers, compliant emissions, and adversary — emits corrections that accumulate to the reduction of the true input at every finalized time. Unconditional; the only hypothesis anywhere is f 0 = 0. Once the frontier empties, the emitted stream is the difference trace of the output (update_ext, Run.emitted_unique). TARGETS2's cancellation scenario is checked end to end as a concrete two-round run (scenario1Run), and a nonvacuity witness guards the quantified statements.

One amendment, recorded in TARGETS2.md step 2: the enumeration's ground set includes the stored (post-consolidation) output support, as the implementation's enumeration does. The pending coverage is then re-derived each round from the current representatives (staleness_rederived), so nothing is carried across an adversary boundary and the re-representation question dissolves (transfer_always). Whether the output-free enumeration also suffices — the empirical finding that output times are not needed for determination — remains open, per the amendment note; it is separately backed by ~48M adversarial rounds of a decidable per-round check in an executable twin of this model.

Checking: cd formal && lake build (no sorries); #print axioms Model.streamCorrectness_holds reports only propext, Classical.choice, Quot.sound. The file header carries a sentence-by-sentence correspondence table to TARGETS2.md.

🤖 Generated with Claude Code

… model

Adds Differential/Model.lean and formal/TARGETS2.md. TARGETS2.md states the
incremental model in prose: rounds delimited by antichain frontiers, a
truncated join-closure enumeration with a carried pending set, and logical
compaction modeled as an adversary that may re-represent the stored input
and output arbitrarily, subject only to preserving accumulations at or
beyond the frontier. Model.lean renders it sentence by sentence and proves
the goal theorem unconditionally (Model.Run.stream_correct,
Model.streamCorrectness_holds): every run, under every such adversary,
emits corrections that accumulate to the reduction of the true input at
every finalized time, and once the frontier empties the emitted stream is
the unique difference trace of the output.

One amendment to the model, recorded in TARGETS2.md step 2: the
enumeration's ground set includes the stored (post-consolidation) output
support, as the implementation's does. The pending coverage is then
re-derived each round from current representatives and never crosses the
adversary boundary. Whether the output-free enumeration also suffices
remains open, per the amendment note.

TARGETS2.md's cancellation scenario is checked end to end (scenario1Run),
and the file builds with no sorries on the standard axioms.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@frankmcsherry
frankmcsherry merged commit b519c4a into TimelyDataflow:master-next Jul 2, 2026
6 checks passed
@frankmcsherry
frankmcsherry deleted the reduce-model-lean branch July 2, 2026 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant