Skip to content

test(templates): restore layout snapshot coverage for shipped presets - #686

Merged
DemchaAV merged 4 commits into
developfrom
test/restore-preset-layout-snapshots
Sep 13, 2026
Merged

DemchaAV merged 4 commits into
developfrom
test/restore-preset-layout-snapshots

Conversation

@DemchaAV

Copy link
Copy Markdown
Owner

Why

33 of the 58 shipped presets had no active layout-snapshot coverage: 16 CV presets, all 15
cover letters, ModernInvoice and ModernProposal. They were gated by pixel tests alone, and
those tolerate 50 000 mismatched pixels, so a shift of a few points in any of them passed
unseen. The suites that once pinned them were deleted along with older template families
(#276, #277, and the v1.6 cleanup 47241c83); their baselines stayed on disk and nothing
read them any more.

backup/template-fixtures-local-work-2026-09-11 held the missing test work, as one commit
on top of v2.2.0 with baselines rendered by v2.2.0. It was reimplemented on current develop
rather than cherry-picked: the fixtures and gates are built from develop's own code, and every
baseline was recorded on this tree.

What changed

  • Coverage restored for 16 CV presets, 15 cover-letter presets, ModernInvoice and
    ModernProposal, through four gates:
    • CvPresetLayoutSnapshotTest and CoverLetterPresetLayoutSnapshotTest read
      CvPresetFixtures and CoverLetterPresetFixtures. Their roster and canonical document
      were moved verbatim out of the two pixel gates, so the pixel gate and the snapshot gate
      render the same presets on the same document. CoverLetterTextLayerGateTest reads the
      letter from its new home.
    • ProposalPresetLayoutSnapshotTest snapshots ModernProposal on the canonical proposal
      its pixel gate already renders.
    • InvoicePresetLayoutSnapshotTest reads a new InvoicePresetFixtures.layoutSnapshotPresets();
      canonicalInvoice(), stressInvoice() and their four consumers are unchanged.
      ClassicInvoice is not listed, because its own suite already pins it on the same invoice
      at the same margin.
  • 33 current baselines added, named <pixel slug>_layout.
  • ShippedPresetLayoutSnapshotCoverageTest: every shipped preset must be rendered by an
    active layout-snapshot suite. The shipped set comes from the templates sources (every
    presets class declaring a public ID), the covered set from the qa sources (presets an
    enabled assertCanonicalSnapshot suite creates directly or through a fixture roster). No
    count and no hand-kept list is asserted, so a preset added without a suite fails it by name.
    A reviewed exemption goes in an allow-list with its reason; the list is empty.
  • 38 orphaned JSON snapshots removed, and 9 dead TemplateTestSupport helpers with
    their imports. Its javadoc now states the coverage rule.

Baseline safety

Every baseline was classified before any was recorded.

  • The 33 current renders were compared with the backup's v2.2.0 baselines:
    19 identical, 14 different.
  • All 14 were explained individually. Each difference was attributed to the develop commit it
    changes at, by rendering on either side of that commit:
Cause Presets What moved
#676, spaced caps set with tracking instead of padded strings classic_serif, sidebar_portrait, mint_editorial, timeline_minimal (CV and letter); part of minimal_underlined, centered_headline, monogram_sidebar heading and name widths only; no coordinate, height or page
d0c946ad (in #633), stacked rows drawn as lists boxed_sections, minimal_underlined, modern_professional, executive, panel (4 fewer nodes each); blue_banner, centered_headline (node kind only) bullet paragraphs become ListNodes
#577, the employer drawn in Monogram Sidebar entries monogram_sidebar 2 added nodes
361ff833 (in #633), a box no longer opens on a page its first unit cannot start on modern_proposal one empty wrapper node
  • C = 0. No suspicious change was found, and no unexplained baseline was accepted.
    Page counts are unchanged in all 14, and every node that moved still sits inside its parent
    and inside the page's content band.
  • The recorded baselines are byte-identical to the renders that were analysed (33 / 33), and
    the 19 identical presets are byte-identical to the backup's baselines too (19 / 19).

ModernProposal is an engine bookkeeping fix, not a pagination regression. The footer note
needs 23.25pt and 15.49pt were left under the acceptance terms. Its paragraph, the only thing
the footer draws, was therefore already on page 1 at v2.2.0, at y 790.64 (the second page;
snapshots count pages from 0). Only ProposalV2ModernFooter[6], the empty page-flow wrapper
around it, still claimed page 0 at y 20.24. It now sits with its paragraph on page 1. Page
count 2 and node count 43 are unchanged, nothing drawn moved, and the pixel baseline recorded
before v2.2.0 still passes.

Coverage

Family Covered
cv 26 / 26
coverletter 15 / 15
invoice 11 / 11
proposal 4 / 4
receipt 1 / 1
rota 1 / 1
total 58 / 58
uncovered 0

On develop before this branch: 25 / 58.

Verification

  • 4 new layout-snapshot gates: 33 / 33
  • affected pixel gates green: CV 16, cover letter 15, invoice 2, proposal 1
  • affected text-layer gates green: CV 52, cover letter 16, invoice 12, proposal 5
  • every *LayoutSnapshotTest in qa plus the coverage test: 74 green
  • templates module: 127 green
  • qa module: 1759 green
  • ./mvnw -B -ntp clean verify: BUILD SUCCESS, 15 / 15 modules, 3258 tests, 0 failures,
    0 errors
    (2 skipped: the pre-existing POSIX assumptions in FileOutputSafetyTest). That is
    develop's 3223 plus the 33 gate tests and the 2 coverage tests.
  • each of the first three commits verified green on its own
  • git diff --check clean

The coverage test was sabotaged and went red each time, naming the right preset: the four new
gates removed (it names exactly the 33), a preset dropped from a roster, a new preset class
with no suite, a suite disabled by a fully qualified @Disabled or no longer calling
assertCanonicalSnapshot, a suite that reads a fixture only for its document, and stale
allow-list entries.

Scope

  • no production code changed
  • no public API changed
  • no pixel baseline changed
  • no documentation changed
  • backup/template-fixtures-local-work-2026-09-11 was not modified
  • this branch is not merged

Lane: test (qa only).

Sixteen layered CV presets and all fifteen cover letters shipped with a pixel
gate and no layout snapshot. The suites that pinned them were deleted with the
classic families in #276 and #277; their baselines stayed behind, and no test
has read them since. The pixel gates tolerate 50 000 mismatched pixels, so a
shift of a few points in any of these presets passes unseen.

- CvPresetFixtures and CoverLetterPresetFixtures hold each family's roster and
  canonical document, moved verbatim out of the two pixel gates, so the pixel
  gate and the snapshot gate render the same presets on the same document. The
  only text change is a javadoc pointer that named the old location.
- CvPresetLayoutSnapshotTest and CoverLetterPresetLayoutSnapshotTest snapshot
  every preset on the roster, as <pixel slug>_layout.
- CoverLetterTextLayerGateTest reads the letter from its new home.

The 31 baselines were recorded by these gates on this tree, after every
difference from the renders these presets last had a snapshot of was
explained. Against their v2.2.0 renders 18 are identical and 13 moved, and
each move traces to a develop commit rendered on either side of it:

- #676, spaced caps set with tracking instead of padded strings: heading and
  name widths only, no coordinate, height or page change. classic_serif,
  sidebar_portrait, mint_editorial, timeline_minimal (CV and letter), and part
  of minimal_underlined, centered_headline and monogram_sidebar.
- d0c946a (in #633), stacked rows drawn as lists: bullet paragraphs become
  ListNodes, four fewer nodes on each of boxed_sections, minimal_underlined,
  modern_professional, executive and panel, and the list half of blue_banner
  and centered_headline.
- #577, the employer drawn in Monogram Sidebar's experience entries: two
  added nodes.

Page counts are unchanged in all 31, and every node that moved still sits
inside its parent and inside the page's content band.
ModernProposal and ModernInvoice had no layout snapshot either; every other
proposal and invoice preset carries a suite of its own.

- ProposalPresetLayoutSnapshotTest snapshots ModernProposal on the canonical
  proposal its pixel gate already renders.
- InvoicePresetFixtures gains layoutSnapshotPresets(), read by
  InvoicePresetLayoutSnapshotTest. It lists ModernInvoice only: ClassicInvoice
  takes the same spec, but ClassicInvoiceLayoutSnapshotTest already pins it on
  the same invoice at the same margin, so listing it would record that page
  twice. canonicalInvoice() and stressInvoice() are unchanged.

modern_invoice_layout is identical to its v2.2.0 render.

modern_proposal_layout differs from its v2.2.0 render in one node:
ProposalV2ModernFooter[6], a bare page-flow wrapper around the footer note,
moves from y 20.24 on pages 0..1 to y 790.64 on page 1. The paragraph inside
it, the only thing it draws, is at y 790.64 on page 1 in both. The note needs
23.25pt and 15.49pt were left under the acceptance terms, so it went to page
1 and the empty wrapper stayed behind claiming page 0 - the defect 361ff83
(in #633) fixed, and the commit this node changes at when rendered on either
side of it. Page count 2 and node count 43 are unchanged, and nothing drawn
moved.
The thirty-three presets restored in the previous two commits went two months
without a layout snapshot and nothing said so. Their suites were deleted along
with another family, their baselines stayed on disk, and every gate that
remained only checked the presets it listed.

ShippedPresetLayoutSnapshotCoverageTest derives the two sets independently
and compares them:

- shipped: every class under a templates presets package that declares a
  public ID, read from the templates sources;
- covered: every preset an active layout-snapshot suite renders, read from the
  qa sources. A suite is a test that calls assertCanonicalSnapshot and is not
  disabled; it covers a preset of its own family it creates directly, or
  through a Stream<Arguments> roster it reads from a fixture. A fixture read
  only for its document covers nothing, and comments and strings are stripped
  before any of this is matched.

Neither side is a hand-kept list and no count is asserted, so a preset added
without a suite fails it by name. An allow-list takes a reviewed exemption
with its reason and is empty; a second test rejects an entry that does not
ship, or that a suite already covers.

Sabotaged, each change confirmed on disk before the run and reverted after:
- the four gates added in this branch removed: fails naming exactly the 33
- mint_editorial dropped from the CV roster: names cv/MintEditorial
- a new preset class with an ID and no suite: names it
- CobaltRota's suite disabled by a fully qualified @disabled, or no longer
  calling assertCanonicalSnapshot: names rota/CobaltRota
- ClassicInvoice's suite rendering another preset while still reading
  InvoicePresetFixtures for its document: names invoice/ClassicInvoice
- an exemption for a covered preset, and one for a preset that does not ship:
  the second test fails, naming each
Thirty-eight baselines under layout-snapshots/canonical-templates had no
reader. Thirty-two lost their suites when the classic families went: the
template_v2_cv and cv-v2/regression files in #277, the template_v2_letter
files in #276. The invoice_standard, proposal_long and four weekly-schedule
layouts were last read by tests removed in the v1.6 cleanup, 47241c8. Nine
TemplateTestSupport helpers that fed those tests had no callers left.

Both removals were proven on this tree immediately before deleting:
- no tracked file names any of the 38 outside the snapshot folders, except
  docs/operations/layout-snapshot-testing.md, whose naming section lists two
  of them as examples beside two names that never existed as files;
- nothing builds their names from parts (template_v2_, regression,
  weekly_schedule), and no test or tool walks the snapshot folders;
- the nine helpers have no qualified call, static import, wildcard import,
  method reference, or string or reflective use; the only calls to them were
  from one another.

TemplateTestSupport now says what the canonical root is for, that a baseline
belongs to the suite that writes it, and that
ShippedPresetLayoutSnapshotCoverageTest holds every shipped preset to it.

Every *LayoutSnapshotTest in qa and the coverage test stay green after the
removal: 34 classes, 74 tests.
@DemchaAV
DemchaAV merged commit 88eb90e into develop Sep 13, 2026
10 checks passed
@DemchaAV
DemchaAV deleted the test/restore-preset-layout-snapshots branch September 13, 2026 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant