test(templates): restore layout snapshot coverage for shipped presets - #686
Merged
Merged
Conversation
Sixteen layered CV presets and all fifteen cover letters shipped with a pixel gate and no layout snapshot. The suites that pinned them were deleted with the classic families in #276 and #277; their baselines stayed behind, and no test has read them since. The pixel gates tolerate 50 000 mismatched pixels, so a shift of a few points in any of these presets passes unseen. - CvPresetFixtures and CoverLetterPresetFixtures hold each family's roster and canonical document, moved verbatim out of the two pixel gates, so the pixel gate and the snapshot gate render the same presets on the same document. The only text change is a javadoc pointer that named the old location. - CvPresetLayoutSnapshotTest and CoverLetterPresetLayoutSnapshotTest snapshot every preset on the roster, as <pixel slug>_layout. - CoverLetterTextLayerGateTest reads the letter from its new home. The 31 baselines were recorded by these gates on this tree, after every difference from the renders these presets last had a snapshot of was explained. Against their v2.2.0 renders 18 are identical and 13 moved, and each move traces to a develop commit rendered on either side of it: - #676, spaced caps set with tracking instead of padded strings: heading and name widths only, no coordinate, height or page change. classic_serif, sidebar_portrait, mint_editorial, timeline_minimal (CV and letter), and part of minimal_underlined, centered_headline and monogram_sidebar. - d0c946a (in #633), stacked rows drawn as lists: bullet paragraphs become ListNodes, four fewer nodes on each of boxed_sections, minimal_underlined, modern_professional, executive and panel, and the list half of blue_banner and centered_headline. - #577, the employer drawn in Monogram Sidebar's experience entries: two added nodes. Page counts are unchanged in all 31, and every node that moved still sits inside its parent and inside the page's content band.
ModernProposal and ModernInvoice had no layout snapshot either; every other proposal and invoice preset carries a suite of its own. - ProposalPresetLayoutSnapshotTest snapshots ModernProposal on the canonical proposal its pixel gate already renders. - InvoicePresetFixtures gains layoutSnapshotPresets(), read by InvoicePresetLayoutSnapshotTest. It lists ModernInvoice only: ClassicInvoice takes the same spec, but ClassicInvoiceLayoutSnapshotTest already pins it on the same invoice at the same margin, so listing it would record that page twice. canonicalInvoice() and stressInvoice() are unchanged. modern_invoice_layout is identical to its v2.2.0 render. modern_proposal_layout differs from its v2.2.0 render in one node: ProposalV2ModernFooter[6], a bare page-flow wrapper around the footer note, moves from y 20.24 on pages 0..1 to y 790.64 on page 1. The paragraph inside it, the only thing it draws, is at y 790.64 on page 1 in both. The note needs 23.25pt and 15.49pt were left under the acceptance terms, so it went to page 1 and the empty wrapper stayed behind claiming page 0 - the defect 361ff83 (in #633) fixed, and the commit this node changes at when rendered on either side of it. Page count 2 and node count 43 are unchanged, and nothing drawn moved.
The thirty-three presets restored in the previous two commits went two months without a layout snapshot and nothing said so. Their suites were deleted along with another family, their baselines stayed on disk, and every gate that remained only checked the presets it listed. ShippedPresetLayoutSnapshotCoverageTest derives the two sets independently and compares them: - shipped: every class under a templates presets package that declares a public ID, read from the templates sources; - covered: every preset an active layout-snapshot suite renders, read from the qa sources. A suite is a test that calls assertCanonicalSnapshot and is not disabled; it covers a preset of its own family it creates directly, or through a Stream<Arguments> roster it reads from a fixture. A fixture read only for its document covers nothing, and comments and strings are stripped before any of this is matched. Neither side is a hand-kept list and no count is asserted, so a preset added without a suite fails it by name. An allow-list takes a reviewed exemption with its reason and is empty; a second test rejects an entry that does not ship, or that a suite already covers. Sabotaged, each change confirmed on disk before the run and reverted after: - the four gates added in this branch removed: fails naming exactly the 33 - mint_editorial dropped from the CV roster: names cv/MintEditorial - a new preset class with an ID and no suite: names it - CobaltRota's suite disabled by a fully qualified @disabled, or no longer calling assertCanonicalSnapshot: names rota/CobaltRota - ClassicInvoice's suite rendering another preset while still reading InvoicePresetFixtures for its document: names invoice/ClassicInvoice - an exemption for a covered preset, and one for a preset that does not ship: the second test fails, naming each
Thirty-eight baselines under layout-snapshots/canonical-templates had no reader. Thirty-two lost their suites when the classic families went: the template_v2_cv and cv-v2/regression files in #277, the template_v2_letter files in #276. The invoice_standard, proposal_long and four weekly-schedule layouts were last read by tests removed in the v1.6 cleanup, 47241c8. Nine TemplateTestSupport helpers that fed those tests had no callers left. Both removals were proven on this tree immediately before deleting: - no tracked file names any of the 38 outside the snapshot folders, except docs/operations/layout-snapshot-testing.md, whose naming section lists two of them as examples beside two names that never existed as files; - nothing builds their names from parts (template_v2_, regression, weekly_schedule), and no test or tool walks the snapshot folders; - the nine helpers have no qualified call, static import, wildcard import, method reference, or string or reflective use; the only calls to them were from one another. TemplateTestSupport now says what the canonical root is for, that a baseline belongs to the suite that writes it, and that ShippedPresetLayoutSnapshotCoverageTest holds every shipped preset to it. Every *LayoutSnapshotTest in qa and the coverage test stay green after the removal: 34 classes, 74 tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
33 of the 58 shipped presets had no active layout-snapshot coverage: 16 CV presets, all 15
cover letters,
ModernInvoiceandModernProposal. They were gated by pixel tests alone, andthose tolerate 50 000 mismatched pixels, so a shift of a few points in any of them passed
unseen. The suites that once pinned them were deleted along with older template families
(#276, #277, and the v1.6 cleanup
47241c83); their baselines stayed on disk and nothingread them any more.
backup/template-fixtures-local-work-2026-09-11held the missing test work, as one commiton top of v2.2.0 with baselines rendered by v2.2.0. It was reimplemented on current
developrather than cherry-picked: the fixtures and gates are built from develop's own code, and every
baseline was recorded on this tree.
What changed
ModernInvoiceandModernProposal, through four gates:CvPresetLayoutSnapshotTestandCoverLetterPresetLayoutSnapshotTestreadCvPresetFixturesandCoverLetterPresetFixtures. Their roster and canonical documentwere moved verbatim out of the two pixel gates, so the pixel gate and the snapshot gate
render the same presets on the same document.
CoverLetterTextLayerGateTestreads theletter from its new home.
ProposalPresetLayoutSnapshotTestsnapshotsModernProposalon the canonical proposalits pixel gate already renders.
InvoicePresetLayoutSnapshotTestreads a newInvoicePresetFixtures.layoutSnapshotPresets();canonicalInvoice(),stressInvoice()and their four consumers are unchanged.ClassicInvoiceis not listed, because its own suite already pins it on the same invoiceat the same margin.
<pixel slug>_layout.ShippedPresetLayoutSnapshotCoverageTest: every shipped preset must be rendered by anactive layout-snapshot suite. The shipped set comes from the templates sources (every
presetsclass declaring a publicID), the covered set from the qa sources (presets anenabled
assertCanonicalSnapshotsuite creates directly or through a fixture roster). Nocount and no hand-kept list is asserted, so a preset added without a suite fails it by name.
A reviewed exemption goes in an allow-list with its reason; the list is empty.
TemplateTestSupporthelpers withtheir imports. Its javadoc now states the coverage rule.
Baseline safety
Every baseline was classified before any was recorded.
19 identical, 14 different.
changes at, by rendering on either side of that commit:
d0c946ad(in #633), stacked rows drawn as listsListNodes361ff833(in #633), a box no longer opens on a page its first unit cannot start onPage counts are unchanged in all 14, and every node that moved still sits inside its parent
and inside the page's content band.
the 19 identical presets are byte-identical to the backup's baselines too (19 / 19).
ModernProposal is an engine bookkeeping fix, not a pagination regression. The footer note
needs 23.25pt and 15.49pt were left under the acceptance terms. Its paragraph, the only thing
the footer draws, was therefore already on page 1 at v2.2.0, at y 790.64 (the second page;
snapshots count pages from 0). Only
ProposalV2ModernFooter[6], the empty page-flow wrapperaround it, still claimed page 0 at y 20.24. It now sits with its paragraph on page 1. Page
count 2 and node count 43 are unchanged, nothing drawn moved, and the pixel baseline recorded
before v2.2.0 still passes.
Coverage
On
developbefore this branch: 25 / 58.Verification
*LayoutSnapshotTestin qa plus the coverage test: 74 green./mvnw -B -ntp clean verify: BUILD SUCCESS, 15 / 15 modules, 3258 tests, 0 failures,0 errors (2 skipped: the pre-existing POSIX assumptions in
FileOutputSafetyTest). That isdevelop's 3223 plus the 33 gate tests and the 2 coverage tests.
git diff --checkcleanThe coverage test was sabotaged and went red each time, naming the right preset: the four new
gates removed (it names exactly the 33), a preset dropped from a roster, a new preset class
with no suite, a suite disabled by a fully qualified
@Disabledor no longer callingassertCanonicalSnapshot, a suite that reads a fixture only for its document, and staleallow-list entries.
Scope
backup/template-fixtures-local-work-2026-09-11was not modifiedLane: test (qa only).