Skip to content

Record which rule produced each braille cell - #207

Open
owjs3901 wants to merge 53 commits into
mainfrom
owjs3901/rule-trace
Open

owjs3901 wants to merge 53 commits into
mainfrom
owjs3901/rule-trace

Conversation

@owjs3901

Copy link
Copy Markdown
Contributor

Adds an opt-in encode_with_trace that returns, for every output cell, the rule that wrote it.

Behaviour is unchanged: fixtures 5141/5141, corpus 455,975/467,121 (97.6139%), marker bench 837/145/305/398.

Across all 467,112 corpus sentences the trace explains every one of the 88,927,183 cells. The remaining gap is the capitals/grade-1 indicators on the pure-UEB path, pinned in a test rather than absorbed into a catch-all slot.

Asking why a word came out the way it did meant reading the rule engine and
guessing. The encoder now offers to say so itself: encode_with_trace returns
the cells alongside a Trace, and the Trace names, for every cell, the rule
that wrote it.

Tracing is opt-in. encode never builds a Trace, so the untraced path keeps
its shape, and the numbers say the same: fixtures stay 5141 of 5141, the
corpus stays at 455,975 of 467,121, and the marker bench still reads
837 / 145 / 305 / 398.

Attribution is partitioned by the engine that owns the input, because the
encoder is really several engines and a caller must be able to tell an
uninstrumented one from a rule that declined to fire. Korean syllables are
split to the article rather than reported as one composite entry, so 안 names
제6항 for its vowel and 제3항 for its 받침 instead of naming the syllable rule
twice. A rule that matched and then skipped produced nothing and is not
recorded: that it ran and that it explains the output are different claims.

Cells no rule object writes are still accounted for. The blank between two
words, a pre-encoded run whose token rule declared no article, and 제29항's
roman indicator, continuation and terminator each name themselves through
the emitter, so an unexplained cell means a genuine gap rather than a
structural one.

Over all 467,112 corpus sentences the trace now explains every one of the
88,927,183 cells. The one shape that had been slipping through was 제35항's
numeric bridge resuming into a lowercase a-j, where UEB 6.5.2 makes the
emitter write a continuation cell — 298 of them, plus two roman indicators
on the same path.

What remains unexplained is the capitals and grade-1 indicators on the
pure-UEB path, where attribution places whole attempts of the contraction
search and an indicator belongs to no attempt. A Korean document never
reaches it; exactly one corpus sentence takes that path at all. The counts
are pinned in a test rather than absorbed into a catch-all slot, because a
slot that swallows anything unclaimed would make the gap unmeasurable.
@github-actions

github-actions Bot commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Braillify testcase report

Suite Passed Total Failed Success rate
Standard testcases 5159 5159 0 100.00%
NIKL 2021 corpus 91276 93115 1839 98.03%
NIKL 2022 corpus 107874 108795 921 99.15%
NIKL 2023 corpus 122702 126693 3991 96.85%
NIKL 2024 corpus 53497 54990 1493 97.28%
NIKL 2025 corpus 80676 83528 2852 96.59%
NIKL corpus (all years) 456025 467121 11096 97.62%

Command: cargo test test_by_testcase -- --nocapture

devfive added 28 commits September 21, 2026 14:56
The linux gate wants every line reached, and the tracing work left two
stretches untested: the registry lookups for the jamo articles, the emitter
slots and an index past its engine's partition, and the whole trace shim in
the node package.

The shim's per-event conversion and its two label tables are now named
functions, because a match arm inlined in a closure can only be reached by
finding an input that produces it — and the math path, for one, no Korean
sentence reaches.
The remaining gap was the traced side of guards no test entered: a rule that
matches and then skips, the token engine's rewrite shapes, the forced UEB
encode, and the math route both when it lands and when it is thrown away.
Each now has a test that goes through the public entry point where one exists.

Two of them no input could reach. The encoder only dropped its origin table
when a transform had already changed the token count, which the traced path
never does, and the emitter built a fallback rule id inside a branch that the
same reasoning made dead. Both now sit in code that every call runs, so what
was unreachable is gone rather than merely excused.

The encoder still emits exactly what it did: fixtures 5141 of 5141, corpus
455,975 of 467,121, marker bench 837 / 145 / 305 / 398.
The spaced ampersand of 제29항 asks whether a Roman word stands on each side,
and the two answers it can reach without finding one had no test: already
encoded output, which proves nothing about what it holds, and the end of the
token stream, which proves nothing at all.
Article 51's body parts a 표제 from its 내용 with a 쌍점 written against the
label and followed by one blank, and the rule already did that — but only when
Korean stood on both sides of the colon. What the 내용 happens to be written in
was never what the article turned on, so `모델명:PN50` and `일시:2006년` were
being run straight on.

The test stays on the left of the colon. Nothing Korean in front of it means
the mark was never a 쌍점 at all but a sign inside a Roman identifier, which is
how `NVH:Noise` and `A:IR` keep running on. [다만 2]'s exceptions are untouched
for the same reason: `오전 10:20` and `요한 3:16` have a figure in front, and
`청군:백군` is still caught by the 대비 쌍 test above.

25 more corpus sentences read correctly, 456,000 of 467,121. The marker bench
reads 838 / 145 / 305 / 398 against 837 / 145 / 305 / 398, one more error over
66 more sentences that now line up word-for-word and enter the comparison at
all (8,572 to 8,638). On the shared sentences the markers are unchanged.
A 가운뎃점 between figures names an event or an issue — 제주4·3, 광주5·18,
통권 제54·55·56호 — and 제5항 writes it ⠐⠆. Print often sets the Korean that
names them against the figures, and the detector judged the whole word, so the
Korean prefix made it fail the numeric test and the word fell through to the
math route, where the dot became a product and 제11항 wrapped it in two blanks.

The prefix is not noise in that judgement; it is the evidence. Reading past it
and judging the figures alone leaves 제주4·3운동 and 제54·55·56호 exactly as they
were, since neither reached the math route to begin with.

25 more corpus sentences read correctly, 456,025 of 467,121, with the marker
bench unmoved at 838 / 145 / 305 / 398.
The tracer answers which rule wrote each braille cell, but sixteen rules
answered with a placeholder rather than an article: two carried a word where a
number belongs, and the rest inherited the trait default that exists precisely
to say nobody has checked yet.

rule_map.json at the repo root settles them. It holds all 448 articles with
their text, so each rule was matched against the one whose wording describes
what it does — 제46항 for the operator spacing that gives rule_math its name,
제53항 for the ellipsis, 제47항 for both fraction detectors, 제60항 for the
asterisk, 제54항 for the quote attachment and for the tortoise-shell gloss that
turns out to be the same bracket rule, 제35항 for the digital notation, 제19항
for the head of the 옛 글자 articles the middle-Korean detector switches into,
제49항 for the two dash and bracket spacing rules added earlier, and RUEB 8.4
for the capitals run, which no Korean article governs.

Two take the honest marker the emitter's inter-word blank already uses: the
space rule encodes the blank cell itself, and the LaTeX merge only joins a
formula split across blanks and writes nothing.

Over 5,160 traced sentences the placeholders fall from 4,331 to 2,378, and what
remains is the math symbol dispatch, which needs an article per branch rather
than one for the whole chain. Behaviour is untouched: fixtures 5141 of 5141,
corpus 456,025 of 467,121, marker bench 838 / 145 / 305 / 398.
The English path wrote its indicators straight into the output without telling
the tracer, so every capital sign and grade-1 sign belonged to no rule at all.
`A1` left one cell unexplained and `Q50 2.2d` left three, and the landing
page had to show those cells as coming from nowhere.

The indicators now carry their RUEB sections: the capital-letter sign is 8.3,
the capitalised-word sign is 8.4, and the grade-1 sign is 5. Spaced numeric
output such as the `2.2` in `Q50 2.2d` takes 6.

Recording them is not simply another attempt. `settle_word_attribution`
credits a spelled-out word to 4.1 only while `attempt_count()` has not moved
since the word began, so booking indicators as attempts would have silently
stripped attribution from the very words they decorate. Indicators are
therefore collected on their own cursor, counted separately, and subtracted
from the broad fallback ranges during alignment so no cell is claimed twice.

The test that pinned the gap open is now three tests that pin it shut, and a
capitalised spelled-out word guards the hazard above.

Attribution only: no output cell changes. Fixtures 5141 of 5141, corpus
456,025 of 467,121, marker bench 838 / 145 / 305 / 398.
The math engine answered the tracer with a placeholder 2,378 times over 5,160
traced sentences. One rule, MathSymbolRule, dispatches more than thirty symbols
to as many different articles, so a single article per rule could never be
honest: the arrow arm follows 제10항, the set arm 제60항, the quantifier arm
제61항, and one number had to stand for all of them.

A rule may now own several registry slots. It declares the extra articles in
variant_metas() and returns the one it actually used through ConsumedWithMeta,
carrying the static itself rather than an index that would change meaning the
moment the list is reordered. Dispatch resolves it by pointer identity within
that rule's own declarations and refuses anything undeclared, so a rule cannot
borrow a neighbour's article by accident.

The flattening forced a second fix. Ids came from the rule's physical position
in the dispatch vector, which stops matching the registry as soon as one rule
occupies several slots; every rule after MathSymbolRule would have resolved to
someone else's metadata. Dispatch now tracks the flattened base instead.

The 221 shortcut characters carry their article in the same record as their
cells, so the two cannot drift apart. Grouping them by article also made the
table say which rule governs what, which it never did before. Seven symbols
keep the honest placeholder because the standard does not name them.

Two comments were simply wrong and are corrected: 제26항 is 행렬, not the
product sign, and @9 is 제61항 1 부정, not 닮음, which is ,' under 제42항.

Every one of the 221 character-to-cell mappings is unchanged, checked pair by
pair against the previous table, and output bytes are identical across the
fixtures and all 467,121 corpus rows. Fixtures 5141 of 5141, corpus 456,025,
marker bench 838 / 145 / 305 / 398. Placeholders fall from 2,378 to 11.
Seven shortcut characters kept the honest placeholder because searching the
standard's text for the character itself found nothing. That was the wrong
search. The standard prints its examples in the internal braille notation, not
in the Unicode characters an editor types, so a symbol is found by matching the
cells it produces against the notation the article prints.

Matched that way, five of the seven name their article plainly. The fraction
slash writes ⠌, which 제7항 1 calls the 분수표 and prints as /. Script R and
the not-similar sign write ⠠⠗ and ⠨⠈⠔, which 제34항 prints as ,R and .@9
for 관계가있다 and 관계가없다. The superscript c writes ⠘⠉, printed as ^c for
여집합 in 제60항 5. The not-implies sign writes ⠨⠒⠒⠕, printed as .33O for
항진명제의 부정 in 제61항 4.

The same reading corrects 제34항's own description: it is the relation-symbol
article, and the negation mark already filed under it belongs to its second
clause, not to some separate rule about negation.

Two symbols still have no article and now say why. ∏ carries the cells of
Greek capital pi, which invites filing it under 제13항 by resemblance; the
standard never mentions it, so it stays unattributed on purpose. ⸩ stands in
for LaTeX's \right., a null delimiter with nothing printed for an article to
govern.

Only the article attached to each entry moved; all 221 character-to-cell
mappings were compared pair by pair and are unchanged. Placeholders over 5,160
traced sentences fall from 11 to 3, all three now the null delimiter.
A \$...\$ span standing alone is a formula and the math engine owns it. The
same span inside English prose stays on the UEB side, where the parser turns it
into a Technical token and rule_11 encodes it. The engine emitted those cells
and told the tracer nothing, so a sentence like \�bc \{D}\$\ explained its
first four cells and left the other six belonging to no rule at all. Where a
whole line was formula and prose together, nothing at all was explained.

RUEB calls this code switching: 14.6.2 is the short inline fragment among
ordinary text, which is exactly this token's shape. 14.6.3 covers the long
terminal passage and is not what the parser produces here.

The cells are recorded on their own channel rather than as a contraction
attempt, because a word is credited to 4.1 only while the attempt count has not
moved since it began; booking these as attempts would have stripped the
attribution from neighbouring spelled-out words. Alignment places the direct
spans first and then searches for word and indicator cells outside them, so no
cell is claimed twice.

Two smaller holes closed with it. The code-switch encoder may try a span, emit
records, then refuse the input and hand it back; those records used to survive
into a trace they no longer described, and are now rolled back to a checkpoint.
And the symbol arms that finish through a shared continue never settled their
pending output, which is why fullwidth = and + went unexplained.

Attribution only: no output cell changes. Over 5,160 traced sentences the
unexplained cells fall from 1,892 to 449 and fully explained sentences rise from
5,119 to 5,127. Fixtures 5141 of 5141, corpus 456,025, marker bench
838 / 145 / 305 / 398.
Chemistry and engineering papers reach for code points the standard names but
this table did not carry, so eight otherwise ordinary lines failed to transcribe
at all rather than producing a single wrong cell.

Two are the same symbol drawn differently. A reaction arrow is written long,
U+27F6, where geometry writes U+2192; an n-ary product is written U+2A09 where
arithmetic writes U+00D7. Same meaning, so same cells and same article as the
form already present — anything else would report two articles for one role
depending on which glyph an author typed.

The third was a gap rather than a variant: 제27항 defines 나누어떨어진다 as \
and 나누어떨어지지않는다 as .\, but only the negated form was here. The plain
sign is now its undotted counterpart, which a test pins so the pair cannot drift.

Nothing already encodable changes: all 221 previous character-to-cell mappings
were compared pair by pair and are untouched, and no line that transcribed
before transcribes differently. Fixtures 5141 of 5141, corpus 456,025, marker
bench 838 / 145 / 305 / 398. Over the 5,160 traced papers, failures fall from
18 to 10.
Engineering papers write resistance with U+2126 OHM SIGN rather than U+03A9
GREEK CAPITAL OMEGA. Unicode declares the two canonically equivalent — the ohm
sign decomposes to omega and nothing else — so they are one character wearing
two code points, and no transcription may tell them apart.

We told them apart. Five papers carrying \[Ω]\ failed outright while the same
text with capital omega transcribed fine. The English side never saw the problem
because it normalises before it reads, which is why \R[Ω]\ already worked while
a bare \Ω\ did not.

The sign now shares omega's cells and article. Folding it in the pipeline
instead would have meant normalising ahead of the NFD step 제65항 5 relies on for
accented Latin, for a gain of exactly this one character: across the 5,160
papers and all 5,141 fixtures, the ohm sign is the only thing normalisation
would have touched.

Nothing already encodable changes: the previous 224 mappings were compared pair
by pair and are untouched, and no line that transcribed before transcribes
differently. Fixtures 5141 of 5141, corpus 456,025, marker bench
838 / 145 / 305 / 398. Failures over the traced papers fall from 10 to 5.
제49항 defines one 홑낫표. Unicode spells it twice, 「 」 at full width and
「 」 at half width, and a statute quoted in a survey paper used the half-width
pair, so the line failed to transcribe while the same words in the full-width
pair transcribed fine.

The standard's own prose writes it half width — 「통일영어점자 규정」 appears
that way in the articles we transcribe from — so refusing the form is refusing
the regulation's own typography.

Both spellings now take 제49항's cells, pinned by a test that compares the pair
rather than restating the cells, so the two can never drift apart.

Nothing already encodable changes: no line that transcribed before transcribes
differently. Fixtures 5141 of 5141, corpus 456,025, marker bench
838 / 145 / 305 / 398. Failures over the 5,160 traced papers fall from 5 to 4.

The remaining four are not gaps we may close by inference. Circled capitals
Ⓐ Ⓑ need 제64항's wrapping combined with 제28항's capital sign, and the article
demonstrates only the lowercase ⓐ, leaving the order of the two indicators
undetermined; that is now a question for 국립국어원 rather than a guess. ≫, ℧
and ℑ appear nowhere in the standard at all.
Every rule the tracer can credit should cite the article it implements, and the
last few commits settled them one family at a time by reading each file. Reading
is how the last one was missed: a search for the placeholders I knew about --
"?", "math", "space" -- cannot find a placeholder nobody thought to look for.

The registry now answers instead. Walking every engine's registered rules and
demanding each section be an article number, a dotted RUEB section, or "-" for
output the standard prescribes without giving it an article, the test found
unicode_fraction_encoding filed under the section "fraction". 한글 제47항 governs
it, and says so with the very character class the rule handles: "분수는 분수표
/을 사용하여 분모, 분수표, 분자 순으로 적고", worked through as ⅔ → #c/#b.

Corpus measurement would never have caught it. The 5,160 traced papers write
their fractions in LaTeX, so this rule never fires there; the defect was real
and silent at the same time.

The one placeholder that remains is listed by name rather than skipped as a
class, so adding a new undeclared rule turns the test red -- and so does
retiring the last placeholder, which should be a deliberate edit rather than a
quiet pass. It stands for the n-ary product sign, which the standard never
mentions, and the right double parenthesis, which represents a LaTeX delimiter
that prints nothing.

No output changes. Fixtures 5141 of 5141, corpus 456,025, marker bench
838 / 145 / 305 / 398.
수학 제35항 to 제39항 give one mark each and in order: 선분 @c, 호 @[,
직선 [3O, 반직선 3O, 각 ?. Three of our marks cited the article next door.

The overline that spans two points is 제35항's segment bar; it sat under
제36항, which is the arc. The two-headed arrow drawn above a pair is 제37항's
line; it sat under 제38항. The single-headed one is 제38항's ray -- an article
whose 붙임 also lends it to vectors -- and it sat under 제39항, which is the
angle and nothing else.

Nothing caught this because the cells were right either way. A reader checking
the transcription against the standard would have been sent to an article that
does not mention the mark in front of them, which is the whole failure the
tracer exists to prevent.

제23항 was checked and left alone. It gives the bar over a variable, 켤레
복소수 and 평균값, the very same @c cells as the segment bar, so the two are
told apart by code point alone: a combining or spacing macron marks a
variable, the overline spans a pair of points. A test now says so, because the
duplication otherwise looks like something to tidy away.

The 제35항 slot also had to be declared in the rule's variant list. Reporting
an article a rule never declared is refused by design, and it refused this one
-- a fixture failed the moment the article moved without the declaration,
which is the invariant doing exactly its job.

Attribution only: all 225 character-to-cell mappings were compared pair by pair
and are unchanged. Fixtures 5141 of 5141, corpus 456,025, marker bench
838 / 145 / 305 / 398.
The Linux coverage gate names ten lines nothing reaches. Two of them are
reachable behaviour that simply had no test.

제25항 writes a summation's bounds as a group and then leaves a blank before
the body, but only when the body runs straight into it -- a summation already
followed by a space, or one ending the expression, must not gain a second
blank. The blank itself was never exercised.

제53항 reads a middle dot as the multiplication sign when the same expression
also composes arithmetically, which is how derivative and product formulas are
written. Nothing built a token stream that put a middle dot beside an equals
sign, so the test for that condition never ran.

Both are driven through the rule directly rather than through a written
expression, because the parser reaches these token shapes only from inputs that
would exercise a dozen other rules at the same time and prove nothing about
these two.

The remaining eight lines are not missing tests: they are a blank line, a
closing brace, a method signature, two fields of a placeholder static, and a
step in the middle of an iterator chain. They need the instrumentation looked
at rather than more assertions, and that has to be read off CI because
tarpaulin cannot run here -- the workspace needs a system Python for pyo3, and
this platform's recorder miscounts by design.

No output changes. Fixtures 5141 of 5141, corpus 456,025, marker bench
838 / 145 / 305 / 398.
제64항 wraps a circled character in 7 7 and works the lowercase case through:
ⓐ is 70a7, the roman sign then the letter. It never shows a capital, so the
order of the roman sign and the capital sign was undetermined and four readings
all fitted the wording. Rather than pick one, the question went to 국립국어원,
who answered on 2026-09-21: the roman sign first, then the capital sign, then
the letter.

Ⓐ is therefore ⠶⠴⠠⠁⠶. A test pins the order with that provenance written
down, because nothing in the article can be used to check it.

The gate that decides whether a character is an enclosed symbol at all listed
only digits and lowercase letters, so the capitals never reached the encoder
even to fail there. They do now.

Exam papers label their choices Ⓐ Ⓑ, and a paper carrying them previously
failed to transcribe as a whole. Fixtures 5141 of 5141, corpus 456,025, marker
bench 838 / 145 / 305 / 398 all unchanged.
The standard names the summation in 수학 제25항 and writes it ,.S, which is
Greek capital sigma. Nothing names the product. We wrote it ,.P anyway -- Greek
capital pi, by analogy -- and marked the article unknown, which reads as an
article we merely have not found yet rather than an extension we invented.

국립국어원 answered on 2026-09-21: the product sign cannot be transcribed. So
the table no longer carries it, and an expression containing it is refused
instead of quietly given cells the standard never granted.

The cells being exactly Greek capital pi's is what made borrowing them look
reasonable, so a test records that and asserts the character is absent, or the
gap invites the same repair again.

The summation is untouched and still cites 수학 제25항. Fixtures 5141 of 5141,
corpus 456,025, marker bench 838 / 145 / 305 / 398.
수학 제38항 writes a ray as 3o,,AB and 제37항 writes a line as [3O,,AB: the
arrow ahead of both capitals, attached. 제10항 writes the arrows that stand
between things -- right, left, up, down and the four diagonals -- and lists the
right arrow among them. We sent every right arrow to 제38항 and every
two-headed arrow to 제37항, so a reaction equation and an ordinary mapping both
claimed to be geometry.

The give-away was in our own table: the left, up, down and diagonal arrows all
sat under 제10항 while the right arrow sat alone under 제38항, which is not a
shape the standard has.

The dispatch now asks what the standard's own notation asks: does the arrow
come ahead of two capitals with nothing before it. If it does, it is drawn over
them and the geometry articles apply. Otherwise it is standing between two
things and 제10항 does.

국립국어원 answered on 2026-09-21 that a chemical reaction arrow follows
과학점자규정 제18항, whose text spells the same cells (+ 5, → 3o, ← {3, ⇄ [7O).
Telling a reaction equation from any other standing arrow needs a chemistry
signal the math engine does not carry, so reaction arrows now reach 제10항
rather than the geometry article they had before -- closer, and honest about
what we can determine. The science article is recorded as remaining work.

Limits are untouched: the arrow in \lim_{n \to \infty} never reaches this rule.

No output changes -- both branches call the same encoder and all 224
character-to-cell mappings were compared pair by pair. Since the cells are
identical either way, only an article assertion can catch a regression here,
and one now does. Fixtures 5141 of 5141, corpus 456,025, marker bench
838 / 145 / 305 / 398.
Removing the n-ary product from the symbol table left nine integration
snapshots still expecting cells for it. They failed on every platform, and I
did not see it because I ran the suite with --lib for the preceding commits,
which skips tests/ entirely. The project's gate is the whole suite, and it says
so; I narrowed it and this is what that cost.

The snapshots record the Ok/Err shape on purpose -- the module says so at the
top -- so the fix is to let them record the refusal rather than to delete the
cases. Each of the nine now reads err: Invalid character, and nothing else in
the two snapshot files moved.

The Π(a,b) dispatch is untouched. It matches Greek capital pi, U+03A0, not the
product sign, and keeps its own unit test.

Whole suite: 5253 + 20 + 8 + 351 + 162 passing, tests/ included this time.
I set out to delete the cell that `\right.` emits. 국립국어원 ruled that nothing
printed means nothing transcribed, `\right.` draws no delimiter, and the cell
sat under a placeholder article -- every sign pointed one way. Deleting it broke
a regulation fixture.

수학 제6항 1 lists the brackets and includes 연립식 괄호, opening `7'` and
closing `,7`. `7'` is two cells, ⠶⠄, and LaTeX writes that opening half as
`\left\{ ... \right.`. So the cell is the brace's second half, and the sentinel
standing for `\right.` carries it. The matrix code already said as much in a
comment about `\begin{cases}`; I did not read it before cutting.

The sentinel now cites 제6항 instead of the placeholder, which is what it should
have cited all along. A test records why a thing named after a right delimiter
belongs to the bracket article, so the same deletion is not attempted again.

With that, no registered math rule keeps the placeholder and no shortcut
carries an unknown article. Over 5,160 traced papers the tracer reports an
article for every cell it explains -- the count of unknown articles reaches
zero. Output is untouched: all 224 mappings compared pair by pair, no traced
line transcribes differently, fixtures 5141 of 5141, corpus 456,025, marker
bench 838 / 145 / 305 / 398.
한글 제53항 writes the ellipsis in prose. 수학 제12항 [붙임 1] claims it back
inside an expression -- "쉼표는 " 으로 적고, 줄임표는 ,,, 으로 적는다" -- and
국립국어원 confirmed on 2026-09-21 that an ellipsis in a formula follows the
math standard.

Both write the same three cells, so nothing in the output could have shown the
wrong choice. The rule for an ellipsis met inside math cited the Korean
article, which also rendered as 수학 제53항 in the trace, and 수학 제53항 is
the derivative.

Article 12 is titled 로마자 변수 표기, so its 붙임 carrying the ellipsis is not
something a reader would guess; the rule now quotes it. A test pins the section
and the 붙임, because only the article separates the two cases.

Prose is untouched: an ellipsis outside a formula still goes through its own
rule under 한글 제53항. No output changes, all 224 mappings compared pair by
pair. Fixtures 5141 of 5141, corpus 456,025, marker bench 838 / 145 / 305 / 398.
The middle dot met inside an expression was filed under 한글 제50항, the
punctuation mark. The cells say otherwise: 제50항 writes 가운뎃점 as ⠐⠆, two
cells, and this table gives the same character a single ⠐.

수학 제2항 [붙임] is where that single cell comes from -- "점으로 표현된 곱셈
기호는 " 으로 적는다" -- so a dot standing between operands is the
multiplication sign, not the punctuation it resembles.

The article was wrong twice over. It named the wrong rule, and because the rule
runs in the math engine the trace rendered it as 수학 제50항, which is 무한대.

A test compares the two tables' cells for the same character, since that
difference is the whole evidence and a section number alone does not show it.

No output changes, all 224 mappings compared pair by pair. Fixtures 5141 of
5141, corpus 456,025, marker bench 838 / 145 / 305 / 398.
The trace decided the series from the engine that produced the cell: math
engine, therefore 수학 제N항. That is wrong whenever a rule implements an
article from another series, and several do. Circled numbers are 한글 제64항
and the math engine encodes them, so the page read them as 수학 제64항, which
is 햇. The degree sign, the colon, the semicolon and the tortoise-shell gloss
are the same shape of error.

Each rule already records the full citation in standard_ref -- "2024 Korean
Braille Standard, 한글 제64항" -- and nothing exposed it. The span now carries
standard_ref and subsection alongside the number, and the label reads the
series from the citation, falling back to the engine only when the citation
does not name one.

The article data was right the whole time; only the display inferred. No cells
change and no rule's article changes.

Fixtures 5141 of 5141, corpus 456,025, marker bench 838 / 145 / 305 / 398.
The middle-Korean detector counted any CJK ideograph as strong evidence that a
word belonged to a historical text, alongside the old jamo and the private-use
syllables. 국립국어원 answered on 2026-09-21 that the mode follows the 옛글자 --
"옛글자가 들어가면 그 글자에 대하여 그렇게 표기합니다" -- and that a Hanja
cannot be transcribed as itself at all, so it is evidence of nothing.

A modern sentence quoting one in parentheses -- 플로리다주(州)로,
지천명(知天命)의 -- is ordinary Korean, and the corpus has seven such sentences.

The trigger turns out to have been inert: with it gone, the regulation fixtures
still pass 5141 of 5141, the corpus still matches on 456,025, the marker bench
still reads 838 / 145 / 305 / 398, and all seven Hanja sentences still agree
with their reference. It fired only where the old jamo or the private-use
syllables were already firing.

A test now pins the negative, because historical texts are in fact full of
Hanja and the range looks like it belongs.

The judgment unit was already the word with a look at its neighbours, which is
finer than the sentence the reply describes, so nothing there needed changing.
Asked which article to cite when one decision rests on several, 국립국어원
answered on 2026-09-21: 다 적습니다. A section may now carry a list.

The English-context punctuation rule is the case that prompted the question. It
does three things at once and each has its own article: 제33항 keeps a comma
between Roman and Korean in the Korean shape, 제34항 drops the Roman terminator
when brackets or quotes enclose the Roman text, and 제49항 gives the punctuation
its cells. It cited 제49항 alone, and its standard_ref pointed at two chapters
by number -- Ch.4 Sec.10 + Ch.6 Sec.13 -- which no reader could turn into
articles.

The registry guard accepts a list without also accepting a rule that never
chose an article, and both the page and the trace harness render one as
제33항·제34항·제49항 rather than the 제33, 34, 49항 a naive join would give.

Over 5,160 traced papers the rule reports all three articles 2,402 times. No
output changes and no other rule's article moves. Fixtures 5141 of 5141, corpus
456,025, marker bench 838 / 145 / 305 / 398.
The placeholder existed as the trait's default so an unchecked rule reported
itself as unattributed instead of borrowing an article. Every math rule now
names a real one, which left the default unreachable and the placeholder static
never read -- two of the ten lines the coverage gate is holding out for.

Making meta required turns the runtime guard into a compile error: a rule
without a checked article no longer builds. That is the stronger statement, and
it is what the placeholder was standing in for all along.

The three dummy rules in the dispatch tests take a stand-in article, marked as
such, since they exercise dispatch and never reach the registry.

No behaviour changes. Fixtures 5141 of 5141, corpus 456,025, marker bench
838 / 145 / 305 / 398.
devfive added 6 commits September 22, 2026 17:45
Three of the lines the Linux gate reports are not branches anyone forgot to
test. They are a step in the middle of an iterator chain, the closing brace of
a for loop, and a condition split across four lines -- positions where the
recorder opens a region that the surrounding code never enters on its own. The
file already carries a note about this for a multi-line matches!(), so the
remedy is the one already used here: write the expression so no such position
exists.

The ampersand lookahead slices instead of skipping, which is what it meant
anyway. The blank-cell scan extends from an iterator rather than pushing inside
a loop. The middle-dot test binds its condition before the if.

Each is the same computation. Whole suite 5253 + 20 + 8 + 351 + 162 passing,
fixtures 5141 of 5141, corpus 456,025, marker bench 838 / 145 / 305 / 398.
The coverage gate reports line numbers from a source I was not reading. The
publish workflow writes a .rustfmt.toml with max_width = 100000 and runs
cargo fmt before tarpaulin, so every multi-line expression is collapsed first
and the reported numbers index that collapsed file. I had been opening those
numbers in my own tree, which is why my last two commits moved the reported
lines around instead of removing them: I was rewriting expressions that were
never the uncovered ones. Reproducing the reformat locally named the real
lines, and they are all code this branch introduced -- which is also why main
sits at 100% with the same surrounding code.

TokenRule::meta() gets the same treatment MathTokenRule got: required rather
than defaulted. The placeholder it returned was dead, because the twenty-six
real rules all declare an article and the ten dummies in token_engine never
have meta() called on them. Requiring it deletes the static and moves the
guarantee to the compiler.

The other four are reachable and now have tests. A measured quantity such as
3cm is emitted in one piece, so the emitter records its span itself; the raw
token rule refuses punctuation outside the four marks the Korean articles
name; an indicator landing inside a rule's cells keeps the fragment before it;
and a token that wrote no cells records no span at all, since a zero-width
span would claim an output position the token never wrote.

Whole suite: 5270 + 20 + 8 + 351 + 162 passing, no new warnings.

Two lines remain, both in the math symbol chain, and they are a real defect
rather than a test gap: the arms for the proportion sign and the sequence
braces call a shortcut table that has no entry for either character, so the
call always fails and the article line after it cannot be reached. The
standard does define both -- 과학점자 제29항 gives 비례 기호 as +3, 수학
제24항 gives 수열 as 7A;N7 -- so the fix is the missing table entries, not a
deletion. That lands next.
The last two uncovered lines were the article expressions in the math symbol
chain for the proportion sign and the sequence braces. Neither could be
reached: both arms call the shortcut table for a character the table has no
entry for, so the call always fails and the line after it is dead. The lines
are mine -- the per-branch articles this branch introduced -- but the dead
arms underneath them are older.

The standard does define the proportion sign. 과학 제29항 writes 비례 기호 as
+3, with 위치에너지 ∝ 질량 as its example, so the table was simply missing the
entry and ∝ was refused as input. It now encodes as ⠬⠒. The article is 과학
제29항, not 수학 제5항: 제5항 is the ratio sign ∶ (U+2236), which the table
already carried and which is a different character and a different rule. The
arm had been crediting ∝ to 제5항, so a new META_SCIENCE_29 takes it and
rule_5's module doc records the split.

The sequence braces are not a missing entry. 수학 제24항 writes 수열 as 7A;N7,
so the braces of a sequence take ⠶ -- but only there. I added '{' => ⠶ and
found out the hard way: it broke set-builder notation, where {x | x ∈ R} must
keep the Korean braces ⠦⠂ and ⠐⠴. Braces are article-24 only in the sequence
context, and nothing in the encoder detects that context, so a flat mapping is
wrong by construction. The arm could therefore never have worked, and the test
covering it said as much in its own comment: the parser routes braces to
OpenParen, so the arm is not on the path. The stub module goes, and its test
now states which path braces actually take. 수학 제24항 stays unimplemented
rather than half-implemented.

Shortcut table diff against HEAD: 1 added, 0 dot patterns changed.
Whole suite: 5269 + 20 + 8 + 351 + 162 passing, no warnings.
Corpus unchanged at 456025/467121 (97.6246%).
Both pushes failed before tarpaulin ever ran, so neither measured anything.
The lint script is oxlint && cargo fmt --check && cargo clippy -D warnings,
and my edits broke the middle and the end of it.

cargo fmt disagreed with two places I had hand-written: the trace assertion
chain in the measurement test, and the use list in symbol_rule after rule_24
came out of it, which needed rewrapping. clippy caught drop(sink) in the
emit test -- Option<TraceSink> implements no Drop, so the call only extends a
lifetime and says nothing. A plain scope ends the borrow and is what I should
have written.

I had been checking cargo test and skipping the lint script, which is the same
narrowing that cost nine snapshots earlier on this branch. Verified here:
cargo fmt --all --check clean, cargo clippy -p braillify --all-targets
-D warnings clean, suite 5269 + 20 + 8 + 351 + 162 passing.
One line was left: the trailing Err in RawTokenRule. It only runs when the
character is one of ? ! : ; and the Korean symbol table then fails to encode
it -- but the match directly above admits exactly those four, and all four are
in that table. The lookup therefore always succeeds and the line below it
cannot execute. A test could not have covered it; only deleting the branch
could.

Folding the lookup into the happy path with map_err keeps the error the
function would report if an entry ever went missing, while removing the second
exit that could not be taken. The earlier match arm keeps its own Err for
characters outside the four, which the test added with it still covers.

fmt clean, clippy -D warnings clean, suite 5269 + 20 + 8 + 351 + 162 passing.
Previous run measured 99.99% with this as the only uncovered line.
The node tests died at module load with a LinkError naming an import the JS
glue plainly exported. My first read blamed Vec<RuleSpan>, and removing it did
move the error -- onto __wbindgen_throw, which every build has. That ruled the
vector out and named the real cause: any #[wasm_bindgen] struct makes the
module import per-field getters back from its own glue, and Bun cannot link
that circular shape. main never hit it because main exports no struct; this
branch introduced the first ones with the trace binding.

So RuleSpan and TraceResult stop being exported classes and become plain
serde structs, and translateToUnicodeWithTrace returns their JSON. The wasm
now imports nothing at all -- __wbg_ count is zero, where it was six -- and
loads cleanly.

The landing side gets simpler for it. readTrace takes the JSON string, parses
once, and the objects it hands to React hold no wasm memory, so the per-span
free() loop and the TraceResult free() are both gone. Field names are
unchanged, so nothing downstream of readTrace moved.

bun test 14178 passing, 0 failing -- it was 2 failing before this.
cargo test -p node 20 passing, fmt and clippy -D warnings clean.
@codecov

codecov Bot commented Sep 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
libs/braillify/src/encoder.rs 100.00% <100.00%> (ø)
libs/braillify/src/korean_char.rs 100.00% <100.00%> (ø)
libs/braillify/src/lib.rs 100.00% <100.00%> (ø)
libs/braillify/src/math_symbol_shortcut.rs 100.00% <100.00%> (ø)
libs/braillify/src/rules/context.rs 100.00% <ø> (ø)
libs/braillify/src/rules/emit.rs 100.00% <100.00%> (ø)
libs/braillify/src/rules/engine.rs 100.00% <100.00%> (ø)
...ibs/braillify/src/rules/english_ueb/contraction.rs 100.00% <100.00%> (ø)
libs/braillify/src/rules/english_ueb/engine.rs 100.00% <100.00%> (ø)
...llify/src/rules/english_ueb/engine/encode_space.rs 100.00% <ø> (ø)
... and 64 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

Changepacks

braillify@2.2.0 → 2.2.1 - libs/braillify/Cargo.toml

Patch

  • Apply the National Institute of Korean Language rulings on the grade-1 symbol and the ever contraction, read a whole-run groupsign as letters where the rules require it, separate a dash that opens the text, and join a spaced hyphen that is not a subtraction sign, and close the editorial gap inside a bracket; corpus accuracy 455,975 of 467,121 sentences.

braillify-c@0.2.0 → 0.2.1 - packages/c/Cargo.toml

Patch

  • Apply the National Institute of Korean Language rulings on the grade-1 symbol and the ever contraction, read a whole-run groupsign as letters where the rules require it, separate a dash that opens the text, and join a spaced hyphen that is not a subtraction sign, and close the editorial gap inside a bracket; corpus accuracy 455,975 of 467,121 sentences.

Braillify@2.2.0 → 2.2.1 - packages/dotnet/Braillify/Braillify.csproj

Patch

  • Apply the National Institute of Korean Language rulings on the grade-1 symbol and the ever contraction, read a whole-run groupsign as letters where the rules require it, separate a dash that opens the text, and join a spaced hyphen that is not a subtraction sign, and close the editorial gap inside a bracket; corpus accuracy 455,975 of 467,121 sentences.

BraillifyNet@2.2.0 → 2.2.1 - packages/dotnet/BraillifyNet/BraillifyNet.csproj

Patch

  • Apply the National Institute of Korean Language rulings on the grade-1 symbol and the ever contraction, read a whole-run groupsign as letters where the rules require it, separate a dash that opens the text, and join a spaced hyphen that is not a subtraction sign, and close the editorial gap inside a bracket; corpus accuracy 455,975 of 467,121 sentences.

braillify-go@2.0.1 → 2.0.2 - packages/go/Cargo.toml

Patch

  • Apply the National Institute of Korean Language rulings on the grade-1 symbol and the ever contraction, read a whole-run groupsign as letters where the rules require it, separate a dash that opens the text, and join a spaced hyphen that is not a subtraction sign, and close the editorial gap inside a bracket; corpus accuracy 455,975 of 467,121 sentences.

braillify@2.2.0 → 2.2.1 - packages/node/package.json

Patch

  • Apply the National Institute of Korean Language rulings on the grade-1 symbol and the ever contraction, read a whole-run groupsign as letters where the rules require it, separate a dash that opens the text, and join a spaced hyphen that is not a subtraction sign, and close the editorial gap inside a bracket; corpus accuracy 455,975 of 467,121 sentences.

braillify@2.2.0 → 2.2.1 - packages/python/pyproject.toml

Patch

  • Apply the National Institute of Korean Language rulings on the grade-1 symbol and the ever contraction, read a whole-run groupsign as letters where the rules require it, separate a dash that opens the text, and join a spaced hyphen that is not a subtraction sign, and close the editorial gap inside a bracket; corpus accuracy 455,975 of 467,121 sentences.

braillify_rb@0.1.0 → 0.1.1 - packages/ruby/Cargo.toml

Patch

  • Apply the National Institute of Korean Language rulings on the grade-1 symbol and the ever contraction, read a whole-run groupsign as letters where the rules require it, separate a dash that opens the text, and join a spaced hyphen that is not a subtraction sign, and close the editorial gap inside a bracket; corpus accuracy 455,975 of 467,121 sentences.

devfive added 18 commits September 22, 2026 22:30
encode_nemeth_spans appended the opening indicator, the maths between the
dollars, the continuation separator and the closing indicator straight onto
the output with no record, so every cell of an inline maths switch inside
English prose went unexplained. The spreadsheet showed it as whole runs of
nothing: on Al<sup>3+</sup>(aq)$+$3e<sup>-</sup>… the five cells of ⠸⠩ ⠬ ⠸⠱
named no rule at all, while the blanks around them were picked up only by the
word-space emitter.

§14.6.2 already owns this output -- it is what the token-level path records
for the same construct -- so the four appends now go through it. They are
recorded where they are appended rather than reconstructed by a later search,
which is the failure mode the rest of this file's attribution has.

Spreadsheet: unattributed cells 449 -> 262, rows 15 -> 10,
857427/857689 cells now name an article (99.9695%, was 99.9477%).
Whole suite 5269 + 20 + 8 + 351 + 162 passing, so no output moved.

What is left is a different gap: prose that spells out literal <sub> markup is
attributed in some positions and not others, which points at the record-then-
search alignment rather than at the emitters. Untouched here.
Two things kept cells of a chemical line from naming a rule, and they had to be
fixed together.

The §8.8.2 branch that gives a two-letter chemical symbol its capitals one at a
time -- KBr, KCl, and here CCl and HCl -- appended the indicator and the letter
with a bare push and recorded neither. Those cells had nothing to attribute
them to at all.

Recording them alone was not enough. align_selected recovers a record's place
by searching the finished output for its cells, and a capital indicator is one
cell that recurs all over a capitalised line, so the search lands on an earlier
occurrence than the one the record wrote. A NonWordAttempt now carries the
offset it was written at, and align_selected prefers it when the output still
holds those cells there and nothing already claims them.

The offset is Option, not usize, on purpose: rule_14's nemeth spans are built
in a buffer that is appended elsewhere, so their position is not known at
record time and they keep the search. Marking them Some was measurably worse
than no offset at all -- a one-cell record matches a wrong offset by accident
and takes a place that belonged to another rule.

Spreadsheet: unattributed 262 -> 258, rows 10 -> 9, 857431/857689 (99.9699%).
Braille output identical on all 5157 rows. Suite 5270 + 20 + 8 + 351 + 162.

Two inputs still fall short and are the next test: aMgO(s)$+$… and
CO$+$H<sub>2</sub>O↔…, both lowercase-initial camel words whose subunit
capitals come from encode_title_subunit, which builds its cells in a local
buffer. Their callers know the offset and should record there.
A mixed-case word is assembled subunit by subunit in its own buffer, and the
capitals between those subunits were pushed into it bare. They had no record,
and they could not usefully have one: a record taken while filling that buffer
knows only its place inside it, and the buffer has not been appended yet.

So the capitals now go through push_indicator against the buffer, and the
append rebases every record taken since a checkpoint by where the buffer
landed. That is what turns buffer-local positions into ones the finished
output can be indexed by. I found the problem by printing out.len() at the
append: it was 0 on words that sit well into the line, which is what a
buffer-local offset looks like.

Spreadsheet: unattributed 258 -> 249, rows 9 -> 8, 857440/857689 (99.9710%).
Braille output identical on all 5157 rows. Suite 5272 + 20 + 8 + 351 + 162.

The rebase only climbs one level. `aMgO` and `dCO` are now whole, but `bC` and
`cMg` in the same line each keep one capital, because the buffer they rebase
onto is itself a buffer that is appended further up. The next step is to carry
the rebase through that second level rather than stop at the first.
encode_nemeth_spans hands each stretch of prose between the dollars to a
closure that encodes it into a fresh buffer, then appends the result. The
records that closure made were still in that buffer's coordinates, so the
rebase added for a word's own buffer only climbed one level and stopped.

Rebasing again at each of those appends carries it the rest of the way. The
balanced equation aMgO(s)$+$�C(s) -> cMg(s)$+$dCO(g)$+$eCO<sub>2</sub>(g)
now names a rule for all 90 of its cells, where two capitals were unexplained
before.

Spreadsheet: unattributed 249 -> 241, rows 8 -> 7, 857448/857689 (99.9719%).
Braille output identical on all 5157 rows. Suite 5272 + 20 + 8 + 351 + 162.
The prose after the last closing indicator went straight through
out.extend(encode_ueb(rest)) while every earlier chunk had been switched to
the rebasing append. Its records kept their buffer-local offsets, so the
capital opening the tail claimed cell 0 instead of the cell it wrote.

Folding that branch into extend_prose finishes the cascade.
encode_simple_ueb_symbols stays as the fallback for prose the UEB encoder
rejects.

Spreadsheet: unattributed 241 -> 233, rows 7 -> 4, 857456/857689 (99.9728%).
Braille output identical on all 5157 rows. Suite 5273 + 20 + 8 + 351 + 162.

What remains is one shape: a word attempt is recorded with its cells but no
position, and the two-cell runs of literal <sub> markup repeat so often that
the forward search settles on the wrong one. Word records would need the same
offset treatment the indicator and direct records just got.
The indicator and direct records learned their position a few commits ago; the
word attempts never did. They are still placed by searching the output for
their cells, and a chemical line repeats the two-cell runs of literal <sub>
markup a dozen times, so the forward scan settles on the wrong occurrence and
then drags past everything after it.

A WordAttempt now carries the same Option<usize>. The settle paths know where
the word started and pass it; the attempts the contraction engine builds in
its own buffer keep None and keep the search, which is correct for them --
they are candidates, and most are discarded. rebase_attributions moves word
offsets along with the others, so the cascade added for nemeth prose chunks
covers them as well.

Spreadsheet: unattributed 233 -> 4, rows 4 -> 2, 857685/857689 (99.9995%).
Braille output identical on all 5157 rows. Suite 5274 + 20 + 8 + 351 + 162.

The four that remain are one cell each: the letter of a single-capital chemical
symbol, whose attempt comes from the contraction engine as a one-cell run with
no position. Placing those means giving the engine's own attempts a position
relative to the word, which is a larger change than this one.
The chemistry articles were the one part of the standard with no fixtures at
all, so there was no way to say what the encoder does or does not handle. The
symbol definitions are unambiguous in the PDF -- each sits alone on its line
with its internal notation beside it -- so they go in first, which is the
order the fixture rules ask for anyway.

Converting the PDF's internal notation was checked against the 2339 existing
korean and math entries first: every one of them round-trips to the expected
and unicode already recorded, so the same converter can be trusted here.

Both groups are marked benchmark, like the corpora, because the chemistry
engine does not exist yet and a regulation fixture that cannot pass would
otherwise fail the gate.

Running them says exactly where the work is. Article 18's reaction symbols
already encode correctly, 6 of 6 -- the plus sign, the arrows, the reversible
sign, gas and precipitate. Article 7's formulas are 0 of 4: the capital
indicator before an element symbol and the subscript after it are what is
missing.

testcase integrity 14171 passing, suite 5274 + 20 + 8 + 351 + 162.
The chemistry articles now have fixtures for every symbol the PDF defines
with both its print form and its internal notation beside it: the mass-number
superscript of article 3, the bond lines of article 10, the ring cores of
article 12, the electrode and salt-bridge marks of article 21, and the sex
signs of article 22. Entries whose print form the PDF only shows as a diagram
are left out rather than guessed at.

The integrity check now covers the science directory too, which is what
checks internal against expected and unicode -- 14301 assertions passing,
up from 14171. It deliberately skips runConversionTests, since that asks the
encoder for output the chemistry engine cannot produce yet.

With all of them registered the picture is no longer a guess:
  article 18 reaction symbols  6/6
  article 21 electrode marks   1/2
  articles 3, 7, 10, 12, 22    0
  science overall              7/26

So the reaction symbols are already right and need nothing. What is missing
is the formula notation itself -- the capital indicator before an element
symbol, the subscript and superscript after it, and the bond lines built on
the same subscript prefix.

Suite 5274 + 20 + 8 + 351 + 162, testcase integrity 14301.
The PDF writes atomic numbers, mass numbers and formula counts as real
subscripts and superscripts. Text extraction flattens them to ordinary digits,
and I copied that flattening straight into the fixture inputs last commit, so
O2 stood where the standard means O2 with a subscript two.

The braille says which it is. Article 3 lists the atomic number eight as
,o;#h -- the element first, the subscript after -- which only makes sense if
the print form carries the eight ahead of the symbol as a subscript. The
internal notation, expected and unicode all stay exactly as the PDF has them;
only the input is corrected.

This also settles how the formulas can be recognised at all. Measured against
the corpus: a bare letter-plus-digit matches 9589 sentences, and even after
keeping only real element symbols it still matches 5318 -- F1, P100, V6, S3,
none of them chemistry. An element symbol followed by a unicode subscript or
superscript matches 4, two of which are chemistry. That is the signal.

testcase integrity 14301 passing, science 7/26 unchanged.
The female and male signs were rejected outright -- Invalid symbol character
-- so adding them to the Korean symbol table is purely additive. Article 22
gives them as 0^x4 and 0^y4, and they now encode as that both alone and
inside a Korean sentence.

Three of the fixtures I wrote last commit had no basis and are removed. The
bond lines of article 10 were entered as bare hyphen, equals and identical-to,
but those characters already mean hyphen, equals and identical-to in ordinary
text -- a bond line only exists inside a structural formula, so it has no
standalone print form to test. The ring cores of article 12 are shown in the
PDF as diagrams, and the hexagon characters I picked for them were a guess.
The three structural examples of article 10 stay, since the PDF prints those.

Article 21's salt bridge is left as it is and still fails: the same character
is parallel under maths article 44, which is what the encoder produces. That
is a context question, not a missing symbol.

science 9/20, with article 18 at 6/6 and article 22 now 2/2.
Corpus unchanged at 456025/467121. Suite 5274 + 20 + 8 + 351 + 162,
testcase integrity 14271.
Maths article 12 runs consecutive capitals together under one caps-word sign,
which is right for AB and wrong for a chemical formula: science article 4 says
the element symbols are all written in grade 1, each with its own capital
sign. Ca(OH)2 came out as the caps-word form and so did NH inside
[Cu(NH3)4](OH)2.

The branch that already spells a matrix name out letter by letter now also
takes formulas. What counts as one is narrow on purpose. Single-letter element
symbols are the same letters maths uses for variables -- P, V, B, C -- so the
sequence only qualifies when the expression carries a subscript, which is the
signal measured earlier: an element symbol followed by a unicode subscript
occurs in 4 corpus sentences out of 467121, against 5318 for a bare letter
and digit.

Ca(OH)2 and [Cu(NH3)4](OH)2 now match the standard, taking article 7 from
0 of 4 to 2 of 4 and science to 11 of 20. Corpus unchanged at 456025/467121.
Suite 5274 + 20 + 8 + 351 + 162, fmt and clippy clean.
I had O2 down as needing no roman indicator and the encoder as being
inconsistent for emitting one. The standard says the opposite. Korean article
68, which is the article that defines how subscripts are written at all,
lists its examples with the indicator attached -- B6 as 0,b;#f, and the square
metre as 0m^#b. The encoder produces exactly that.

What I had copied was science article 7's definition line, which shows only
the formula's own cells because that is all a definition line shows. A fixture
whose input is the formula runs the whole pipeline, so it gets the indicator
the same way B6 does.

The subscript sign also loses its standalone entry. Its print form is a
semicolon, which already means semicolon -- the same mistake as the bond lines
removed earlier.

Article 7 now passes 3 of 3 and science stands at 12 of 19. Corpus unchanged
at 456025/467121, testcase integrity 14266.
Science article 3 puts the element symbol first and the numbers after it as
subscript and superscript. Maths articles 18 and 19 do the opposite for a left
superscript or subscript: the index stays in front and is wrapped in a group.
Both are right for their own notation, so 7Li came out as the maths form with
the seven leading and parenthesised.

The superscript and subscript rules now hand the following tokens to a shared
helper that writes an element symbol with its capital sign and reports how
many tokens it took. When it reports nothing the maths path runs unchanged,
which is what keeps a left superscript on an ordinary variable intact -- n
over x still groups the way article 18 asks.

The helper checks against a list of real element symbols rather than any
capital letter, because a one-letter symbol is the same letter maths uses for
a variable.

Article 3 now passes 2 of 2 and science stands at 14 of 18. Corpus unchanged
at 456025/467121. Suite 5274 + 20 + 8 + 351 + 162, fmt and clippy clean.
Science article 10 writes a chain compound by joining the element symbols with
bond lines -- a subscript sign followed by 1, 2 or 3 for single, double and
triple -- and article 4 puts the whole thing in a capitals passage once three
single-letter symbols run together. The encoder had none of it: O=C=O came out
as maths, with the equals signs as comparison operators.

A token rule in the normalisation phase now recognises the chain and writes
it whole. Recognition is deliberately narrow: the letters must be real
one-letter element symbols, the marks must be bond lines, and there must be
at least three elements. A-B is left alone because A is not an element, which
is the same guard the subscript work used.

Two of the three fixtures pass. H-O-H does not, and it is not this rule's
doing: its characters are all ASCII, so the document is routed to English
before any token rule runs, and it comes out as a hyphenated English word.
Moving that decision is a separate change.

science 16/18. Corpus unchanged at 456025/467121. Suite 5281 + 20 + 8 + 351
+ 162, fmt and clippy clean.
H-O-H is all ASCII, so the document went to the English engine before any
token rule ran and came out as a hyphenated word. The UEB eligibility check
now declines a chain of elements, which sends it down the Korean path where
the structural formula rule writes it.

That broke B-U-S. UEB 8.7.1 spells a word out with hyphens between its
letters, and boron, uranium and sulphur are all one-letter element symbols, so
the same shape is both. Across every fixture and the whole corpus that is the
only collision -- measured, not assumed.

A dictionary check could not tell them apart: hoh is in CMUdict. Chemistry
can. A structural formula writes out a whole molecule, so every atom carries
exactly as many bonds as its valence -- H-O-H does, with hydrogen at one and
oxygen at two, while the end letters of B-U-S and S-O-S carry one bond each
and are not molecules. Double and triple bonds are never used for spelling, so
the check applies only to chains made of single bonds.

Science article 10 now passes 3 of 3 and science stands at 17 of 18, with
English 8.7.1 back at 14 of 14. Corpus unchanged at 456025/467121. Suite
5283 + 20 + 8 + 351 + 162, fmt and clippy clean.
The salt bridge and the parallel sign are the same character. Article 21
writes the bridge as two electrode cells with a blank on each side; maths
article 44 writes parallel as its own sign, which is what the encoder always
produced. Neither is wrong, so the fixture that tested a bare parallel sign
against the bridge form could never pass -- a bare sign is parallel, the same
way a bare hyphen is a hyphen. It is replaced with the two cell diagrams the
PDF actually prints.

A normalisation token rule now recognises a cell diagram and writes it whole:
electrodes and the bridge spaced, element symbols capitalised one by one with
subscripts after them, state symbols in the Korean round brackets article 18
asks for, polarity signs, and Korean inside the diagram wrapped in the Korean
indicator and terminator. The recogniser needs a bridge and an electrode,
must start and end on a formula or a polarity sign, and takes Korean only as
the label of the electrode right after it -- so a sentence's own opening words
are never pulled in, and AB parallel to CD is left to maths.

Science now passes 18 of 18. Corpus unchanged at 456025/467121. Suite 5290 +
20 + 8 + 351 + 162, fmt and clippy clean.
The science groups were registered as benchmarks because the chemistry engine
did not exist and a regulation fixture that cannot pass would fail the gate.
They all pass now, 18 of 18, so the benchmark flag comes off: test_by_testcase
requires them at 100% like every other regulation group, and a regression in
any chemistry article now fails the build instead of lowering a percentage.

The integrity file drops the comment that explained why science had no
conversion run, and science joins runConversionTests.

One thing the conversion run cannot prove: it wraps its expect in a try whose
catch swallows the assertion, so it never fails for any directory. That is
older than this change and is left alone here; test_by_testcase is what
actually enforces the answers.

Suite 5290 + 20 + 8 + 351 + 162, testcase integrity 14279, corpus unchanged
at 456025/467121.
CI measured 99.96% with seven lines uncovered, and reproducing its reformat
locally named each one.

One was unreachable: a chain of five or more odd-length characters always has
at least three elements, so the separate three-element check could never fire.
It is gone.

Four were real branches with no case: a lone capital that is not an element,
a diagram that opens on an electrode, a middle word that is neither formula
nor label, and the carbon and unknown-valence arms of the valence table,
which short-circuiting had kept every existing case from reaching.

The last was the prime arm in the element-run check, and following it showed
a wrong answer rather than a missing test: primes were skipped, so O prime H
counted as two element symbols. A primed capital is a maths variable, so the
check now requires every token in the run to be an element capital, which
also removes the arm.

Suite 5295 + 20 + 8 + 351 + 162, science 18/18 enforced, corpus unchanged at
456025/467121, fmt and clippy clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant