Skip to content

Commit 494f53e

Browse files
marcobambiniclaude
andcommitted
Run the fp8 test, and cover the reader whose guard was load-bearing
Two things on top of tomatotomata's checks, which are unchanged and were right: the mutants I planted in unblock_scale — returning q untouched, transposing the block dims, dropping the [:M, :N] crop, removing the shape guard — were all caught, and expected_dequant being an independent index-array implementation rather than a second call to the code under test is what gives it that. It never ran. run.sh invoked it as bare `python3`, and torch is not a system package on the machines that test this: CLAUDE.md says it is never a repo dependency and every other torch checker goes through uv. So the whole thing reported "torch not installed" and skipped everywhere, CI included — where the Linux job is the one that installs uv, which is to say the one place it does get to run. Through run_uv now, with the guard on uv rather than python3, since without uv run_uv exits 127 and the catch-all would report FAIL where a SKIP is meant. 56 passed / 0 failed / 2 skipped becomes 57 / 0 / 2. And the missing-companion check exercised ST only. Deleting ShardReader's own `raise` in convert.py, so it returns the tensor unscaled instead, kept the suite green — a silent wrong answer in the reader #26's description called out as "a second reader and was easy to miss". Both readers now, and that mutant is killed. The same mutation against ST survives, and should: without its guard, raw() still raises KeyError naming weight_scale_inv, so the contract the test asserts — refuses, and says which tensor — holds either way. That guard buys a better message, not a different decision, and asserting its exact prose would test the wording rather than the behaviour. Also wrote down the case nothing here can catch, which is the reason the tile size is read from config rather than inferred: 300 rows against 3 scale rows admits both 128 and 100, both pass every shape check, and the wrong one applies each scale to the wrong rows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 6d61fb2 commit 494f53e

2 files changed

Lines changed: 41 additions & 11 deletions

File tree

tests/run.sh

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1342,10 +1342,19 @@ head_ "converter"
13421342
# mode, so keep its tile mapping under a small synthetic test instead of
13431343
# relying on a multi-hour model conversion. Exit 77 is an explicit skip when
13441344
# torch is unavailable, never a pass.
1345-
if ! command -v python3 >/dev/null 2>&1; then
1346-
sk "fp8 block-scale mapping" "python3 not installed"
1345+
#
1346+
# Through uv, like every other torch checker here: torch is not a dependency
1347+
# of this repo and is not a system package on the machines that run this. As
1348+
# bare `python3` the whole check reported "torch not installed" and skipped
1349+
# everywhere, CI included — where the Linux job is the one that installs uv,
1350+
# so this is exactly where it does get to run.
1351+
if ! command -v uv >/dev/null 2>&1; then
1352+
# The guard is on uv rather than python3 for the same reason: without uv
1353+
# run_uv exits 127 and the catch-all below would call that a failure.
1354+
sk "fp8 block-scale mapping" "uv not installed"
13471355
else
1348-
out=$(python3 tests/test_fp8_blocks.py 2>&1); rc=$?
1356+
out=$(run_uv run --quiet --with torch --no-project \
1357+
python tests/test_fp8_blocks.py 2>&1); rc=$?
13491358
case "$rc" in
13501359
0) ok "fp8 block scales, partial tiles, missing companions, and reader agreement" ;;
13511360
77) sk "fp8 block-scale mapping" "torch not installed" ;;

tests/test_fp8_blocks.py

Lines changed: 29 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,16 @@
66
These tests use small synthetic tensors so they do not need model weights. A
77
missing torch installation is an explicit skip, matching tests/run.sh's rule
88
that unavailable prerequisites must never look like a pass.
9+
10+
The one case nothing here can catch, stated because it is the reason the tile
11+
size is read from the checkpoint's config rather than inferred from the two
12+
shapes: a *compatible but wrong* block size. 300 rows against 3 scale rows
13+
admits both 128 (the truth, with a partial last tile) and 100 (a clean split).
14+
Both satisfy the shape check below, both produce a tensor of the right size,
15+
and the wrong one applies every scale to the wrong rows. No assertion over
16+
shapes can separate them, which is why `unblock_scale` takes `block` as an
17+
argument instead of deriving it — the check that matters happened before this
18+
file was reached.
919
"""
1020

1121
import json
@@ -81,14 +91,25 @@ def test_partial_last_row_and_column_are_cropped_after_mapping():
8191

8292

8393
def test_missing_scale_companion_is_rejected():
84-
with tempfile.TemporaryDirectory() as root:
85-
write_safetensors_model(root, (2, 3), include_scale=False)
86-
try:
87-
ST(root).tensor("weight")
88-
except KeyError as exc:
89-
assert "weight_scale_inv" in str(exc)
90-
else:
91-
raise AssertionError("fp8 tensor without its scale companion was accepted")
94+
"""Both readers refuse, not just the one that is easy to remember.
95+
96+
ST and ShardReader each carry their own copy of this guard, and covering
97+
only ST left the convert.py one free to return the tensor unscaled: a
98+
mutation that deleted its `raise` kept the whole suite green. That is the
99+
silent-wrong-answer this file exists to prevent, in the reader #26's own
100+
description called out as "a second reader and was easy to miss".
101+
"""
102+
for label, read in (("mxfp4.ST", lambda r: ST(r).tensor("weight")),
103+
("convert.ShardReader", lambda r: ShardReader(r).get("weight"))):
104+
with tempfile.TemporaryDirectory() as root:
105+
write_safetensors_model(root, (2, 3), include_scale=False)
106+
try:
107+
read(root)
108+
except KeyError as exc:
109+
assert "weight_scale_inv" in str(exc), f"{label}: {exc}"
110+
else:
111+
raise AssertionError(
112+
f"{label} accepted an fp8 tensor with no scale companion")
92113

93114

94115
def test_gross_scale_shape_mismatch_is_rejected():

0 commit comments

Comments
 (0)