Keep composite scores stable when the leaderboard is filtered - #1032
Merged
Conversation
Closes #702. Each profile's 100-point reference was computed over the rows left after filtering, so filtering changed the scores themselves. Because every profile's leader moves by a different factor when a subset is selected, the sum could reorder — the reported case, reproduced exactly: main this branch unfiltered rage 265 (#33) roda 233 (#36) rage 265 (#33) roda 233 (#36) filter ruby rage 678 (#3) roda 775 (#1) rage 265 (#1) roda 233 (#2) Rage outscores Roda overall but placed below it as soon as the board was filtered to Ruby, with both scores inflated ~2.9x. The predicate that decided which rows to show was also deciding what to normalize against. Those are two different questions, so they are now two predicates: outOfLeague is this entry ranked somewhere else entirely? Types are separate leagues — engine entries are scored on their own subset of profiles, so an engine result must never set the reference for a framework entry. hidden is it merely filtered out of view? The search box and the tuned toggle are display filters and now have no effect on any score. Normalization uses only the first, so the numbers are a property of the framework rather than of the current view. This covers the per-profile maxima, the memory-efficiency maxima, and the json-comp bytes-per-response reference, which had the same problem. Per the discussion on the issue, the old behaviour is still worth having — comparing a subset against itself is a real use case — so it becomes an explicit "Rescale to selection" toggle next to the memory one, off by default, carried in the URL as rescale=1. With it on, the filtered view reproduces the previous numbers exactly (678 / 775 above). The header blurb states which rule is active, so a screenshot is never ambiguous. Verified in headless Chrome against the real data, on this branch and on main: unfiltered ordering is unchanged, filtering to Ruby now leaves both scores untouched, and toggling rescale on reproduces main's filtered output value for value.
MDA2AV
added a commit
to Kaliumhexacyanoferrat/HttpArena
that referenced
this pull request
Jul 24, 2026
Six merges landed since the last sync: MDA2AV#1030, MDA2AV#1031, MDA2AV#1032, MDA2AV#1033, MDA2AV#1034 and MDA2AV#1036. One real conflict, in the composite blurb, where both sides had changed the same line for different reasons: main (MDA2AV#1032) made the text state which normalization rule is active, so a screenshot can't be ambiguous about whether scores were rescaled to the current filter this branch pointed the "How it works" link at /docs/scoring/composite -score/ instead of the hash route Both are kept: MDA2AV#1032's wording with this branch's real URL. Everything else merged on its own, but the two structural ones were checked rather than assumed: MDA2AV#1034 moved results from 52 shared per-profile arrays to one file per framework. Git applied that to the renamed generator correctly — it still loads site/data/results/*.json and still emits 1941 rows, and compare.sh kept both its per-framework read and the --compare flag from MDA2AV#1031. MDA2AV#1030 added three echo-ws-limited doc pages after this branch's SEO pass, so they were written with seo_title/description already; all 126 pages now carry both. Generator output on the merged tree: 1941 rows, 126 static doc pages, a 126 -page search index and a 127-URL sitemap. All six workflows validate against a loader that rejects duplicate keys, which is what MDA2AV#1036 had to fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #702.
Each profile's 100-point reference was computed over the rows left after filtering, so filtering changed the scores themselves. Since every profile's leader moves by a different factor when a subset is selected, the sum could reorder.
Reproduced exactly, in a browser against the real data:
mainrubyRage outscores Roda overall but placed below it as soon as the board was filtered to Ruby — with both scores inflated ~2.9×.
The cause
One predicate decided both which rows to show and what to normalize against. Those are different questions, so they're now two:
outOfLeaguehiddenNormalization uses only the first, so a score is a property of the framework rather than of the current view. This covers the per-profile maxima, the memory-efficiency maxima, and the
json-compbytes-per-response reference — which had the same problem.Both behaviours, as discussed on the issue
So the old behaviour becomes an explicit Rescale to selection toggle beside the memory one — off by default, carried in the URL as
rescale=1. With it on, the filtered view reproduces the previous numbers exactly (678 / 775 above). The header blurb states which rule is active, so a screenshot is never ambiguous.Verification
Headless Chrome against the real
data.js, run on this branch and onmain: unfiltered ordering unchanged, filtering to Ruby now leaves both scores untouched, and toggling rescale on reproduces main's filtered output value for value.Docs updated in
composite-score.md.🤖 Generated with Claude Code