bench(functions): group the 63 bench targets into 9 - #24342
Open
Dandandan wants to merge 1 commit into
Open
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #24342 +/- ##
==========================================
+ Coverage 81.13% 81.17% +0.03%
==========================================
Files 1112 1109 -3
Lines 388220 388033 -187
Branches 388220 388033 -187
==========================================
- Hits 314985 314977 -8
+ Misses 54696 54513 -183
- Partials 18539 18543 +4 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Each `[[bench]]` target is a separate binary that statically links this crate,
arrow and criterion, so building this crate's benchmarks meant 63 link steps.
Linking, not compiling, was the bulk of it: rebuilding a *single* bench target
took 10.6s, of which the compile is a fraction.
The 63 targets are grouped by their `required-features`, which keeps that
semantics exactly as it was:
string_expressions 16 benches
math_expressions 16
unicode_expressions 12
datetime_expressions 8
regex_expressions 4
misc 4 (no required features)
leaving `crypto`, `encoding` and `dictionary_encoding` standalone, since they
are the only members of their feature groups. Each group file declares the
individual benchmarks as modules and lists their criterion groups in one
`criterion_main!`.
The benchmark code itself is untouched: no `bench_function`,
`benchmark_group` or `bench_with_input` line changes anywhere in the diff. Per
file the change is only dropping `criterion_main!` (a module cannot define
`main`), dropping the now-unused `criterion_main` import, dropping
`extern crate criterion` (a no-op since edition 2018), and pointing the four
users of `benches/helper.rs` at `crate::helper`.
`autobenches = false` keeps cargo from picking the module files back up as
targets of their own.
Running a single benchmark still works, via criterion's filter:
cargo bench --bench math_expressions -- power
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Dandandan
force-pushed
the
perf/consolidate-function-benches
branch
from
August 13, 2026 19:23
b381bac to
a6b5f3e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
None. Found while profiling compile times (#24325, #24326, #24329, #24330,
#24338, #24339) — this is the largest remaining item I measured.
Rationale for this change
datafusion-functionsdeclares 63[[bench]]targets. Each one is aseparate binary that statically links this crate, arrow and criterion, so
building the crate's benchmarks means 63 link steps.
Linking, not compiling, is the bulk of it: rebuilding a single bench target
takes 10.6s, almost all of it link, while the 63 benchmark files together are
only ~10.5k lines of source.
This is paid by
cargo bench, bycargo build --all-targets, and by CI'sclippy --all-targets.What changes are included in this PR?
The targets are grouped by their
required-features, which preserves thatsemantics exactly — a group can only be built when its feature is on, just as
each individual bench could before:
string_expressions["string_expressions"]math_expressions["math_expressions"]unicode_expressions["unicode_expressions"]datetime_expressions["datetime_expressions"]regex_expressions["regex_expressions"]misccrypto,encodinganddictionary_encodingstay standalone — they are theonly members of their feature groups, so grouping them would buy nothing.
Each group lives in its own directory,
benches/<group>/, withmain.rsdeclaring the individual benchmarks as modules and listing their criterion
groups in a single
criterion_main!. Cargo does not auto-discover targetsinside those directories, so no
autobenches = falseis needed — which alsokeeps
cargo macheteworking (it stops scanningbenches/when autodiscoveryis disabled).
benches/helper.rsstays shared at the top level, reached with#[path = "../helper.rs"].The benchmark code itself is untouched. There is not a single change to a
bench_function,benchmark_grouporbench_with_inputline anywhere in thediff — git reports every moved file as a 94–99% similarity rename. Per file the
change is only:
criterion_main!— a module cannot definemaincriterion_mainimportextern crate criterion(a no-op since edition 2018, present in 4 files)benches/helper.rsatcrate::helperAre these changes tested?
Interleaved with
mainso machine drift cancels out:cargo clean -p datafusion-functionsthen…cargo build --benchescargo check --benchesEvery benchmark still runs, and the counts show nothing was dropped:
-- --test)string_expressionsunicode_expressionsdatetime_expressionsdictionary_encodingregex_expressionsmiscencodingcrypto--features crypto_expressions)Also clean:
cargo clippy -p datafusion-functions --all-targets,cargo fmt --check, andcargo machete --with-metadata.Filtering a single benchmark still works —
cargo bench --bench math_expressions -- powerselects exactly one.cargo metadatareports 10 bench targets, against 64 onmain; both countsinclude the auto-discovered
helper, which is unchanged by this PR.Are there any user-facing changes?
Only for running an individual benchmark. Where you would previously write
you now select the group and filter by name:
The filter is criterion's own, matching on benchmark id, so it also narrows to a
single case within a benchmark. No library changes.
If this approach looks right, roughly 100 more bench targets across the other
crates could get the same treatment — I did this one crate first so the tradeoff
is visible on a real diff before it spreads.
🤖 Generated with Claude Code