Skip to content

perf(build): use line-tables-only debug info in the dev profile - #24339

Merged
Dandandan merged 1 commit into
apache:mainfrom
Dandandan:perf/dev-debuginfo
Aug 13, 2026
Merged

perf(build): use line-tables-only debug info in the dev profile#24339
Dandandan merged 1 commit into
apache:mainfrom
Dandandan:perf/dev-debuginfo

Conversation

@Dandandan

@Dandandan Dandandan commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Adresses: #13814

Found while profiling compile times for #24325 / #24326 / #24329 / #24330,
which removed the trait-solving cost from the four crates on the critical path
and left LLVM as the dominant remaining cost.

Rationale for this change

dev is the profile behind every cargo build and cargo test, so its debug
info is generated over and over. debug = "line-tables-only" keeps file and line
numbers — panics and RUST_BACKTRACE output stay just as useful — and drops the
variable-level DWARF that only an interactive debugger consumes.

Measured per crate, interleaved with the baseline so machine drift cancels
out. The flag is passed to the crate under test only, so cached dependency
artifacts stay valid and nothing else moves between the two measurements:

crate debug = 2 line-tables-only
datafusion-physical-plan 8.89s 7.43s −16%
datafusion-functions-aggregate 5.86s 4.63s −21%
datafusion-physical-expr 4.57s 3.59s −21%
datafusion-functions 4.64s 4.04s −13%
datafusion-expr 4.54s 3.57s −21%
datafusion-functions-nested 4.37s 3.06s −30%
datafusion-optimizer 3.91s 3.12s −20%
datafusion-common 3.73s 3.10s −17%
datafusion-sql 3.57s 2.43s −32%
datafusion-datasource-parquet 3.27s 2.44s −25%
datafusion-datasource 1.90s 1.42s −25%
datafusion-physical-optimizer 1.22s 0.99s −19%
sum 50.5s 39.8s −21%

The saving is codegen-side, as you would expect: datafusion-catalog, which
spends its time in the trait solver rather than in LLVM, moves only 7.4s → 7.0s.

Artifacts shrink as well — libdatafusion_physical_plan.rlib goes from 141MB
to 100MB
.

What changes are included in this PR?

One setting on [profile.dev], plus an update to the profile documentation block
above it, which currently advertises "full debug info" for dev.

Are these changes tested?

Are there any user-facing changes?

For anyone stepping through DataFusion in a debugger, local variable inspection
needs CARGO_PROFILE_DEV_DEBUG=2 cargo build (or a local override in
.cargo/config.toml); the comment in Cargo.toml says so. Everything else —
panic locations, backtraces, #[test] failures — is unchanged.

🤖 Generated with Claude Code

`dev` is the profile behind every `cargo build` and `cargo test`, so its debug
info is generated over and over. `line-tables-only` keeps file and line numbers
-- panics and `RUST_BACKTRACE` output stay just as useful -- and drops the
variable-level DWARF that only an interactive debugger consumes.

Measured per crate, interleaved with the baseline so machine drift cancels out.
The flag is passed to the crate under test only, so cached dependency artifacts
stay valid and nothing else moves:

    crate                          debug=2   line-tables-only
    datafusion-physical-plan         8.89s        7.43s
    datafusion-functions-aggregate   5.86s        4.63s
    datafusion-physical-expr         4.57s        3.59s
    datafusion-functions             4.64s        4.04s
    datafusion-expr                  4.54s        3.57s
    datafusion-functions-nested      4.37s        3.06s
    datafusion-optimizer             3.91s        3.12s
    datafusion-common                3.73s        3.10s
    datafusion-sql                   3.57s        2.43s
    datafusion-datasource-parquet    3.27s        2.44s
    datafusion-datasource            1.90s        1.42s
    datafusion-physical-optimizer    1.22s        0.99s
    -------------------------------------------------------
    sum                             50.5s        39.8s   (-21%)

The saving is codegen-side, as expected: `datafusion-catalog`, which spends its
time in the trait solver rather than in LLVM, moves only 7.4s -> 7.0s.

Artifacts shrink too -- `libdatafusion_physical_plan.rlib` goes from 141MB to
100MB.

Verified that backtraces keep their file and line numbers under the new setting:

    thread 'main' panicked at datafusion/core/src/bin/print_functions_docs.rs:38:9
       at ./datafusion/core/src/bin/print_functions_docs.rs:38:9

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Dandandan Dandandan changed the title build: use line-tables-only debug info in the dev profile perf(build): use line-tables-only debug info in the dev profile Aug 13, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.14%. Comparing base (ab12f5e) to head (50f75f1).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #24339   +/-   ##
=======================================
  Coverage   81.14%   81.14%           
=======================================
  Files        1112     1112           
  Lines      386933   386933           
  Branches   386933   386933           
=======================================
+ Hits       313967   313976    +9     
+ Misses      54476    54463   -13     
- Partials    18490    18494    +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@comphead comphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Dandandan makes sense for local builds.

@Dandandan
Dandandan added this pull request to the merge queue Aug 13, 2026
Merged via the queue into apache:main with commit e76f1af Aug 13, 2026
42 checks passed
@Dandandan
Dandandan deleted the perf/dev-debuginfo branch August 13, 2026 19:19
@alamb

alamb commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants