Skip to content

perf(build): don't build the doc-generator binaries by default - #24338

Merged
Dandandan merged 1 commit into
apache:mainfrom
Dandandan:perf/gate-doc-bins
Aug 13, 2026
Merged

perf(build): don't build the doc-generator binaries by default#24338
Dandandan merged 1 commit into
apache:mainfrom
Dandandan:perf/gate-doc-bins

Conversation

@Dandandan

@Dandandan Dandandan commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Adresses: #13814

Found while profiling compile times for #24325 / #24326 / #24329 / #24330.

Rationale for this change

datafusion/core/src/bin/ holds three binaries that regenerate the docs under
docs/source/user-guide: print_config_docs, print_runtime_config_docs and
print_functions_docs. Cargo auto-discovers them and they have no
required-features, so every cargo build links all three — each one
~174MB, since each links the whole datafusion rlib.

Nothing in normal development uses them. They are run by
dev/update_config_docs.sh and dev/update_function_docs.sh, and by the CI job
that checks the committed docs are up to date.

Two places where this shows up:

Cold builds. The three binaries link after every other unit has finished,
so they sit on the critical path with nothing to overlap with.
cargo build --timings shows them occupying the last 3.5s of a
cargo build -p datafusion (~8.8s of CPU), after the last library unit
completes.

The tightest inner loop — touch a file in core, rebuild. All three are
relinked every time:

before: 3.0s  2.4s
after:  1.3s  1.1s

What changes are included in this PR?

The three binaries move behind a new non-default docs_generation feature, and
the two dev/ scripts pass --features docs_generation.

Using required-features means declaring the [[bin]] targets explicitly, since
auto-discovered targets cannot carry it.

Are these changes tested?

  • cargo build -p datafusion no longer produces the three binaries
  • cargo build -p datafusion --features docs_generation does
  • ./dev/update_config_docs.sh still regenerates
    docs/source/user-guide/configs.md byte-identically (empty git diff
    afterwards), which is what the CI doc check compares

dev/update_function_docs.sh uses the same invocation pattern and all three of
its call sites were updated; CI exercises both scripts.

Are there any user-facing changes?

The three binaries are no longer built by a default cargo build. Anyone who
ran them directly needs --features docs_generation — same as the dev/ scripts
now do. No library API changes.

If you would rather these lived outside the published crate altogether, moving
them to a small non-published dev/ crate would have the same effect on build
times; I went with the smaller change.

🤖 Generated with Claude Code

`datafusion/core/src/bin/` holds three binaries that regenerate the docs under
`docs/source/user-guide`: `print_config_docs`, `print_runtime_config_docs` and
`print_functions_docs`. They are auto-discovered by cargo with no
`required-features`, so every `cargo build` links all three -- each ~174MB,
against a crate whose own rlib they link in full.

They are only ever run by `dev/update_config_docs.sh` and
`dev/update_function_docs.sh` (and by the CI job that checks the committed docs
are up to date), so they now sit behind a non-default `docs_generation` feature,
and those scripts pass `--features docs_generation`.

Effect on the two builds that matter:

- Cold `cargo build -p datafusion`: the three binaries link *after* everything
  else has finished, so they sit on the critical path with nothing to overlap
  with. `cargo build --timings` shows them occupying the last 3.5s of the build
  (~8.8s of CPU).
- Edit-a-file-in-core and rebuild, the tightest inner loop -- they are relinked
  every time:

      before: 3.0s  2.4s
      after:  1.3s  1.1s

Verified that a default `cargo build -p datafusion` no longer produces them,
that `--features docs_generation` does, and that `./dev/update_config_docs.sh`
still regenerates `docs/source/user-guide/configs.md` byte-identically.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions github-actions Bot added development-process Related to development process of DataFusion core Core DataFusion crate labels Aug 13, 2026
@Dandandan Dandandan changed the title build: don't build the doc-generator binaries by default perf(build): don't build the doc-generator binaries by default Aug 13, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.17%. Comparing base (ab12f5e) to head (17ab914).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24338      +/-   ##
==========================================
+ Coverage   81.14%   81.17%   +0.03%     
==========================================
  Files        1112     1109       -3     
  Lines      386933   386746     -187     
  Branches   386933   386746     -187     
==========================================
- Hits       313967   313957      -10     
+ Misses      54476    54295     -181     
- Partials    18490    18494       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@comphead comphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Dandandan good catch

@Dandandan
Dandandan added this pull request to the merge queue Aug 13, 2026
Merged via the queue into apache:main with commit f171587 Aug 13, 2026
40 checks passed
@Dandandan
Dandandan deleted the perf/gate-doc-bins branch August 13, 2026 18:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate development-process Related to development process of DataFusion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants