Skip to content

chore: downsize sql_planner_extended logical_plan_optimize sample size to 5#23659

Merged
2010YOUY01 merged 2 commits into
apache:mainfrom
Jefffrey:sql_planner_extended_sample_size
Jul 18, 2026
Merged

chore: downsize sql_planner_extended logical_plan_optimize sample size to 5#23659
2010YOUY01 merged 2 commits into
apache:mainfrom
Jefffrey:sql_planner_extended_sample_size

Conversation

@Jefffrey

Copy link
Copy Markdown
Contributor

on my M4 Macbook pro, it estimates it would take 50 minutes (!) to run the benchmark

datafusion (main)$ cargo bench -p datafusion --bench sql_planner_extended logical_plan
   Compiling datafusion-common v54.0.0 (/Users/jeffrey/Code/datafusion/datafusion/common)
...
   Compiling datafusion v54.0.0 (/Users/jeffrey/Code/datafusion/datafusion/core)
    Finished `bench` profile [optimized] target(s) in 8m 11s
     Running benches/sql_planner_extended.rs (/Users/jeffrey/.cargo_target_cache/release/deps/sql_planner_extended-c026520b80bba716)
Gnuplot not found, using plotters backend
Benchmarking logical_plan_optimize: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 3077.4s, or reduce sample count to 10.
Benchmarking logical_plan_optimize: Collecting 100 samples in estimated 3077.4 s (100 iterations)

and running it via the bot just times out: #23215 (comment)

so might actually be useful to reduce the sample size so it can actually run and be of use to us

@github-actions github-actions Bot added the core Core DataFusion crate label Jul 17, 2026

@2010YOUY01 2010YOUY01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me, thank you.

This micro bench takes 30s to run, 10 samples still takes a lot of time. Perhaps we can do 5? The result should not be very noisy since it's a very long-running bench case.

@codecov-commenter

codecov-commenter commented Jul 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.67%. Comparing base (7ca6e54) to head (a3bfb11).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #23659   +/-   ##
=======================================
  Coverage   80.66%   80.67%           
=======================================
  Files        1086     1086           
  Lines      366694   366691    -3     
  Branches   366694   366691    -3     
=======================================
+ Hits       295806   295817   +11     
+ Misses      53265    53244   -21     
- Partials    17623    17630    +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Jefffrey

Copy link
Copy Markdown
Contributor Author

Makes sense to me, thank you.

This micro bench takes 30s to run, 10 samples still takes a lot of time. Perhaps we can do 5? The result should not be very noisy since it's a very long-running bench case.

i agree, downsized to 5 👍

@Jefffrey Jefffrey changed the title chore: downsize sql_planner_extended logical_plan_optimize sample size to 10 chore: downsize sql_planner_extended logical_plan_optimize sample size to 5 Jul 17, 2026
@2010YOUY01
2010YOUY01 added this pull request to the merge queue Jul 18, 2026
Merged via the queue into apache:main with commit 1734d4b Jul 18, 2026
38 checks passed
@Jefffrey
Jefffrey deleted the sql_planner_extended_sample_size branch July 18, 2026 03:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants