Skip to content

feat(true/false): add benchmarks for startup performance#10996

Merged
sylvestre merged 2 commits intouutils:mainfrom
naoNao89:feat/true-false-benchmarks
Mar 2, 2026
Merged

feat(true/false): add benchmarks for startup performance#10996
sylvestre merged 2 commits intouutils:mainfrom
naoNao89:feat/true-false-benchmarks

Conversation

@naoNao89
Copy link
Contributor

Add benchmarks for true/false commands to measure startup performance. Includes both function call and process spawn benchmarks.

Closes #10837

@oech3
Copy link
Contributor

oech3 commented Feb 17, 2026

I think --help/--version in not needed since they are not frequently used.

@oech3
Copy link
Contributor

oech3 commented Feb 17, 2026

while someone might use true any strings via dummy symlinks.

@naoNao89 naoNao89 force-pushed the feat/true-false-benchmarks branch 2 times, most recently from 425649c to 578c129 Compare February 17, 2026 16:00
@naoNao89
Copy link
Contributor Author

oke

├─ false_consecutive_calls    188.8 µs      │ 312.7 µs      │ 228.1 µs      │ 229.5 µs      │ 100     │ 100
├─ false_no_args              2.073 µs      │ 8.998 µs      │ 2.218 µs      │ 2.36 µs       │ 100     │ 200
├─ false_spawn_no_args        2.022 ms      │ 10.3 ms       │ 2.737 ms      │ 3.181 ms      │ 100     │ 100
├─ false_spawn_with_args      1.964 ms      │ 21.17 ms      │ 3.123 ms      │ 4.002 ms      │ 100     │ 100
├─ false_uu_app_construction  83.68 µs      │ 1.522 ms      │ 102 µs        │ 127.6 µs      │ 100     │ 100
╰─ false_with_args            2.171 µs      │ 20.25 µs      │ 2.582 µs      │ 2.886 µs      │ 100     │ 100

@naoNao89 naoNao89 force-pushed the feat/true-false-benchmarks branch from 578c129 to 909f2b4 Compare February 17, 2026 16:01
@oech3
Copy link
Contributor

oech3 commented Feb 17, 2026

We want to sacrifice performance for cold uu_app() calls for hot pathes. So we don't want to bench them.
(In other words, bench for hot pathes are used to block accidentaly re-adding uu_app to hot pass by refactoring by someone)

Add benchmarks to evaluate true/false command performance,
addressing GitHub issue uutils#10837.

Benchmarks include:
- Function call benchmarks (measure uumain directly)
  * true_no_args, true_help, true_version, true_consecutive_calls
- Process spawn benchmarks (measure real command startup)
  * true_spawn_no_args, true_spawn_help, true_spawn_version

This allows comparison between hot path vs cold path performance
and internal logic vs real-world startup time.
@naoNao89 naoNao89 force-pushed the feat/true-false-benchmarks branch from 909f2b4 to 50e5eea Compare February 17, 2026 16:10
@naoNao89
Copy link
Contributor Author

oke, my changes satisfy that requirement

@sylvestre
Copy link
Contributor

🆕 20 new benchmarks

I don't think we need 20 benchmarks for true & false
please make them 2 for each

@oech3
Copy link
Contributor

oech3 commented Feb 26, 2026

#10996 (comment)

Should we keep uumain bench only (considering we can't do anything for Rust's runtime itself)?

@sylvestre
Copy link
Contributor

yeah

@github-actions
Copy link

GNU testsuite comparison:

GNU test failed: tests/misc/io-errors. tests/misc/io-errors is passing on 'main'. Maybe you have to rebase?
Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/tail/inotify-dir-recreate (passes in this run but fails in the 'main' branch)

@codspeed-hq
Copy link

codspeed-hq bot commented Feb 28, 2026

Merging this PR will improve performance by 4.74%

⚡ 1 improved benchmark
✅ 293 untouched benchmarks
🆕 8 new benchmarks
⏩ 42 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 Simulation false_no_args N/A 2.2 µs N/A
🆕 Simulation false_consecutive_calls N/A 28.6 µs N/A
🆕 Simulation true_no_args N/A 2.2 µs N/A
🆕 Simulation true_consecutive_calls N/A 28.3 µs N/A
🆕 Memory true_no_args N/A 33 B N/A
🆕 Memory true_consecutive_calls N/A 33 B N/A
🆕 Memory false_no_args N/A 33 B N/A
🆕 Memory false_consecutive_calls N/A 33 B N/A
Simulation shuf_input_range[1000000] 89.7 ms 85.6 ms +4.74%

Comparing naoNao89:feat/true-false-benchmarks (667f1ca) with main (a611f98)

Open in CodSpeed

Footnotes

  1. 42 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@oech3
Copy link
Contributor

oech3 commented Mar 2, 2026

Is this OK to merge? Also I think RAM bench is not needed (not sure can we remove them or not...).

@sylvestre sylvestre merged commit d24bf80 into uutils:main Mar 2, 2026
162 of 163 checks passed
@oech3
Copy link
Contributor

oech3 commented Mar 2, 2026

I think true and false have some useless signal handling coming from #[uucore::main], but I can't remove them soon.

@naoNao89 naoNao89 deleted the feat/true-false-benchmarks branch March 4, 2026 11:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

true/false: Add benchmark

3 participants