-
Notifications
You must be signed in to change notification settings - Fork 3
fix: incorrect link of flamegraph and URI #41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR fixes the marker emission pattern for CodSpeed benchmarking by ensuring sample markers wrap around all benchmark repetitions, with benchmark start only being called once on the first repetition. Previously, start_benchmark was called on every repetition, which would emit multiple nested MARKER_TYPE_SAMPLE_START/END pairs. Now the markers are emitted in the correct format where a single sample start/end pair wraps around all benchmark iterations, with multiple benchmark start/stop markers within.
Changes:
- Added
IsFirstRepetition()helper method to check if a benchmark runner is on its first iteration - Modified
start_benchmarkcall to only execute on the first repetition - Added
MARKER_TYPE_SAMPLE_STARTmarker emission at the beginning ofstart_benchmark() - Added
MARKER_TYPE_SAMPLE_ENDmarker emission at the beginning ofend_benchmark()
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| google_benchmark/src/benchmark_runner.h | Added IsFirstRepetition() method to check if benchmark is on first repetition |
| google_benchmark/src/benchmark.cc | Modified condition to only call start_benchmark on first repetition |
| core/src/codspeed.cpp | Added sample start/end markers around benchmark execution |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Merging this PR will degrade performance by 32.51%
Performance Changes
Comparing |
0eff824 to
e311da2
Compare
e311da2 to
660b1f9
Compare
GuillaumeLagrange
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Feels like there are still some issues.
According to the code, we should never see the expensive_operation w/ fibo here
This seems to already have been the case 2 months ago: https://codspeed.io/CodSpeedHQ/codspeed-cpp/runs/692085eb175fc14ee5058b33?uri=examples%2Fgoogle_benchmark_cmake%2Fpause_timing_bench.hpp%3A%3ABM_large_setup&runnerMode=Instrumentation§ionId=benchmark-comparison-section-single-run-benchmarks I'll create a new ticket and fix it there |
040ba03 to
660b1f9
Compare
We modified the logic to exclude the warmup, which caused this regression. We can only have 1 pair of SampleStart/Stop markers (generated by measurement_start/stop) for one benchmark.
Instead of moving the
measurement_start/stopfunctions around to exclude the warmup, we now pass a flag that prevents adding theBenchmarkStart/Stopmarkers.