Edge-Triggered Epoll and Backend Improvements#96
Merged
sgerbino merged 10 commits intocppalliance:developfrom Jan 31, 2026
Merged
Edge-Triggered Epoll and Backend Improvements#96sgerbino merged 10 commits intocppalliance:developfrom
sgerbino merged 10 commits intocppalliance:developfrom
Conversation
Enhance the benchmark infrastructure with machine-readable output and
improved usability for CI/automation workflows.
Changes to bench/common/benchmark.hpp:
- Add metric, benchmark_result, and result_collector classes
- result_collector serializes to JSON with metadata (backend, timestamp)
- benchmark_result supports fluent API for adding metrics
- Add add_latency_stats() helper for statistics objects
Changes to all benchmark executables:
- Add --output <file> to write JSON results (stdout still works)
- Add --bench <name> to run a single benchmark instead of all
- Add --help with list of available benchmarks
- Refactor to use run_benchmarks() function for consistent structure
- Add descriptive comments explaining what each benchmark measures
and why it's useful
JSON output format:
{
"metadata": {"backend": "epoll", "timestamp": "..."},
"benchmarks": [
{"name": "...", "metric": value, ...}
]
}
This enables:
- Programmatic consumption of results (CI, regression tracking)
- Quick iteration by running only specific benchmarks
- Better documentation of benchmark purpose
Add a mock HTTP server benchmark that measures request throughput using read_until with dynamic buffers. Both implementations use equivalent composed operations for fair comparison. Benchmark scenarios: - single_conn: Single connection, sequential requests - concurrent: Multiple concurrent connections (1, 4, 16, 32) - multithread: Multi-threaded run() with varying thread counts
Replace per-operation epoll_ctl(ADD)/epoll_ctl(DEL) with persistent registration using EPOLLONESHOT. File descriptors are registered once and re-armed via epoll_ctl(MOD) when operations need to wait. - Add descriptor_data struct to track per-fd registration state - Implement lazy registration (register on first wait, not on open) - Use EPOLLONESHOT to auto-disarm after events - Apply to both sockets and acceptors - Remove legacy per-operation registration code paths Reduces epoll_ctl calls from 2 to 1 per waiting I/O operation.
- Add bench/common/backend_selection.hpp with shared dispatch_backend() utility for runtime backend selection - Update all corosio benchmarks to support --backend and --list options - Change make_socket_pair to accept basic_io_context& for polymorphic context support, enabling benchmarks to use any backend type
Switch from level-triggered to edge-triggered epoll (EPOLLET) mode. This reduces epoll_ctl syscalls by registering descriptors once with all events rather than modifying per-operation. Atomic operations coordinate between reactor and cancellation paths to prevent races. Readiness caching handles edge events that arrive before operations are registered. Also enables socket_stress tests on Linux.
Enable stress tests to run on both epoll and select backends using template-based test implementations. Fix timer starvation in the select scheduler that caused tests to hang when synchronous I/O completions were continuously posted without going through the reactor. Timers are now processed at the start of each do_one() iteration, matching the epoll scheduler behavior.
Set impl_ptr immediately after work_started() to keep the socket/acceptor impl alive while the operation is pending. Previously, impl_ptr was only set in sync completion and cancel paths, leaving a window where the reactor could complete an operation after the impl was destroyed. Fixes segfault in accept stress test on macOS.
Replace explicit atomic_thread_fence(seq_cst) with seq_cst on the atomic operations themselves. The fence after every operation registration was expensive (10-100 cycles). Moving the ordering guarantee into the store and exchange operations achieves the same synchronization with less overhead.
Remove line divider comments and section headers that just repeat function names. Add brief Javadoc to impl classes. Preserves meaningful comments that explain why, not what.
- Rename async_wait() to wait() for consistency with timer - Change documentation to use capy::cond::canceled instead of capy::error::canceled for error condition comparisons - Add usage examples showing correct cancellation checking pattern - Update all references in tests, docs, and implementation comments
📝 WalkthroughWalkthroughThis PR refactors the epoll backend to use persistent per-descriptor registration instead of per-operation state tracking, introduces a new Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 2✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Comment |
|
An automated preview of the documentation is available at https://96.corosio.prtest3.cppalliance.org/index.html If more commits are pushed to the pull request, the docs will rebuild at the same URL. 2026-01-31 02:53:16 UTC |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #96 +/- ##
===========================================
+ Coverage 82.14% 82.21% +0.06%
===========================================
Files 56 58 +2
Lines 4951 5195 +244
===========================================
+ Hits 4067 4271 +204
- Misses 884 924 +40
... and 6 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
This was referenced Feb 3, 2026
Merged
Merged
Merged
Merged
This was referenced Feb 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces edge-triggered epoll (EPOLLET) with persistent descriptor registration, significantly improving latency under concurrent load while maintaining throughput parity. It also includes bug fixes, documentation improvements, and new benchmarking infrastructure.
Key Changes
1. Edge-Triggered Epoll Implementation
Switched from level-triggered to edge-triggered epoll with persistent registration:
epoll_ctlcallsEPOLLETflag for more efficient event notificationread_ready/write_readyflags cache edge events that arrive before an operation is registered, preventing missed eventsseq_cstfor critical synchronization between operation registration and reactor event delivery2. Performance Results
Latency improvements (lower is better):
HTTP Server (concurrent connections):
3. Bug Fixes
impl_ptr = shared_from_this()immediately afterwork_started()in async operation pathsdo_one()loop to prevent timers from starving when handlers are continuously posted4. Documentation Improvements
signal_set::async_wait()towait()for consistency withtimer::wait()capy::cond::canceledinstead ofcapy::error::canceled5. New Benchmarking Infrastructure
--backendflag to select epoll or select--jsonflag for machine-readable benchmark results--onlyflag to run specific benchmark suites6. Multi-Backend Test Support
Socket stress tests now run on both epoll and select backends:
boost.corosio.socket_stress.*- epoll backendboost.corosio.socket_stress.*.select- select backendSummary by CodeRabbit
Release Notes
API Changes
async_wait()towait()Improvements
capy::cond::canceledDocumentation
wait()API usage✏️ Tip: You can customize this high-level summary in your review settings.