feat(sqlite): Add SqliteRwPool with a single writer and multiple readers#4177
feat(sqlite): Add SqliteRwPool with a single writer and multiple readers#4177emschwartz wants to merge 6 commits intolaunchbadge:mainfrom
SqliteRwPool with a single writer and multiple readers#4177Conversation
…ling Adds SqliteRwPool, a connection pool that maintains a single writer and multiple readers for SQLite WAL-mode databases. Queries are automatically routed based on SQL analysis — SELECTs, EXPLAINs, read-only PRAGMAs, and read-only WITH CTEs go to readers; everything else goes to the writer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use try_from instead of truncating cast, unwrap_or instead of unnecessary closure, and unwrap_or_default for default construction. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
emschwartz
changed the title
SqliteRwPool with a single writer and multiple readersSqliteRwPool with a single writer and multiple readers
Replace tokio::spawn with sqlx_core::rt::spawn so the test works with any async runtime (e.g. async-global-executor), not just Tokio. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Note the failing check is unrelated to this PR: |
Yeah, I'm not super comfortable with this. I think it should be an explicit choice. Not only is that the more Rust-y way to do it, there's lots of possible cases that could break these heuristics, like one of these keywords appearing in a SQL comment. In fact, the code But I don't think the complexity and fragility is a good trade-off anyway because the reader would not be able to determine at a glance which pool is going to be used for a given query. Given that this whole issue arose because the API didn't inform you clearly enough that an informed decision needed to be made here, trying to make this just "work" automagically seems antithetical. |
Users must now explicitly call .reader() or .writer() to route queries. The Executor impl on &SqliteRwPool always delegates to the writer pool. This removes the SQL classification heuristics and simplifies the Executor impl to a direct delegation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Fair enough! Removed the auto-routing. |
2 participants
Since SQLite is single-writer, it is more performant to have a single writer and multiple readers and queue writes at the application level than to have different processes or tasks competing to obtain the exclusive write lock. This PR adds an
SqliteRwPoolthat implements this behavior.Based on the discussion in https://www.reddit.com/r/rust/comments/1r7eh9v/comment/o652285/
This is not a breaking change, because it only adds new functionality. It might make sense to update the
sqlx::sqlitemodule docs to more strongly encourage users to use this pool type, but I'll leave that for the maintainers to decide.Two features that I want to specifically call attention to are:
Auto-routing queries: by default, this implementation will route queries to either the writer or reader pool depending on whether the SQL statement contains write or transaction keywords. This behavior can be disabled, and users can also callreaderorwriterto explicitly use the given pool for a query.closeon the pool, it will attempt to run a passive checkpoint to merge the WAL back into the main database as long as there are no other processes reading from the database at that time. This behavior can be turned off with a setting. While this does implement a good practice, it might be considered out of scope for a library likesqlx.This PR was written with the help of Claude Code. I directed it and reviewed the code it generated.