.pink

POST · RUST

The db_task pattern

A dedicated SQLite worker thread for async Rust. Published . About 6 minutes to read.

All posts apoptotic case study
Tension

Async Rust meets SQLite

SQLite is an exceptionally well-engineered library. rusqlite is an exceptionally well-engineered binding. Neither is async. Neither has any reason to be: SQLite calls do real, blocking, file-system work, and async runtimes are explicitly bad at hiding that.

The naive integration is to wrap a rusqlite::Connection in a Mutex (or an Arc<Mutex<_>>), drop calls into tokio::task::spawn_blocking, and call it a day. That works. It also has a long tail of subtle problems: contention on a single mutex, prepared statements re-keyed across calls, and the unsettling property that every database access spends some time on a tokio worker thread that is now blocked.

Pattern

The pattern in 30 seconds

Spawn one OS thread, give it the connection, and let async code talk to it through a channel:

  • The async side has a Database handle holding an mpsc::Sender<DbRequest>.
  • Each request includes a oneshot::Sender<Result<T, _>> for the reply.
  • The worker thread owns a Connection, reads requests in a loop, runs them, and sends replies back.
  • On shutdown, dropping the last Database closes the channel; the worker observes that, finalizes, and exits.

This is essentially the same shape as an actor — except the messages are SQL operations, and the actor's mailbox is the write serialization point.

Why

Why a thread, not a Mutex

  • No contention. There is no lock to wait on; only the channel, which is engineered for throughput.
  • No spawn_blocking per call. The blocking happens once, in one place, on a thread that exists for that purpose.
  • Connection state is local. Prepared statements live in scope of the worker. No HashMap<Sql, Statement> behind a mutex.
  • Transactions don't leak. A transaction begins and ends inside one request handler; it cannot be picked up by a different async caller mid-transaction.
  • Migrations are obvious. Run them at thread startup, before the worker enters its loop. No "did the migration run yet?" race.
Tradeoffs

What you give up

There is no parallelism for reads. Every request is serialized through one thread. SQLite has WAL mode and concurrent-read tricks; they do not apply if every read goes through the same thread.

For the workloads that drive this pattern — a Discord bot's modest write rate, or an emulator service's bookkeeping — that's fine. If your workload is "thousands of concurrent reads while writes proceed", this pattern is the wrong answer; you want a connection pool with WAL mode and per-task connections.

Migrations

Pragmas and migrations

The worker thread is the right home for both. The boot sequence I use is:

  1. Open the connection.
  2. Set pragmas: journal_mode = WAL, foreign_keys = ON, busy_timeout, optionally synchronous = NORMAL.
  3. Run the migration script — typically a list of CREATE TABLE IF NOT EXISTS statements plus a small migration table.
  4. Run an integrity check.
  5. Enter the request loop.

All of that happens before the Database handle is returned to the rest of the application. If anything fails, the application never starts — which is the correct behavior. A bot that boots without a working database is actively harmful.

Backpressure

Backpressure and shutdown

The mpsc channel is bounded. If the bot is overwhelmed, async callers will eventually wait on send rather than queue infinite work. That backpressure flows naturally to whatever produced the work in the first place; there is no silent unbounded queue.

Shutdown is symmetric. When the application is exiting, all Database handles are dropped. The receiver returns None. The worker breaks out of its loop, runs PRAGMA optimize, closes the connection, and the thread exits. From the outside, that looks like a graceful exit; from the inside, it's three lines of code.

Practice

Where I use it

The pattern is implemented identically in dc_bot/src/db_task.rs (the apoptotic Discord bot) and in emu_service. Once both services were using it, the family resemblance made everything else easier — error handling, logging, failure modes, even the test scaffolding shape.

There's a small amount of boilerplate per service (the request enum, the handler match, the typed reply channels), but it's mechanical and writes itself once you've done it twice.