Twelve Years Of Distributed SQLite

TL;DR

Before you orderOffer from Amazon

Get comfort and recovery gear delivered free with Prime

• Fast, free delivery on millions of items
• Prime Video, Amazon Music and more included
• Member-only deals all year

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

A retrospective by rqlite creator Philip O’Toole traces distributed SQLite from rqlite’s 2014 release to a broader category of projects and managed services. It describes three approaches—consensus-based clusters, streamed backups and mergeable writes—and argues they address different needs rather than competing for one use case.

Philip O’Toole, creator of the open-source database project rqlite, has published a retrospective tracing distributed SQLite from its 2014 release to a field that now includes backup tools, alternative replication designs and managed cloud databases. The account matters because it documents how projects built for different failure and consistency needs helped turn replicated SQLite into a wider software category.

O’Toole says he published the first short announcement of rqlite in September 2014 and put its source on GitHub that weekend. The project added a Raft consensus log in front of SQLite so writes could be replicated across a cluster. Its design requires writes to go through a leader and receive acknowledgement from a majority; a minority separated by a network partition stops accepting writes. O’Toole describes that as a CP design, prioritizing consistency and availability of a majority over writes from an isolated minority.

The project began as a solution to a practical need at jut.io, where a clustered system required a small relational store. O’Toole says rqlite first used the goraft library and moved to HashiCorp’s Raft implementation in early 2016. He presents rqlite as a general-purpose server that users run, distinguishing it from Canonical’s dqlite, which was developed from 2017 as an embedded database library for products including LXD and later MicroK8s.

Later projects took different routes. Litestream, released in 2021 by Ben Johnson, streams SQLite’s write-ahead log to object storage for recovery, without coordinating a cluster through consensus. In 2022, cr-sqlite added conflict-free replicated data types, or CRDTs, for merging changes from multiple writers. Turso developed libSQL, a fork of SQLite, and a hosted service with edge replicas. O’Toole’s timeline also includes SQLite Cloud, Cloudflare D1 and SQLite storage in Durable Objects.

Three Replication Needs, Three Designs

The history offers a practical way to distinguish systems that can otherwise be grouped under the broad label distributed SQLite. A consensus cluster such as rqlite aims to keep a service operating through node failures while requiring agreement from a majority. Litestream instead targets low-cost recovery if a single machine fails, accepting that restoration may take time. CRDT-based systems target applications where multiple writers can make changes independently and merge them later.

Those trade-offs affect how developers build and operate applications. A replicated cluster can reduce reliance on restoring a database after a failure, but it brings coordination and deployment requirements. Streaming backups keep a simpler single-node setup, while merge-based systems accept eventual consistency in exchange for independent writes. O’Toole’s account argues that these approaches answer different questions; the proliferation of projects does not mean one design fits every application.

The shift from software people deploy themselves to managed database products also changes who handles operational work. Services such as D1 and Turso package SQLite-related storage behind hosted platforms. That can make the technology easier to consume, but the source account does not provide comparative measurements of reliability, cost or performance across these offerings.

From rqlite to Hosted SQLite

O’Toole says rqlite was, to his knowledge, the first open-source system to combine Raft and SQLite, while acknowledging earlier and parallel work. Expensify had used a Paxos-style protocol internally and released Bedrock in 2016; ActorDB, an Erlang project pairing SQLite databases with Raft, appeared around the same period as rqlite. These examples qualify any account that treats rqlite as the first attempt to replicate SQLite at all.

Canonical’s dqlite followed in 2017 with a different packaging goal: a database library embedded in its products rather than a standalone server. Litestream’s 2021 release helped popularize a contrasting argument, that many applications need backup and recovery more than a multi-node database. In 2022, projects including cr-sqlite and libSQL further widened the approaches, while cloud providers began presenting SQLite-based storage as a managed service.

O’Toole frames the category as having three broad design points: consensus replication, log streaming for recovery, and merging concurrent changes. This is his interpretation of the project history, not a formal classification adopted by every project. His account also notes that Litestream’s work influenced rqlite’s version 8.0 approach to handling SQLite’s write-ahead log.

What the Retrospective Cannot Establish

The account is a first-person history by a project founder, rather than an independent survey of the field. O’Toole says that, as far as he can tell, rqlite was the first open-source Raft-and-SQLite system; the supplied material does not establish that claim through a broader review of earlier software. He also says the category has grown to a dozen projects, but does not provide a complete list or define which systems count.

The retrospective does not compare the projects using shared benchmarks, adoption figures or operational data. It also does not explain the details of every managed service’s replication architecture. Claims about project influence, relative popularity and commercial success should therefore be read as O’Toole’s account unless supported by separate evidence. The source says Supabase acquired Turso in 2026 but gives no date, terms or further detail about the acquisition.

How the Projects Develop

The retrospective does not announce a new rqlite release or set out a forward schedule. The next developments to watch are the projects’ own releases and documentation: how rqlite handles replication and WAL changes, how backup tools approach recovery, and how hosted platforms describe their consistency, availability and data portability.

For developers choosing among these systems, the immediate next step is to match the design to the failure they need to handle: a node loss where waiting for restoration is unacceptable, a single-machine failure where backups are sufficient, or an application that needs independent writers and later conflict resolution. Comparisons of actual service guarantees, costs and recovery times remain necessary; the retrospective does not settle those questions.

Key Questions

What is distributed SQLite?

It describes projects that extend SQLite with replication or remote recovery. The approaches differ: some coordinate writes across a cluster, some stream database changes to storage for recovery, and others merge concurrent updates.

What did rqlite add to SQLite?

According to creator Philip O’Toole, rqlite placed a Raft consensus log in front of SQLite and replicated writes across a cluster. Its design requires a leader and acknowledgement from a majority for writes.

How is Litestream different from rqlite?

Litestream streams SQLite’s write-ahead log to object storage to support recovery after a machine failure. Rqlite uses consensus across a cluster, aiming to continue serving through a node failure without waiting for a restore. The projects address different operational needs.

Does the article establish that rqlite was the first distributed SQLite project?

No. O’Toole says rqlite was, to his knowledge, the first open-source system to combine Raft and SQLite. He also describes earlier or parallel efforts, including Expensify’s internal replication system and ActorDB.

Which distributed SQLite approach should a developer use?

The retrospective does not recommend one system for every application. The choice depends on whether the priority is cluster availability, low-cost recovery from a single-machine failure, or independent writes that can be merged later.

Source: hn