ADR 0004: PostgreSQL SQL runs on a durable RocksDB engine replicated by Raft

ADR 0004: PostgreSQL SQL runs on a durable RocksDB engine replicated by Raft

Status: Accepted Date: 2026-10-23 Scope: orbit-rs Deciders: Ravindra Boddipalli

Context

Measured on 2026-10-23, the shipped server persisted only CREATE TABLE, DROP TABLE and INSERT; SELECT, UPDATE and DELETE ran in memory and tables vanished on restart. Raft was leader election only: no log entries were checked or appended, and data was not replicated.

Decision

SQL runs on a durable engine (sql/durable/) with RocksDB as the source of truth, typed values, snapshot transactions with one writer at a time, and write-ahead recovery. Replication is Raft over that engine (raft/sql.rs): a write is acknowledged only after a majority has persisted it; followers refuse writes (25006); pre-vote, a leader lease, snapshots and single-server membership changes are implemented. It was built in five milestones, M1 (persistent single node) to M5 (authentication, TLS, backup).

Consequences

• Known limits at 2026-10-23, tracked as work toward ADR 0001: the working set must fit in RAM on each node; reads scan (no index lookups); one writer at a time; a Raft snapshot travels as one in-memory frame; ids can skip after a crash or failover but are never reused.
• Write latency in a 3-node cluster measured about 1.7x (p95) and 5x (p99) PostgreSQL’s.

Verification

Flow cases pass on a three-node cluster with the leader killed mid-run; committed_rows_survive_a_killed_server; the Raft simulator (loss, duplication, reordering, partitions, crash and restart). Results are recorded in Symphony’s docs/DATA_DURABILITY.md.