The overview and status pages quote “500k+ messages/second”, “~50MB typical (vs ~300MB JVM equivalent)”, “<100ms cold start (vs 2-5s)”, “1-5ms P99 latency (vs 10-50ms)” and “enterprise-grade with proven benchmarks”, with no benchmark attached. The only latency measurements taken while writing ADR 0009’s implementation were on a debug build: about 0.35 ms per simple statement, commits averaging about 11 ms, and one query that took 48 ms at 9,000 rows because it copied the table. Those numbers show the claims cannot be left unchecked, and also that a debug-build number must never be quoted.
Decision
A performance figure appears in documentation only if it comes from a benchmark in the repository (benchmarks/, orbit/*/benches/) that anyone can run with one documented command, and the figure is published with:
the commit, date and build profile (release only);
the hardware (CPU, cores, memory, storage) and operating system;
the workload (statement mix, data size, client count, connection pooling, replication factor and whether writes were acknowledged by a majority);
the statistic (p50, p99, throughput), the run length and the number of runs, with the spread;
for any comparison with another system (PostgreSQL, a JVM implementation), that system run on the same hardware and workload by the same harness, with its version and configuration.
A figure that is a design target is written as a target, never as a result. Per-core and “vs” figures need the per-core and baseline measurements. A number older than the last change to the code path it measures is marked stale in the document or removed.
Consequences
The current “500k+”, memory, binary size, cold start and P99 claims are removed until a benchmark backs each, or kept as clearly labelled targets.
benchmarks/ gets a results file per release candidate; documents quote it rather than copy numbers by hand (ADR 0017).
Performance work (such as the whole-table MAX/MIN change that cut tenant setup from 173 s to 35 s) is recorded as a before/after in the commit, not in marketing text.
Canonical wording
Performance: (), measured on at commit , release build; reproduce with ``.
When none exists: “Performance: not yet benchmarked.”
Verification
Each quoted figure links to its result file; scripts/check_adr.py flags “vs JVM” and “proven benchmarks” wording; a reviewer can reproduce a figure with the command shown.