ADR 0002: Compatibility is stated only as a conformance pass rate

ADR 0002: Compatibility is stated only as a conformance pass rate

Status: Accepted Date: 2026-10-24 Scope: orbit-rs Deciders: Ravindra Boddipalli

Context

Earlier documents quoted estimates (“~85%”, “94%”, “100% wire compatible”) that counted parsing as implemented. A recorded-SQL replay of Symphony’s statements (about 336,000, zero differences against PostgreSQL 18) was read as general compatibility although it covers only the SQL Symphony sends.

The conformance corpus (tests/conformance/) is recorded from real PostgreSQL 18 and run over the wire by tests/orbit-protocols/postgres_conformance.rs, with a ratchet in baseline.json. On 2026-10-24 it stood at 768 of 816 records (94.1%). The corpus has no records yet for the extended protocol, COPY, ORM traffic, pg_dump, psql \d* or concurrency, so that figure describes only what is recorded.

Decision

Compatibility is stated only as a pass rate on a corpus that real PostgreSQL 18 also passes, with the date and the corpus size. Estimates, feature-inventory percentages and words such as “complete” or “100%” are not used. A pass rate is always followed by what the corpus does not cover.

The corpus grows toward the goal in ADR 0001: the PostgreSQL pg_regress suite, the extended protocol, COPY, ORM and driver traffic, pg_dump round trips, psql \d*, and concurrency and isolation anomalies. Every file must pass 100% on real PostgreSQL 18 before it is added.

Consequences

• The ratchet in baseline.json may only rise; adding records may lower the percentage, and that is reported rather than hidden.
• Documents quote the figure and date from the conformance plan, and update it when it changes.

Canonical wording

Conformance: N of M records pass (P%), measured YYYY-MM-DD against the PostgreSQL 18 conformance corpus. The corpus does not yet cover: .

When no corpus has been recorded for a protocol, the wording is: “Compatibility: not yet measured.” It is never replaced by an estimate or by a status word such as “Complete” or “Production Ready”.

Prior measurement (2026-10-25): PostgreSQL 768 of 816 (94.1%); Redis RESP 247 of 666 (37.1%, Redis 8.10.2); MongoDB 59 of 134 (44.0%, 8.3.13); AQL 94 of 258 (36.4%, ArangoDB 3.12.12); CQL 63 of 208 (30.3%, Cassandra 5.0.9); MySQL 60 of 300 (20.0%, 9.7.4); Cypher/Bolt 9 of 115 (7.8%, Neo4j 2026.10.1). OrbitQL, HTTP REST and gRPC: not yet measured. Each corpus is a first, small one. Not yet covered: the extended protocol, COPY, ORM traffic, pg_dump, psql \d*, concurrency.

Current (2026-10-27): PostgreSQL 789 of 816 (96.7%) against the same 816-record corpus. The extended protocol, COPY, ORM traffic, pg_dump, psql \d* and concurrency remain unmeasured.

Verification

make conformance (ratchet); make conformance-reference (the corpus passes 100% on real PostgreSQL 18, needs Docker); make adr-check (retired claims).