Relay — ADR Deep Dives

Version: 1.0 Companion to: 05-sad.md §9 Purpose: The ADRs in the SAD are deliberately terse — a decision, its drivers, and its rejected alternatives in a paragraph. This document is the long form: for each ADR, the problem as it actually presents itself, the options with their real trade-offs, why the decision falls out, and the conditions under which it should be reversed. The SAD remains the source of truth for what was decided; this document exists so the why survives contact with a skeptical reviewer.

Each deep dive follows the same shape: Problem → Options → Analysis → Decision → Consequences → Revisit when.

ADR-01 — TypeScript/Node.js for all services

Problem

Six services need a runtime. The conventional microservices instinct is "right tool per service" — Go for the gateway, maybe Rust for hot paths, Node for the dashboard. The question is whether polyglot buys anything here, and what it costs.

Options

1. TypeScript/Node everywhere — one language, one toolchain, shared packages.
2. Go for gateway + API, TS for dashboard/SDK — the "correct" pairing by folk wisdom.
3. Polyglot showcase — a different language per service, to demonstrate range.

Analysis

The decisive observation: the SDK must be TypeScript regardless (FR-SDK-01 targets browsers, Node, React Native). The WebSocket protocol has two ends, and the frame types, cursor semantics, and idempotency-key logic exist on both. If the server is also TS, the protocol lives in one shared package (@relay/protocol) consumed by gateway, API service, and SDK — a frame type change is one commit, and drift between server and client serialization becomes a compile error instead of a production incident. If the server is Go, that contract is maintained twice, forever, by hand or by codegen machinery that is itself a maintenance surface.

Go's genuine advantages for the gateway — goroutine-per-connection simplicity, lower memory per socket, no GC pauses of consequence — are real but not binding at v1 scale. Node's event loop handles tens of thousands of mostly-idle sockets acceptably (this is precisely the workload it was designed for); the risk is CPU-bound work on the loop, which for us means JWT verification and HMAC signing. Both are measured in tens of microseconds per operation and both can move to worker threads or a sidecar if profiling demands (risk R7's contained escape hatch).

The polyglot showcase deserves explicit rejection because it optimizes for the wrong audience signal. Reviewers with production experience read "five languages, one author" as five half-maintained toolchains, five CI configurations, and no depth anywhere. Depth in one stack — with the protocol-sharing payoff above — is the stronger portfolio statement, and D8 (one engineer must run and reason about it) makes it the only sustainable choice.

Decision

TypeScript/Node 20+ everywhere. Shared workspace packages for protocol types, error codes, and the repository layer's tenant-scoping interfaces.

Consequences

Positive: one pnpm workspace, one test runner, one lint config; protocol drift structurally impossible; every service is readable by anyone who can read the SDK. Negative: no in-language escape hatch for CPU-bound hot spots (mitigated: worker threads, then sidecar per R7); per-instance connection ceiling likely lower than Go's (mitigated: it's a fleet-size coefficient, not a correctness issue — measure per R2).

Revisit when

Gateway profiling shows >20% of event-loop time in crypto or serialization at target load; or a second team with different language competence joins.

ADR-02 — NATS JetStream over Kafka

Problem

The architecture's spine (D5, D6) is a durable queue between the operational path and its asynchronous consumers: webhook dispatch, analytics ingestion, dashboard live-stream. The reflexive industry answer is Kafka. The question is whether Kafka's strengths apply to this workload, and what they cost a solo operator.

Options

1. Apache Kafka — the default for event streaming.
2. NATS JetStream — durable streams over the NATS transport.
3. Redis Streams — reuse the already-deployed Redis.
4. Postgres-as-queue (SKIP LOCKED polling on the outbox, no broker at all).

Analysis

Operational asymmetry. Kafka is a distributed system you operate: broker and partition planning up front, rebalancing on expansion, ISR tuning, per-topic retention configuration, JVM heap care, and a consumer-group rebalance model with its own failure modes (rebalance storms under slow consumers). Post-ZooKeeper KRaft removes a component but not the discipline. NATS is a single ~20 MB Go binary; a three-node JetStream cluster is three processes and a config file, and it runs in the local docker-compose (NFR-MNT-03) without ceremony. Streams and consumers are created declaratively by the services themselves — no topic-provisioning workflow. Under D8 this asymmetry is not marginal; it is the decision.

Throughput mismatch. Kafka's architecture — partitioned append-only logs, zero-copy reads, offsets as the only per-consumer state — makes millions of events per second cheap. Our envelope is 10 k events/s (NFR-SCL-05) under ASM-04's 10 M messages/day. JetStream's practical range extends well past our ceiling for this event size. Kafka here is a freight train delivering groceries: the capacity is real and the operational mass is paid whether used or not.

Consumer-model fit. Walk the actual consumers. The webhook dispatcher needs per-message acknowledgment, individual redelivery, and delay-tiered retries (FR-WHK-03's 1 s → 2 h schedule). That is JetStream's native model: pull consumers, explicit ack, nak with backoff, max_deliver feeding dead-letter handling. Kafka acknowledges positions, not messages — retrying message N while progressing past it means building the standard retry-topic topology (retry-1s, retry-5s, …) or adopting a framework; meaningful custom infrastructure to replicate a consumer config. The SSE service needs ephemeral, per-tenant-filtered subscriptions: JetStream subject hierarchies (events.msg.created.{env}) give wildcard filtering for free, and ephemeral consumers cost nothing. In Kafka, per-tenant filtering means either topic-per-tenant (partition explosion at 10 k tenants) or consume-everything-and-discard. Relay's eventing is hierarchical and filtered by nature — NATS's home turf.

Semantics we must build anyway. Kafka's strongest card is exactly-once machinery and the Connect ecosystem. But the outbox (ADR-06) makes publication at-least-once by construction — a relay crash between publish and published_at republishes. Every consumer is therefore idempotent regardless of broker (dedupe on event ID; EIR-WHK-04 pushes the same discipline to customers), and metering's correctness rests on the daily reconciliation against Postgres (FR-ANL-06), which is a requirement — billing accuracy cannot rest on any pipeline's promises. Kafka's stronger guarantees purchase nothing we are not already obligated to build. The broker's guarantees are belt; reconciliation is the suspenders we must wear anyway.

Why not Redis Streams: it couples the durability spine to the deliberately-lossy ephemeral store (ADR-07). One Redis incident would then have two blast radii, and it forces AOF/fsync persistence configuration onto a component currently run in its simplest, fastest mode. Failure-domain separation is worth one more moving part.

Why not Postgres-as-queue: viable at v1 scale and zero new infrastructure — but it puts consumer polling load on the operational database (grazing CON-01's spirit), lacks subject filtering for the SSE path, and the outbox relay already gives us the Postgres-side durability; the broker adds fan-out and consumer independence cheaply.

Decision

NATS JetStream, 3-node cluster, R3 replication on streams. Subjects events.{domain}.{action}.{env}; durable pull consumers for dispatcher and ingester; ephemeral consumers for SSE.

Consequences

Positive: near-zero operational load; retry/DLQ semantics from consumer config; one transport for durable and ephemeral messaging; trivially local-runnable. Negative: no connector ecosystem (any future "stream events to customer's warehouse" feature means hand-built sinks); weaker exactly-once story (already mitigated by design); smaller hiring/knowledge pool than Kafka.

Revisit when

Sustained volume approaches ~100 k events/s; or a customer-facing event-export feature makes the Connect ecosystem valuable; or long-horizon replayable history becomes a product requirement (today: 24 h buffer + Postgres as truth). Migration is contained by construction: ADR-06 means production always starts at the outbox — swap the relay's target and re-implement consumer subscription; event production is untouched.

ADR-03 — Per-channel sequences via last_sequence row lock

Problem

FR-MSG-02/03 demand strict, server-assigned, per-channel ordering, and the resume protocol (FR-RTM-03) plus fan-out dedup (ADR-07) lean on it. Where do gap-tolerant, monotonic, per-channel sequence numbers come from in a system with multiple API instances?

Options

1. UPDATE channels SET last_sequence = last_sequence + 1 … RETURNING inside the message-insert transaction (row lock).
2. Per-tenant sequence — one counter per environment.
3. Postgres SEQUENCE objects, one per channel.
4. Time-based IDs (Snowflake/ULID) — coordination-free.

Analysis

The row-lock option is usually dismissed on sight — "a lock on every message!" — and the dismissal is wrong twice over. First, the contention scope is a single channel's row: sends to different channels touch different rows and never contend. Second, and more fundamentally, serializing a channel's writes is the requirement, not a side effect. FR-MSG-03 says one channel's messages form a strict order; some mechanism must decide that order; the row lock is that mechanism, colocated with the insert in one transaction so ordering and durability commit atomically. A lock cycle is sub-millisecond, giving hundreds of messages per second per channel — an order of magnitude past human conversation, and past even bot-heavy channels under FR-CHN-07's member cap.

Per-tenant sequences create a genuinely hot row — every send in an environment serializes on one counter — converting a non-problem into a real bottleneck for the benefit of a simpler resume cursor (a scalar instead of a per-channel map). Bad trade: the cursor map is bounded by channel membership and FR-RTM-04's truncation already handles the long tail.

Per-channel SEQUENCE objects: unbounded catalog objects (one per channel, forever), and sequences are non-transactional — a rolled-back send burns a value, producing gaps that are unordered with respect to commits, which poisons the client's "gap ⇒ refetch" heuristic. Our gaps (from rare rollbacks) are safe precisely because sequence assignment commits with the row.

Time-based IDs are coordination-free and globally unique but only approximately ordered (clock skew across instances) and never dense. FR-MSG-03 requires exact order; the SDK's gap detection requires knowing whether a gap means "missed message" — Snowflake gaps are meaningless. Wrong tool.

Decision

Option 1. SELECT … FOR UPDATE on the channel row (or the equivalent atomic UPDATE … RETURNING), sequence assigned and message inserted in one transaction.

Consequences

Positive: exact ordering with zero additional infrastructure; dedup and resume become integer comparisons; gaps are rare and harmless. Negative: a pathological single-channel firehose serializes (accepted: it must, per the requirement); hot-channel throughput is bounded by lock-cycle time (measured in the hundreds/s, far above need).

Revisit when

A legitimate use case needs >500 msg/s in one channel (e.g. IoT telemetry misusing chat) — at which point the answer is "that is not a chat channel," a product boundary, not an architectural fix.

ADR-04 — Single writer: only the API service touches PostgreSQL

Problem

The gateway could plausibly write messages directly (it receives the frames) and read backfill directly. Two services touching the database is normal microservices practice — why forbid it?

Options

1. API service is sole DB client; gateway calls internal HTTP for writes and backfill.
2. Both services share a data-access library against the same DB.
3. Gateway owns messages, API owns the rest (database-per-service orthodoxy).

Analysis

Count the invariants that live at the data layer: sequence assignment under lock (ADR-03), idempotency via partial unique index (DR-03), tenant scoping on every query (D4, the Sev-0 property), tombstone semantics (FR-MSG-08), outbox atomicity (ADR-06). Every one is easy to enforce in one codebase behind one repository layer and notoriously easy to violate in the second codebase during a refactor six months later. The shared-library option (2) is the seductive one — "the code is shared, so the invariants are shared" — but libraries version-skew across services, and the cross-tenant test suite (NFR-SEC-09) would have to run against two deployment surfaces forever. Isolation is a correctness property; correctness properties want one enforcement point.

Option 3, orthodox database-per-service, dissolves on contact with the domain: messages, channels, memberships, and users are one transactional fabric (a send checks membership, locks the channel, inserts the message — one transaction). Splitting ownership converts local transactions into sagas for zero scaling benefit, since these entities scale together (§4.2's table).

The cost of option 1 is one intra-cluster HTTP hop (~1–2 ms) on send and on backfill. Against NFR-PRF-01's 250 ms p95 end-to-end budget, that is noise. The subtler cost is that backfill load lands on the API service; ADR-04 therefore names its own escape hatch — backfill is a read, reads don't threaten invariants, so a read-replica path for the gateway is a safe future carve-out (and S4's storm analysis leans on exactly this).

Decision

Option 1, with the repository layer's constructors requiring an environment_id and raw connection access lint-forbidden outside it.

Consequences

Positive: every data invariant has one home; the cross-tenant suite attacks one surface; schema migrations coordinate with one deployer. Negative: API service is on the gateway's critical path (mitigated: it is stateless and horizontally scaled); +1 hop latency (measured, accepted).

Revisit when

Backfill reads exceed ~30% of API-service load in production — then open the read-replica path, not the write path.

ADR-05 — Sends travel through the WebSocket, writes through the API

Problem

Clients hold two channels to Relay: the WebSocket and REST. Which carries message sends? The answer shapes the SDK's offline story and the server's write topology.

Options

1. WS frame → gateway → internal HTTP → API service (chosen).
2. REST-only sends; the socket is receive-only.
3. Gateway writes to Postgres directly on WS sends.

Analysis

Option 3 is already dead (ADR-04). The live question is 1 vs 2, and it is decided by Tuan's journey, not by server elegance. The SDK queues messages offline and flushes them in order on reconnection (FR-SDK-07). If sends are REST, the flush interleaves two transports: the socket is resuming (backfill streaming in) while HTTP requests race it carrying queued sends — two independently-retrying, independently-ordered paths that must agree on the final order. Keeping sends on the socket makes the flush a sequenced stream on one connection: reconnect, resume, replay queue, in that order, on that pipe. Mobile radio economics agree — one warm connection beats per-send HTTP handshakes for battery and latency (NFR-PRF-01's p50 < 100 ms is comfortable over an established socket, tight if a TLS handshake is in the loop).

REST sends still exist — FR-MSG-13's backend-originated sends use them — but they route to the identical API-service code path. One write path, two entrances: the idempotency, sequencing, and outbox logic cannot diverge between transports because it is the same function.

Decision

Option 1. The frame protocol carries message.send; the gateway is a forwarding client of the API service.

Consequences

Positive: single-pipe ordering for the offline flush; one write path; gateway stays stateless with respect to persistence. Negative: gateway must implement request/response correlation over the internal hop (ack routing); a gateway crash mid-forward leaves the client unacked — resolved by the client's retry with the same idempotency key, which is the designed path (FR-MSG-04), not an edge case.

Revisit when

Never independently — this decision is downstream of ADR-04 and moves only if it does.

ADR-06 — Transactional outbox between Postgres and JetStream

Problem

Every state change must produce an event (webhooks, analytics, live dashboard) — and metering (FR-ANL-06, 0.1% accuracy) means events cannot be silently lost. But Postgres and JetStream cannot commit atomically. This is the textbook dual-write problem; the question is which textbook answer fits.

Options

1. Transactional outbox — event row commits with the state change; a relay drains to JetStream.
2. Publish after commit — write DB, then publish, hope.
3. Publish before commit — publish, then write DB.
4. CDC (Debezium) — tail the WAL, derive events.

Analysis

Publish-after-commit fails silently in the gap: crash between commit and publish, and the event never existed — a message was sent and no webhook fired, no metering row landed. The failure is invisible: nothing errors, the books are just wrong, and FR-ANL-06's reconciliation would page someone weekly with unexplainable drift. Publish-before-commit inverts the failure into phantom events (webhook for a message that rolled back) — worse, because it lies outward to customers.

CDC is the industrial answer and it is architecturally correct — the WAL is already a transactional event log. It is rejected on D8 grounds alone: Debezium means running Kafka Connect (or equivalent), schema-to-event mapping configuration, and WAL-format coupling — an operational subsystem larger than the problem. The outbox is ~50 lines: INSERT INTO outbox in the transaction, a relay loop doing SELECT … FOR UPDATE SKIP LOCKED → batch-publish → mark published. SKIP LOCKED makes the relay horizontally scalable (competing relays skip each other's batches) — R4's mitigation is built into the query.

The outbox has a second, quieter payoff: it is the abstraction seam that makes ADR-02 reversible. Every event originates in a Postgres table with a subject and payload; "which broker" is a relay configuration detail. And the failure matrix's JetStream row depends on it — broker down, events accumulate in Postgres, relay drains on recovery.

At-least-once is inherent (crash between publish and mark → republish), which is why consumer idempotency is a system-wide discipline (dedupe on event id; the consumer template bakes it in per R5) rather than a per-consumer choice.

Decision

Option 1, with the relay as a small loop inside the API service initially (promotable to its own deployment if outbox depth alarms fire).

Consequences

Positive: no lost events, no phantom events, broker-swappable, buffer-on-broker-outage for free. Negative: at-least-once forever (embraced, not mitigated); relay adds seconds-scale latency to event availability (within FR-ANL-04's 60 s bound with two orders of margin); outbox table needs pruning (delete published rows older than N days — trivial).

Revisit when

Event volume makes polling wasteful (~50 k events/s) — then reconsider CDC with the team size that volume implies.

ADR-07 — Fan-out fabric: Redis pub/sub, at-most-once, by design

Problem

A message committed in Postgres must reach every connected member's gateway instance. This fabric runs at the system's highest frequency. How durable must it be?

Options

1. Redis pub/sub — fire-and-forget, at-most-once.
2. JetStream for live fan-out — durable.
3. Core NATS pub/sub — fire-and-forget on a broker the stack already runs.
4. Gateway-to-gateway mesh — direct peer distribution.

Analysis

The instinct says the fan-out fabric must be reliable — it carries the product. The architecture says otherwise, and the reasoning is the intellectual core of the design: durability already lives in Postgres, and ordering already lives in sequence numbers (ADR-03). A client that misses a pub/sub frame sees the next frame's sequence, detects the gap, and refetches — or, if fully disconnected, resumes by cursor on reconnect (FR-RTM-03). The recovery machinery must exist anyway for network failures on the last hop (Tuan's tunnel — no broker fixes his radio). Given that machinery, a lost pub/sub frame is indistinguishable from a lost WiFi packet, and both heal identically.

Amended 2026-08-30 (chapter 3.20). The sentence above is true of every payload this fabric carried when it was written, and false of one added since. A membership revocation has no sequence and no cursor: a client cannot detect a gap in it, and there is nothing to refetch. A dropped one is not a delayed frame, it is a client that keeps receiving a channel it was removed from, indefinitely.

The clause "given that machinery" is the load-bearing part, and it names machinery a revocation does not have. ADR-20 supplies the missing half — a periodic re-read of the connection's own memberships, which is that payload's cursor — and the fabric stays lossy for the reason this record gives. What does not survive is the general claim. Anything new put on this fabric has to answer the question this paragraph answers for messages: what recovers it? "The fabric is allowed to lose it" is only half an argument, and the other half is per payload.

Once the fabric is permitted to be lossy, it should be — Redis pub/sub is single-digit milliseconds, requires zero consumer state, and one SUBSCRIBE chan:{id} per channel per gateway instance. JetStream for live fan-out means durable consumer state per channel-per-gateway for deliveries that are ephemeral by nature — heavy machinery to avoid a problem the cursor already solves, plus higher per-message latency on the hottest path. The mesh option reinvents a message broker with O(n²) connections and a discovery protocol; it is what you build when you don't have Redis, and we have Redis.

Core NATS is the honest competitor, and it deserves a better answer than the other two get. It is at-most-once, subject-based with wildcards, comparably fast, and already deployed — ADR-02 put JetStream in the stack for the durable spine, and core subjects come along with it. The case against is not mechanical. It is that Redis does not leave: ADR-10 keeps presence as Redis keys with TTLs, and rate-limit buckets follow. So fan-out on NATS gives the gateway two broker clients where it had one, deletes nothing from the deployment, and splits the ephemeral concerns of a single service across two systems. Choosing Redis keeps a clean mapping — gateway to Redis, api and workers to NATS — where each broker's guarantee matches its cargo.

That mapping has an exception, and it is older than the chapter that made it obvious. Chapter 3.8 gave the api its own Redis client for rate-limit counters, so "api and workers to NATS" stopped being exactly true then; chapter 3.18 gave the api a second one, for the fan-out, on the hottest path there is. The api now holds two broker clients — which is the cost this analysis rejected core NATS for imposing on the gateway, relocated rather than avoided.

The Decision below is untouched by that. "Publish once per message to chan:{channel_id}" still holds: a message enters through exactly one door, and whichever service accepted it publishes once. The Revisit when clauses are untouched too — they are about gap-refetch rate and subscription scale, and neither is what changed. What changed is that the selection argument's tidiest line is no longer literally true, and a reader comparing it against 05-sad.md's component diagram deserves to be told so rather than left to reconcile them. The genuinely interesting alternative is the one nobody proposed: move presence to NATS KV and remove Redis entirely. That is a coherent architecture and a much larger decision; it belongs to a deployment review, not to the fan-out fabric's ADR.

AND THE GATEWAY HALF WENT IN CHAPTER 4.5, WHICH IS THE HALF THE MAPPING IS NAMED FOR. FR-ANL-01 asks for an analytical event on every connection open and close, and the only producer that can see one is the gateway. It now holds a NATS client — five dependencies to six, none of them a broker before this.

So the sentence two paragraphs up, "fan-out on NATS gives the gateway two broker clients where it had one", describes a gateway that no longer exists: it has two. NATS fan-out would now add none and remove none. The cost this analysis rejected core NATS for has gone to zero on the side it was measured, and the paragraph above already recorded it arriving on the other side as "relocated rather than avoided". Both halves of the clean mapping are spent, and it now describes no service in this platform.

What survives is the half that never depended on an arithmetic: Redis does not leave. ADR-10 keeps presence as Redis keys with TTLs and rate-limit buckets follow, so a gateway without a Redis client is not a thing this design can produce — and a NATS-only proposal is still the larger decision v1.1 named, because it would have to move presence to NATS KV and delete a store from the deployment. The refusal stands and its price does not.

The Decision below is untouched by that, and the Revisit when clauses are untouched too — the same two sentences the amendment above needed, for the same reason. What changed is the selection argument, twice now, from both directions. A reader who has followed this record from chapter 3.8 has watched a tidy line become a historical note in four moves, and the honest version of that is to say so rather than to keep the line.

The Redis-down case (failure matrix): fan-out pauses, gateways detect and cycle clients, clients resume by cursor against Postgres. Degraded latency, zero loss — the design's claim, "Redis holds nothing that is a source of truth," is load-bearing here and audited in §6.3.

Decision

Option 1. Publish once per message to chan:{channel_id}; every gateway instance holding a member subscribes.

Consequences

Positive: minimal latency on the hottest path; zero broker state; the failure story is the reconnect story, already built and tested. Negative: SDK must implement gap detection (bounded: a sequence comparison and a refetch); brief Redis outages surface as client reconnect cycles rather than invisible buffering.

Revisit when

Gap-refetch rate in production suggests pub/sub loss is frequent rather than exceptional (would indicate Redis saturation — scale Redis, not the design); or channel-count × gateway-count makes subscription management dominate (see R3's wildcard-subject sketch).

ADR-08 — ClickHouse single-node in v1, schema designed for cluster

Problem

The analytical store needs 10 k inserts/s (NFR-SCL-05) and 2 s p95 tenant-scoped 90-day queries (NFR-PRF-06). Does it need to be highly available on day one?

Options

1. Single node + backups, cluster-shaped schema.
2. Replicated cluster (3+ nodes, ZooKeeper/Keeper) from day one.
3. ClickHouse Cloud.
4. No ClickHouse — Postgres for analytics too.

Analysis

Option 4 first, because it is the question every reviewer asks: CON-01 forbids it, and CON-01 exists because the workloads are irreconcilable, not stylistically different. Metering scans ("count messages per tenant per day for 90 days") are columnar, compression-friendly, append-only workloads; running them against the row store means either table scans competing with live message inserts or a forest of pre-aggregation tables maintained by triggers — the operational database degrading in service of the meter. The two-store split is the architecture's founding decision (vision §7).

Between 1, 2, and 3, the deciding fact comes from the failure matrix: ClickHouse unavailability is survivable by design. The ingester pauses, JetStream absorbs 24 h (NFR-REL-08), dashboards go stale, messaging feels nothing (FR-ANL-03). HA for a component whose outage costs "stale dashboards for the outage duration" buys almost nothing, and a replicated cluster brings Keeper coordination, replication queues, and ReplicatedMergeTree operational lore — the single largest step-up in operational complexity of any option in this document, paid to protect against a tolerable failure. ClickHouse Cloud removes the ops but adds a cloud dependency against NFR-MNT-06 and a bill disproportionate to v1 data volumes.

Single-node throughput: MergeTree ingests hundreds of thousands of rows/s on modest hardware when batched (DR-11 mandates batching); 10 k/s is not a sizing exercise. Query targets hold because DR-07's ORDER BY (environment_id, ts) makes every dashboard query a tenant-contiguous range read, and DR-10's materialized rollups keep metering off raw tables entirely.

The schema hedges the future deliberately: monthly partitions, tenant-first ordering keys, and rollup views are exactly the shapes that survive a later move to ReplicatedMergeTree + Distributed — that migration is data movement, not schema redesign.

Decision

Option 1: one node, nightly backups to object storage, cluster-shaped schema.

Consequences

Positive: near-zero analytics ops; full NFR headroom; contained future migration. Negative: node loss means dashboard gap until restore + replay from the stream (bounded by 24 h retention + backup age — worst case loses analytical history between backup and outage, never operational data); no query failover.

Revisit when

The stream's 24 h buffer stops covering realistic restore time; or metering data becomes contractually irreplaceable (SLA credits computed from it) — the trigger is data criticality, not query volume.

ADR-09 — Dashboard live view over SSE, not WebSocket

Problem

FR-DSH-02 — the live event stream that converts Mai's first-message anxiety into confidence (journey Stage 4's highest-leverage feature) — needs server → browser push under 2 s. Which transport?

Options

1. SSE via a thin relay service off JetStream.
2. Reuse the production WebSocket gateway.
3. Polling.

Analysis

The traffic is strictly one-directional (server → dashboard; the dashboard issues commands over normal REST), which is SSE's exact shape: plain HTTP, native EventSource reconnection with Last-Event-ID, no frame protocol to design, no upgrade negotiation, friendly to proxies and HTTP/2 multiplexing.

Option 2 is the tempting consolidation — "we already have a WebSocket gateway" — and it is rejected for a reason worth stating precisely: it would couple observer load to the observed system. The gateway is the component whose degradation the dashboard exists to reveal; putting the dashboard on the gateway means an incident's diagnostic tool degrades with the incident. It would also entangle two authentication models (end-user JWTs vs. dashboard OAuth sessions) in the security-critical connection path, and pollute the gateway's scaling signal (connection count, S1) with sessions that behave nothing like end users. The SSE service is ~200 lines: ephemeral JetStream consumer filtered to events.*.*.{env}, forwarded as SSE frames.

Polling at the 2 s bound means a dashboard tab hammering the API at 0.5 Hz per widget — crude, chatty, and it turns "live" into "jittery."

Decision

Option 1. Separate small service, ephemeral consumers, dashboard-session auth.

Consequences

Positive: gateway isolation preserved; near-zero protocol surface; browser-native reconnection. Negative: one more (tiny) deployment; SSE's one-directional limit means any future dashboard interactivity over the stream would force a rethink (none is planned — commands are REST).

Revisit when

The dashboard needs bidirectional streaming (e.g. an interactive debugging console) — then a dashboard-dedicated WebSocket, still never the production gateway.

ADR-10 — Presence in Redis with TTL, no dedicated service

Problem

Presence (FR-RTM-06/07) is conceptually a service — "who is online" — and microservice instinct says to build one. What is presence, actually, and where does it want to live?

Options

1. Redis keys with TTL, transitions published on channel subjects (chosen).
2. Dedicated presence service with its own store and API.
3. Presence in Postgres (a last_seen column).

Analysis

Presence is derived connection state with a grace period — nothing more. The gateway already knows connections (it terminates them) and already maintains the Redis connection registry (D2). Presence is one more key with a 30 s TTL refreshed by the same heartbeat: the data's producer, natural store, and consumers (gateways, for fan-out to co-members per FR-RTM-07) are all already in place. A dedicated service would subscribe to the gateway's connection events to maintain a copy of state the gateway already has — a service whose entire job is being a cache invalidation problem.

The TTL design also answers the hardest presence question — crashed clients — for free: a gateway that dies stops refreshing; keys expire; users appear offline within the grace period, no reaper process, no tombstone protocol. And durability analysis says presence wants none: after a Redis flush, every heartbeat repopulates within 30 s. Green dots are self-healing; engineering durability for them is engineering waste. Postgres presence (option 3) is the opposite error — heartbeat-frequency writes hammering the operational store for data with a 30-second half-life.

The honest cost is fan-out volume, not storage: each transition publishes to every channel the user belongs to. SRS Open Question 3 (opt-in presence per channel) stays open, with ADR-10 carrying the measurable trigger.

Decision

Option 1. presence:{env}:{user} with 30 s TTL; transitions published on the member channels' subjects.

Consequences

Positive: zero new components; crash-correct by expiry; presence load scales with the gateway that generates it. Negative: presence fan-out rides the same pub/sub fabric as messages (acceptable while it is a minority of volume — the trigger below guards it); 30 s offline-detection latency is the grace period working as specified.

Revisit when

Presence exceeds ~30% of gateway publish volume in load tests — then presence subjects get their own fabric or channels opt in, per Open Question 3.

Superseded in part by ADR-19 (chapter 3.19). Two sentences above are now out of date and are left standing because an accepted record is not edited. The subject clause — "transitions published on the member channels' subjects" — is replaced by presence:{channel_id}. And the Analysis says Open Question 3 "stays open": it does not. SRS v1.4 closed it as not opt-in per channel, confirming rather than reversing the reasoning above. Everything else in this record stands — Redis, TTL, no dedicated service, the same audience — and the ~30% trigger below is still undischarged.

ADR-11 — Custom emoji as shortcodes in plain text, with a read-time resolution map

Problem

Custom emoji (FR-EMJ) must appear inside messages — the one data structure the entire architecture treats as sacred. Every invariant bled for elsewhere — byte-exact storage, tombstones, edit history, idempotent retries, export — is defined over text as an opaque string. How do emoji enter without breaking that?

Options

1. :shortcode: in plain text + read-time resolution map (chosen).
2. Structured message entities — text as an AST of runs and emoji nodes (Slack-style blocks).
3. Unicode Private Use Area code points mapped to custom emoji.
4. Server-side rewrite — replace shortcodes with image URLs/markup at write time.

Analysis

Option 2 is the "proper" design in a chat application and the wrong one in chat infrastructure. The moment text becomes a tree, every sacred invariant must be re-derived over trees: what does an edit diff of an AST mean (FR-MSG-07 stores prior_text — prior tree?); what does a tombstone preserve; what does byte-exact (FR-EMJ-01) even mean; what does the customer's export (FR-MOD-05) contain; how does Priya's support tool render a five-year-old entity schema version? Each question is answerable; the sum of answers is a permanent complexity tax on the platform's core object, paid to benefit one read-side feature. Option 1 inverts the dependency: the write path stays emoji-ignorant — send, edit, tombstone, idempotency, export are all still string operations — and emoji become a pure read-side concern: scan the returned page's text for :[a-z0-9_]{2,64}:, attach a sidecar map {shortcode → image_url | unresolved}.

This inversion is what makes FR-EMJ-10 (deleting a pack never mutates history) free by construction rather than carefully implemented: stored messages never contained the image, only the name. Priya's dispute record (journey 3, stage 3) renders as written regardless of pack lifecycle — the resolution just degrades to literal text.

Option 3 (PUA code points) achieves compactness at the price of honesty: text becomes uninspectable (a moderation export full of U+E0xx is meaningless), collides across environments, and breaks the "text is what the user wrote" property that both audit and FR-EMJ-01 rest on. Option 4 (write-time rewrite) mutates user content — the original shortcode is unrecoverable, edits diff against rewritten text, and a pack rename would strand old markup. Both are rejected on the same axiom: stored text is testimony; the platform annotates it but never rewrites it.

Accepted wart: literal :text: that predates a matching shortcode will resolve once the shortcode exists. The grammar makes accidental matches rare, and the client fallback makes them benign — a mis-rendered emoji, never lost text.

Decision

Option 1, resolution assembled server-side on message-bearing responses (see ADR-12 for how that stays cheap).

Consequences

Positive: zero change to write-path semantics; FR-EMJ-10 free; REST consumers get rendering data without SDK logic; export/audit unaffected. Negative: clients must render from map + text (the SDK ships the segmenter, FR-EMJ-13); the accepted-wart above.

Revisit when

Rich message structure (mentions with identity, formatted blocks) becomes a genuine platform requirement — entities would then be designed once, holistically, not retrofitted for emoji alone.

ADR-12 — Environment-scoped resolution map, cached whole, version-invalidated

Problem

ADR-11 puts resolution-map assembly on history reads — the hottest read path, with a 100 ms p95 budget (NFR-PRF-03). Naïve implementation (join shortcodes against the emoji table per page) taxes every read for a feature most messages don't use. How does resolution become nearly free?

Options

1. Whole-environment map, cached under a version-stamped key (chosen).
2. Per-request lookup of just the shortcodes present.
3. Resolve client-side only (SDK fetches pack data; server returns raw text).

Analysis

The enabling observation is a size bound that exists by construction: FR-EMJ caps packs at 200 emoji and shortcodes at 64 chars; even a pathological 100-pack environment is a <1 MB map. Small enough to load in one Redis round trip and — the real win — to memoize in-process per API instance. Steady-state resolution is then a hash lookup against local memory while scanning the page's text: nanoseconds, no I/O, the 100 ms budget untouched.

The classical objection to whole-map caching is invalidation, and the version-stamp pattern dissolves it: emoji_version on the pack row (DR-13) increments on any mutation; the cache key is emoji:{env}:{version}. A mutation doesn't invalidate anything — it makes the old key unreachable, exactly the CDN cache-busting move. No purge fan-out to API instances, no TTL-staleness window on correctness (an instance holding version N serves slightly stale emoji for one in-process-cache lifetime after a pack edit — cosmetically stale, never wrong-tenant, and pack edits are rare against constant reads).

Option 2 keeps a Postgres or Redis round trip on every message-bearing response and its cost scales with page content — the hot path taxed forever to avoid a bounded cache. Option 3 pushes resolution to clients: REST-only consumers (customer backends fetching history for their own processing, Priya's tooling) would receive unresolvable text, and every client re-implements resolution — duplicated logic to save the server a hash lookup it can do for free.

Decision

Option 1. Redis holds the versioned map (24 h TTL as garbage collection, not correctness); API instances memoize per version; pack mutations bump emoji_version transactionally with the mutation.

Consequences

Positive: resolution at memory speed on the hot path; invalidation is unreachable-key simple; REST and SDK consumers get identical resolution. Negative: one cold-start rebuild per environment per mutation (rare); a bounded staleness window after pack edits (cosmetic); the map's size bound must be defended — raising the 200-emoji pack cap now has an architectural stakeholder.

Revisit when

A legitimate customer need breaks the size bound (packs ×10, or emoji-per-pack ×10) — then per-pack cache sharding, resolved lazily by the shortcodes actually present.

ADR-13 — Media bytes never transit Relay compute

Problem

Hosted media reverses a founding exclusion (SRS Appendix B). The exclusion's reasons were real: storage cost, bandwidth, virus scanning, CDN, and erasure semantics. Reversing it responsibly means the design must answer each reason, not outvote them. The central question: does Relay proxy uploads and downloads, or broker access to storage it never touches?

Options

1. Presigned direct-to-storage — API issues upload/download URLs; bytes flow client ↔ object storage.
2. Proxied uploads/downloads through the API (or a dedicated media API) service.
3. Public-read bucket with unguessable keys; uploads presigned.

Analysis

Option 2 puts every media byte on Relay's network path: a 100 MB video upload occupies a Node request slot for its duration, upload bandwidth becomes an API-service scaling dimension, and download traffic — the multiplied side, since one upload is read by every channel member — turns Relay into a CDN it would then need to actually build. This is precisely the cost basket the original exclusion priced as disproportionate; choosing it voluntarily would make the reversal indefensible. Option 1 dissolves the basket: object storage's durability, bandwidth, multipart handling, and lifecycle rules are bought, not rebuilt, and Relay's involvement per media operation is one metadata row plus one HMAC — NFR-PRF-08's <100 ms is easy because there are no bytes.

Option 3 fails on authorisation shape. FR-MED-08 requires media access to follow channel membership — a revocable, changing relation. Public-read-with-obscure-keys is irrevocable-by-construction: a key shared once is shared forever, and membership removal (FR-RTM-10's analogue for bytes) has no mechanism. Time-limited signed URLs are the compromise position: authorisation is evaluated at mint time against membership, and the leak window is bounded (1 h) rather than infinite. DR-16 (never persist signed URLs) exists to keep that window real.

Tenant-prefixed object keys (DR-15) do quiet work: compliance erasure and tenant export become prefix operations against storage, aligning the bytes' lifecycle with the rows'.

Decision

Option 1. MinIO in local dev; any S3-compatible store in production (ASM-06 added to the SRS to make the dependency explicit).

Consequences

Positive: near-zero media load on Relay compute; provider-grade durability and bandwidth; the storage bill is the scaling ceiling (S8), not throughput. Negative: coarser auth granularity (bounded, above); upload completion observed asynchronously; a hard dependency on S3 semantics (acceptable — S3-compatibility is the closest thing object storage has to POSIX).

Revisit when

A customer segment requires media behind their own network boundary (BYO-bucket federation) — an extension of this design, not a reversal.

ADR-14 — The scan pipeline gates bytes, never messages

Problem

Hosting user uploads without scanning is a liability no metering revenue covers; scanning takes seconds. Where do those seconds go — into the send path, the receive path, or neither?

Options

1. Async scan; attach-while-pending; media.updated fan-out on transition (chosen).
2. Scan before send — media must be ready before a message may reference it.
3. Scan on first download.

Analysis

Option 2 reads as the conservative choice and is actually the coupling the architecture most forbids: it inserts a CPU-bound, seconds-long, third-party-engine-dependent stage into the message send path. NFR-PRF-01's latency budget dies; worse, scanner outage becomes message outage for any media-bearing send — a new fate-sharing edge violating D5's spirit. And the UX it produces (spinner between "send" and "sent") contradicts the expectation consumer messengers have set: the photo appears immediately, sharpens later.

Option 1 splits the concern cleanly along a line the system already knows how to draw: the message (text + a media reference) flows through the ordinary send path at ordinary speed; the bytes are gated — no signed URL is minted for pending or rejected media. Recipients render a placeholder from the attachment's state; the worker's pending → ready transition emits media.updated on the referencing channels through the same outbox → fan-out machinery every other state change uses. No new delivery concepts — the state machine even rhymes with the SDK's existing message states (sending/sent/failed ↔ pending/ready/rejected), which halves the teaching burden.

Rejection as a first-class, rendered terminal state (FR-MED-09) is Priya's requirement: a dispute reconstruction must distinguish "an upload was attempted and rejected" from "a message was deleted" — a broken-image glyph destroys evidence that an explicit marker preserves.

Option 3 moves the latency to the recipient's tap (the worst moment), re-scans per cache miss, and means unscanned bytes rest in storage indefinitely — the liability deferred, not addressed.

Decision

Option 1. ClamAV as a worker sidecar; probe and thumbnail in the same pipeline pass; transitions via internal API per ADR-04's single-writer rule.

Consequences

Positive: send latency untouched by media; scanner capacity degrades time-to-ready only; one delivery machinery serves messages and media transitions alike. Negative: recipients may see placeholders that resolve to rejection (accepted — the expectation exists); pending objects need a reaper (FR-MED-10's 24 h job); the worker is the one genuinely CPU-bound service in an I/O-bound fleet (its own HPA signal, and ADR-01's worker-thread posture from day one).

Revisit when

Scan-evasion incidents or legal counsel demand synchronous review for specific tenants — then a per-environment strict_media flag flips option 2 on for those who need it, priced accordingly.

ADR-15 — NestJS for the API service

Problem

The API service began as a walking skeleton: one health route over node:http. By the end of Phase 2 it owns the write path; through Phases 3–4 it accumulates tenancy, key management, channels, membership, history, moderation, emoji packs, and the dashboard's read APIs — dozens of endpoints, every one needing the same things: input validation at the boundary, the EIR-API-04 error envelope, auth context, request-scoped tenancy, and a documented contract (EIR-API-07 requires a complete OpenAPI 3.1 spec). The question is whether those cross-cutting needs are met by conventions we hand-roll and document, or by a framework whose conventions arrive pre-documented.

Options

1. Keep hand-rolled node:http — extend the skeleton's serve() helper with routing, validation plumbing, and a hand-maintained OpenAPI document.
2. Bare Fastify/Express — a router with middleware, everything else still hand-rolled.
3. NestJS — modules, dependency injection, guards/pipes/interceptors, first-class validation, generated OpenAPI.

Analysis

Option 1 is honest at skeleton scale and quadratic after it: every endpoint hand-wires validation, error shaping, auth, and docs, and every convention lives only in this repository's heads and READMEs. D8 cuts against that — one engineer cannot afford to be the framework's sole author and its sole documentation. Option 2 buys routing but leaves the expensive parts (validation discipline, OpenAPI, DI for testability) as bespoke work. Option 3 buys exactly the cross-cutting layer this surface needs, at two real costs: a decorator-based programming model (which demands a build/transform step and ends any run-the-TS-directly purity) and a materially larger dependency tree.

The costs concentrate where request shape matters and mechanisms don't. That describes the API service's REST surface precisely — and describes the gateway not at all. The gateway's entire job is socket mechanics: the connection registry, the resume buffer, the subscribe-before-backfill ordering. Abstraction between that code and the socket is surface without benefit, so the framework stops at the gateway's door.

Isolation deserves its own sentence, because frameworks tempt exactly the failure D4 forbids: putting tenancy checks in guards — handler-land. The repository layer sits beneath the framework and its constructor still requires an environment_id; the guard authenticates and resolves the tenant, the data layer enforces it. The framework changes who calls the repository, never what the repository demands.

Decision

NestJS for the API service only. The gateway remains frameworkless; workers remain plain consumers. Validation, error envelopes, and OpenAPI generation ride the framework; isolation stays in the repository layer.

Consequences

A build step joins the API service's toolchain (decorators are not erasable syntax). Endpoint chapters spend their words on semantics instead of plumbing. EIR-API-07 becomes generated output instead of a standing documentation debt. The skeleton's shared serve() plumbing survives on the gateway side.

Revisit when

Framework overhead appears on a latency-budget path (NFR-PRF-01), or a core-loop mechanism has to fight the abstraction the way it would on the gateway — Fastify is the named fallback, and the repository layer's independence is what makes that swap survivable.

ADR-16 — Drizzle for the API service's data layer

Problem

The repository layer is written against pg with hand-typed row interfaces. It is correct, visible, and increasingly verbose: every query hand-declares its result shape, and NFR-MNT-02's coverage expectations want the compiler carrying more of that load. The question is which data-layer tool adds type safety without hiding the SQL the core loop is made of — the row lock (ADR-03), the idempotency conflict (DR-03), the partial unique index, the CHECK constraints.

Options

1. Stay on raw pg — maximum visibility, hand-maintained types.
2. Prisma — schema-first ORM, generated client.
3. TypeORM — decorator entities, the historical NestJS pairing.
4. kysely — a typed SQL query builder with zero schema ownership.
5. Drizzle — SQL-shaped typed queries, schema defined in TS, SQL migrations generated.

Analysis

The test is the five mechanisms. Prisma fails it structurally: DR-03's partial unique index cannot be expressed in its schema at all, and FOR UPDATE exists only through raw escape hatches — the abstraction is weakest exactly where correctness lives, so its costs would be paid everywhere and its benefits suspended on every hot path. TypeORM passes the mechanism test but fails the confidence test: result typing is loose where NFR-MNT-02 wants guarantees, and its API's two personalities make conventions harder, not easier.

kysely and Drizzle both pass. kysely owns nothing — the hand-written migrations stay the only schema artifact — which is the purest fit for "the SQL is the source." Drizzle owns a TS schema definition, and pays that cost back with schema-level constraint definitions (the partial index and CHECKs live next to the tables that own them), relational query ergonomics for the read-heavy Phase 3–4 surface, and .for("update") / .onConflictDoNothing() reading almost exactly like the SQL they emit.

The drift risk is the TS schema diverging from §6.1's SQL. The mitigation is mechanical: drizzle-kit generates SQL migration files, those files are reviewed and applied as SQL, and generated DDL is diffed against §6.1's definitions. The applied SQL remains what runs.

Decision

Drizzle inside the repository layer. The layer's constructor discipline (D4) is untouched; Drizzle is its query engine, never a client that escapes it. Raw SQL islands remain permitted inside the layer where the builder falls short. Migrations remain versioned, forward-only SQL files.

Consequences

Queries gain end-to-end types without losing their SQL shape. The schema exists twice — once as §6.1's SQL truth, once as TS definitions — with drift checked, not assumed away. The isolation suite and the lint ban on raw driver imports carry over unchanged.

Revisit when

The TS schema and applied SQL drift in practice, or new queries routinely bypass the builder for raw SQL — either signals that kysely's schema-free model (the named fallback) should take over.

ADR-17 — Turborepo for build orchestration

Problem

The workspace's three-command gate (lint, typecheck, test) has run everything, everywhere, on every invocation since chapter 1.1 — the right call when every task was fast and no task depended on another. ADR-15 breaks both properties at once: NestJS's decorators give the API service a real compile step, and a compiled @relay/protocol must exist before the services that import it can build. Phases 2–4 add four more services and several packages. The question is what runs the tasks — not what installs the packages; pnpm keeps that job — so that gate time scales with the size of a change rather than the size of the workspace.

Options

1. Stay on plain pnpm -r scripts — recursive, uncached, orderless.
2. Turborepo — a task graph with content-hash caching over existing package scripts.
3. Nx — the heavyweight: plugins, generators, a daemon, distributed execution.
4. Bazel (and kin) — hermetic, polyglot, industrial.

Analysis

Plain pnpm -r fails the new requirements structurally: it has no task ordering beyond package topology per command, and no memory — a one-line docs change re-typechecks six services. It was the correct choice until this ADR's drivers existed, which is exactly why it lasted this long.

Bazel solves problems Relay does not have (polyglot builds, thousand-engineer repos) at a cost D8 cannot pay: its model replaces the npm-script world instead of running it. Nx and Turborepo both fit the actual need. Nx is more capable — and the capability arrives as surface: a daemon, code generators, plugin versions to track. The deciding observation is that every capability the gate needs (task graph, caching, filtering) is the part of Nx that Turborepo is, without the rest.

Turborepo's real cost is trust: a cache hit is a claim that nothing relevant changed, and the claim is only as good as turbo.json's declared inputs and outputs. An undeclared input (an env var, a config file outside the package) produces a stale green — the most dangerous failure a gate can have. The mitigation is the same discipline the fence rules already established: declarations are reviewed like code, and tagged chapter checkpoints run against a clean cache, so a lying cache cannot survive a tag.

Decision

Turborepo as the task runner over the unchanged pnpm workspace. Every turbo run target remains an ordinary package script — turbo orders and caches, packages still own their commands. Remote caching waits for CI (Part 6).

Consequences

The gate's cost now tracks the change, not the workspace; the protocol-before-services build order is declared once instead of implied. One new artifact (turbo.json) joins the reviewed set, and the degradation path stays open: delete it and pnpm -r still runs everything, slowly and correctly.

Revisit when

A stale-cache incident survives the input-declaration discipline, or the task graph starts encoding dependency knowledge that belongs in package.json — either way Nx (the named fallback) or plain recursion takes over.

ADR-18 — Two user populations: platform humans and tenant end users, never merged

Problem

Two kinds of person interact with Relay. A customer's end users — Tuan, the dispatcher — exist inside an environment and never sign in to Relay at all; their identity is whatever external_id the customer already had for them. The people who run a Relay account sign in with GitHub or Google, may own several organisations, and belong to no environment. Should one table hold both?

Options

1. Two tables, one above the tenant boundary and one below.
2. One users table with a nullable environment_id, null meaning "platform human".
3. One table with a view per population, hiding the nullable column behind names.

Analysis

Option 2 is the one that sounds economical and is disqualified by a single column. A platform human belongs to no tenant, so the tenant column has to be nullable — and a nullable tenant column is precisely the shape Principle I exists to prevent, the one FR-TEN-06 rules out in words ("every persisted operational and analytical record shall carry a non-null tenant identifier"). The consequence is not aesthetic. Chapter 2.1 built the repository so that a query without an environment_id cannot be expressed; with a nullable tenant, isolation stops being enforced by construction and becomes a thing reviewers must notice. The most important requirement in the system (FR-TEN-05) would then rest on vigilance, which is exactly the trade the constitution refuses.

Option 3 changes nothing that matters. The column is still nullable; the view only makes it harder to see.

Option 1 costs two tables and an honest admission: "who is this?" is answered differently on either side of the boundary. That is not a wart — it is what the boundary is. Below it, identity is the customer's external_id, scoped to an environment. Above it, identity is a provider account, scoped to nothing, because a person is not a tenant.

The identity key is worth its own note. It is (provider, provider_account_id), never the email: emails change hands, providers may withhold them, and a provider that does release one has usually not verified it. That makes account linking — the same person arriving via GitHub and then Google — a deliberate later feature rather than an accident of matching strings, and chapter 3.1 says so where a reader will meet it.

Decision

Option 1. users stays exactly what Part 2 made it; humans and memberships join the schema above the boundary, and no row ever crosses.

Consequences

Positive: FR-TEN-06 remains true of every operational row without exception, so the repository's scoping stays enforceable by construction; roles (FR-TEN-07) have somewhere to live; and the isolation gauntlet (3.7) has an unambiguous answer to "which population does this identifier belong to?". Negative: two tables to reason about, and any future feature touching both populations — audit trails naming an actor, for instance — must say which one it means. That cost is paid in clarity rather than in correctness.

Revisit when

A product requirement genuinely spans both populations — a customer's end user who is also a Relay account holder, wanting one login. Even then the answer is likely a link table rather than a merge, because the tenant column's nullability is the thing that must not change.

ADR-19 — Presence on its own subject grammar

Problem

ADR-10 decided where presence lives and, in the same sentence, which subject carries it: "transitions published on the member channels' subjects". Nine chapters later the first implementation of presence went to write that sentence into code, and the fabric it names cannot carry a transition. The chan:{channel_id} path is typed to messages at three points, spread across two files: a publish(message: Message) signature and a messageCreatedSchema parse in services/gateway/src/fanout.ts, and the literal message.created send inside session.ts's deliver, a function fenced by ten chapters of tutorial. A presence payload published there today produces fanout.invalid_payload and no frame at all.

So the question is not the one ADR-10 answered. It is: what does presence publish on, given that the answer ADR-10 assumed is unavailable without editing the highest-volume path in the system?

Options

1. A second subject grammar, presence:{channel_id}, in its own module (chosen).
2. An enveloped payload on chan:{channel_id} — a discriminated union, parsed on receipt, dispatched by kind.
3. A pattern subscription (PSUBSCRIBE relay:*:{channel_id}) covering both.

Analysis

Option 2 is the one that honours ADR-10's letter, and it is the expensive one. The parse it adds runs on every message every instance receives, forever, to serve traffic that ADR-10 itself describes as a minority of volume — the cost lands entirely on the path that is not being changed. It is also the option that makes cross-kind mis-delivery a thing tests must defend against: with one subject, "a message must never arrive as a presence frame" is an assertion someone has to remember to write, where with two subjects it is a property of the topology. And there is a deploy window nobody would enjoy: during a rolling restart, an old instance receiving new enveloped payloads emits fanout.invalid_payload for every transition on every channel until it drains.

Option 3 is the one that gets worse with scale. Redis matches every published channel against every registered pattern; a fabric whose per-publish cost grows with the number of subscribers is the wrong shape for the component that exists to fan out.

Option 1's honest price is the subscription count, and this is where a number replaces an argument. A channel now carries two subscriptions instead of one, so a user in twenty channels issues forty SUBSCRIBEs at connect rather than twenty. Measured on the running Redis with CONFIG RESETSTAT and INFO commandstats: two instances across three channels produced cmdstat_subscribe calls=12 — six fan-out, six presence, one per channel per instance, exactly, with the second local member of a channel adding none. ioredis takes a variadic subscribe(...), so the doubling costs no extra round trips.

The placement inside the codebase argues itself once stated. The event spine already keeps its own subjectFor in internal.ts rather than in fanout.ts; each fabric owning its subject grammar is this codebase's existing habit. A new file is also a whole-file fence for the tutorial and leaves two earlier chapters' diff hunks untouched, which a shared file would not.

Decision

Option 1. presence:{channel_id}, generated by subjectForPresence in a new protocol module alongside the fabric payload schema, published and consumed by a new gateway module that owns its own Redis clients. fanout.ts is not edited.

This supersedes ADR-10's subject clause and nothing else. Presence still lives in Redis, still expires rather than being reaped, still has no service of its own, and still reaches exactly the members of the channels the user belongs to.

It also closes SRS Open Question 3 — "should presence be opt-in per channel?" — as no. ADR-10 answered it provisionally and the answer holds, though not for the reason the question anticipated: a per-channel toggle is a column, a defaulting rule, an API surface and a UI, bought to solve a volume problem that has still not been measured. The cheaper half of that remedy is what Option 1 above already delivers.

Consequences

Positive: the message hot path is byte-identical; cross-kind mis-delivery is structurally impossible rather than test-enforced; a rolling deploy of a presence-aware instance beside a presence-blind one produces no errors in either direction, because the blind one is not subscribed to the new subject. Negative: one more subject grammar for a reader to learn, two subscriptions per channel per instance, and a fifth and sixth Redis client in the gateway — a subscriber and a command client, because a connection in subscribe mode cannot run SET.

The uncomfortable consequence is the honest one: this is half of ADR-10's own revisit remedy, taken before ADR-10's trigger fired. ADR-10 said that above ~30% of publish volume, "presence subjects get their own fabric or channels opt in". Presence subjects now have their own fabric. Nothing here measured publish volume at scale — the reason is the typed fan-out, not the threshold — so the trigger is not discharged, and neither is NFR-SCL-01. A decision that arrives at a prepared destination by an unrelated road has not tested the road that was prepared.

Revisit when

Presence fan-out exceeds ~30% of gateway publish volume in load tests — ADR-10's trigger, inherited intact, and now answerable only by the second half of its remedy, channels opting in. Or when the doubled subscription count becomes the binding constraint on connections per instance, which is the measurement NFR-SCL-01 has been owed since the SAD was written.

ADR-20 — Membership on a third subject grammar

Problem

A membership change has to reach two audiences that a channel subject can only serve one of. A removal is fine: the removed user is still a member at the instant it publishes, so member:{channel_id} reaches them and the remaining members in one go. An addition is not, and the reason is topology rather than taste — the instance holding the new member is not subscribed to that channel, because not being in it is precisely what is changing.

Options

One subject per channel. Cheapest, and cannot deliver an addition.

One publish per remaining member's user subject. Correct, and replaces one publish with one per member — a thousand of them at FR-CHN-07's ceiling, for one removal.

Two shapes: the channel's and the principal's. Two publishes for an addition, one for a removal, and the second shape is the first event in this system addressed to someone to tell rather than a thing to hear about.

Analysis

ADR-07 permits a lossy fabric because "durability lives in Postgres, and the client's cursor

• sequence-gap detection recovers anything missed". A revocation has neither. It is not in a stream, a client cannot detect a gap in it, and there is nothing to refetch. A dropped one leaves a client receiving a channel it was removed from, indefinitely — FR-RTM-10's failure, not a delivery delay.

Constitution IV requires a new mechanism to preserve the recovery property, so the periodic re-read exists to be this payload's cursor. Sixty seconds comes from the connection budget: NFR-SCL-01's 10,000 per instance at one re-read each is 167 requests per second, where five seconds would be 2,000.

Decision

Two subject shapes and a sixty-second re-read per connection.

Consequences

Under fabric loss a revocation lands within sixty seconds rather than FR-RTM-10's five, exceeding the clause by 55 seconds. The revocation is guaranteed; what is bounded is how late it can be. That is stated here rather than left for a reader to derive, because the alternative — reading the two numbers as one — turns a backstop into a poll.

Revisit when

The re-read's request rate becomes a constraint on connections per instance, or a clause is written that bounds a post-loss revocation.

ADR-21 — Typing on a fourth subject grammar

Problem

Typing was the one real-time kind that looked like it needed no new grammar: per channel, ephemeral, no membership question. ADR-19 refused chan:{channel_id} for presence because the message path is typed to messages at three points — and if that were the whole cost, typing might have been worth paying it.

Options

Reuse chan:{channel_id}. Widen the types, loosen the parse, edit the highest-volume path in the system.

An enveloped payload on chan:. ADR-19 rejected this for presence: a discriminated-union parse on every message every instance receives, and during a rolling deploy an old instance logs fanout.invalid_payload for every keystroke on every channel.

A fourth grammar. One more subject shape, one more reference-counted subscription per channel, and nothing else changes.

Analysis

The count was wrong, and re-deriving it is what settled the chapter. ADR-19 says three typed points; the grep returns eight lines covering seven — onDelivery, publish(message: Message) and a deliver type in fanout.ts, the messageCreatedSchema parse there, and three separate literal message.created sends in session.ts. The argument is not weakened by the correction; it is stronger, and it had been carried forward by two chapters without anyone re-running it.

The envelope option is worse here than it was for presence, because typing is higher frequency by orders of magnitude — the invalid-payload log during a rolling deploy is per keystroke rather than per transition.

Decision

typing:{channel_id}. One shape, where ADR-20 needed two: a typing signal is only ever interesting to people already in the channel, so there is no audience the channel's subject cannot reach.

Consequences

Four grammars share one Redis and every gateway subscribes to a string, so the topology now needs a test of its own — five subject builders, pairwise distinct for the same id, plus a delivery test proving four kinds arrive under four types. A builder can be distinct while a handler is wired to the wrong one.

And a rule, reached three times from three starting points: a fabric owns its subject grammar, and a kind that cannot share a payload type cannot share a subject.

Revisit when

A fifth kind arrives whose payload the typing fabric could carry unchanged — at which point the question is whether two kinds share one grammar, not whether typing should have moved.

ADR-22 — The typing expiry belongs to the receiving client

Problem

FR-RTM-08 says "Typing indicators shall expire automatically after 5 seconds without renewal and shall not be persisted." Read plainly, that is a server obligation. The server cannot meet it.

Options

A Redis key with a five-second TTL. The obvious reading of the clause, and it fails on its own terms: the gateway would learn an indicator had lapsed and have no frame to say so with. A key that expires silently is a key nobody can act on.

A state field and a typing.stop frame. Complete, and it edits a schema published since chapter 1.3 that twenty chapters of clients parse — to add a message whose loss is unrecoverable.

The client's timer. Five seconds from the last frame per (channel, user), counted where the indicator is drawn.

Analysis

A dropped renewal self-corrects within one interval; a dropped stop frame leaves an indicator showing for ever. That asymmetry decides it. Chapter 3.20 reached the opposite conclusion for membership from the same test — a revocation has no cursor, so it needed a backstop — and the two records are one argument with the inputs reversed. A lost typing frame converges on the truth; a lost revocation converges on a lie.

The clause's second half is met absolutely rather than approximately: nothing is stored anywhere, which is stronger than any TTL would have been.

Decision

No key, no timer, no stored fact. The platform emits and stops emitting; the disappearance is the client's.

Consequences

A customer implementing a client from the published documents alone will not expire an indicator, because FR-RTM-08's plain reading puts the timer on the server. This record is the correction. The clause itself is unchanged — a requirement is the customer's contract, and a chapter does not rewrite one to match its code.

The gateway holds a two-second debounce per connection and channel, so a keystroke is not a publish: 2.5 renewals per expiry window, chosen so one dropped publish does not make an indicator flicker.

Revisit when

An SDK exists in this repository. The timer would then have a home the platform owns, and "the client" would stop meaning "code we do not control".

ADR-23 — Five slot keys, not a sorted set

Problem

FR-RTM-09 says "A user shall be permitted up to 5 concurrent connections." Five is a number about a person, and connections for one person land on whichever gateway instances the load balancer picked — CON-02 forbids sticky routing for correctness. So the count lives outside every instance's memory, in Redis, and the question is what shape it takes there.

§6.3 of the SAD has carried an answer since the first draft, and carried it as a correction: conn:{env}:{user} as a set of instance IDs does not work, because a Redis TTL is per key and not per set member, so one live instance refreshing the key keeps a dead instance's entry alive for ever. The row's prescribed fix is a sorted set scored by heartbeat time, pruned with ZREMRANGEBYSCORE on read.

Options

The sorted set with Lua. ZREMRANGEBYSCORE, ZCARD, ZADD in one script, which is the only version that makes the cap atomic. Constitution VII allows a second language into this repository only through a superseding ADR carrying profiling evidence.

The sorted set with add-then-verify. No Lua: add the member, count, remove yourself if the count is over five. It never over-admits. It also refuses both of two connections arriving when four places are held — each adds, each counts six, each backs out, and the fifth place stays empty with two people told they are at their limit. Safe and wrong.

A single counter. INCR on connect, DECR on close, and it cannot expire per member: a crashed instance's increment is permanent. That is §6.3's original defect wearing a different shape rather than a fix for it.

One key per place. conn:{env}:{user}:{slot} for slots 0 to 4. SET NX PX claims, SET IFEQ PX renews, and a conditional one-millisecond tombstone releases.

Analysis

The atomicity is the requirement, and it is the whole of the argument. A cap is not a count that is checked; it is a place that is claimed. SET NX settles a race inside the command — the loser gets nil and walks to the next slot — so no check-then-act window exists to lose, and no Lua is needed to close one. This was measured rather than assumed: replacing SET NX with a GET followed by a SET admits all twelve of twelve simultaneous attempts.

Constitution VII's evidence cannot be produced by this repository. The largest fixture holds five channels. NFR-SCL-01's ten thousand connections per instance is a budget the SAD itself flags as unmeasured (R2, "the SAD's single most urgent action item"). An ADR arguing that a sorted set outperforms five keys at that scale would be arguing from a lane that cannot see the difference — and the honest reading of the principle is that the ADR is not available, not that the principle is inconvenient.

Making the member a key makes the TTL per member by construction. The defect §6.3 recorded is not worked around here; it stops existing. A slot key expires on its own schedule because it is its own key.

The driver that was wrong. The first draft of this decision argued that a returning connection could not resurrect a place it had lost, because the renewal used SET XX and XX would refuse a key that had been re-claimed — citing presence.ts:195, which refreshes with XX for a related-looking reason. XX tests existence, not ownership. Measured on Redis 8.10.0: SET k B XX against a key holding A returns OK and the value becomes B. So under XX a connection whose place had expired and been taken would silently take it back, and six connections would be open against a count of five. The analogy did not transfer because presence's value is the literal "1" and carries no identity — there is nothing for a comparison to be wrong about. The renewal uses IFEQ, which compares before writing.

The decision survived its own broken driver, which is the least comfortable way for one to be right. Constitution VII says disagreement attacks the driver; an ADR is therefore not allowed to ship carrying a false one.

Decision

Five keys, conn:{env}:{user}:{slot}. Three commands, all conditional: SET NX PX to claim, SET IFEQ PX to renew, SET <tombstone> IFEQ PX to release. The TTL is the only unconditional way a place is freed, which is what makes a crashed instance recoverable without anybody sweeping anything.

Consequences

Counting a user's connections costs five reads. There is no ZCARD here, and the count comes back as a by-product of the walk — the index of the slot that was claimed. Nothing in chapter 3.22 needs the count without also claiming a place.

A tombstone means free, not busy. The release cannot use DEL, which has no ownership check and would let a connection whose place had already been re-claimed delete the new owner's key. It writes a value no connection id can equal, with a one-millisecond expiry, only if the place is still ours. The walk claims a tombstoned slot rather than stepping over it — and reading it the other way was a defect that reached the integration suite: a deploy releases all five places at once, and a walk that skips tombstones finds none free and refuses the reconnect with connection_limit_reached, whose documented remedy is to close one of the connections the client is holding. They went with the old instance.

The cap is enforced, and it fails open. An unreachable registry accepts the connection and logs that five was not checked, because Redis is not a source of truth here (Principle IV) and a cap that denies service when its bookkeeping is unavailable has chosen the wrong failure.

Revisit when

A chapter needs a user's connection count without claiming a place — an admin API, a dashboard, a support tool answering "why can this person not connect". Five reads is the wrong shape for a question asked from outside the connection path, and by then the load test R2 has been owed since the first draft will have produced exactly the profiling evidence Constitution VII asks for. The sorted set returns with an argument it does not have today.

ADR-24 — Message revisions take a fifth subject grammar

Problem

FR-RTM-05's last two kinds get producers in chapter 3.23. The fabric that carries a message between gateway instances is chan:{channel_id}, and its payload has been a Message since chapter 2.2 — §6.3's row calls it "the fan-out fabric" and the code's comment says the subject "has always carried a wire frame's payload rather than a shape of its own."

Neither new kind fits that shape, for two different reasons, and the second is the one that makes this an architecture decision rather than a refactor.

Options

Ride chan: unchanged. Impossible for a deletion: messageSchema.text is z.string() and a tombstone has none. The same constraint gave message.deleted its own frame payload in this chapter, and two places in the api had already refused to publish a tombstone and said why.

Widen chan:'s payload to a discriminated union. An edit is a Message, so only the kind is missing — and the kind was never on the fabric at all: the receiving gateway stamped type: "message.created" at the call site. Adding a discriminator means every existing consumer branches on a field it has never seen, and a grammar four chapters have treated as message-shaped stops being one.

Two new subjects, one per kind. Precise, and it doubles the subscription bookkeeping: a gateway would subscribe to both for every channel it holds, because a client that wants edits wants deletions.

One new subject with a discriminator in the payload.

Analysis

A kind that cannot share a payload type cannot share a subject. Three chapters reached that independently before this one — presence, membership, typing — and each took a grammar rather than widening chan:. The rule is not about tidiness: a subject's payload schema is the contract two instances agree on across a rolling deploy, and a union is a contract that changes shape under both of them at once.

The choice between one subject and two is ADR-20's, not ADR-19's. Presence took a single-purpose subject because presence is one thing. Membership took one subject carrying change: "added" | "removed", because an addition and a removal are two things that happen to one membership and a receiver wants both. An edit and a deletion are two things that happen to one message. The shape follows the audience, not the arity.

No environment on the payload, unlike membershipFabricSchema. Membership needs it because member:{env}:{user} names a user, unique only within an environment, and a receiving gateway must check it before acting. A channel id is a UUID and identifies its tenant transitively — the assumption chan:{channel_id} has relied on since chapter 2.2.

Decision

revision:{channel_id}, carrying { kind: "updated", message } or { kind: "deleted", message } as a discriminated union of strict objects. The deleted arm reuses the wire frame's own payload schema rather than restating it, so the fabric and the frame cannot drift.

Five grammars now: chan:, member: in two shapes, presence:, typing:, revision:.

Consequences

The gateway stops deciding the kind at the call site. It reads it from the payload, which is the change that makes an edit distinguishable from a creation — and the falsification for that is in the chapter: stamp message.created again and the delivery test goes red.

A sixth grammar is now likelier than a fifth was, and that is worth saying rather than discovering. Four of these five arrived one per chapter, each with the same argument. If a seventh kind appears, the question to ask first is whether the subject count is the design or a symptom — this record is the fourth in a row and none has asked it.

Revisit when

A mutation arrives that is not per-channel: a moderation sweep across an environment, or a retention job deleting by age. member:{env}:{user} exists because membership found exactly that case one chapter after taking its channel subject, and this grammar would need its principal-addressed sibling for the same reason.

Reading the twenty-four together

Three themes recur, and naming them is the best summary of the architecture's character:

Durability has one home; everything else is allowed to be cheap. Postgres holds truth (ADR-04, -06); therefore the fan-out fabric may be lossy (ADR-07), presence may evaporate (ADR-10), the analytical store may go down (ADR-08), and the broker may be modest (ADR-02). Every "surprisingly relaxed" choice is purchased by one strict one.

The write path is sacred; features live on the read side. Sequences commit with rows (ADR-03), sends converge on one code path (ADR-05), and emoji never touch stored text (ADR-11, -12). When a new feature threatens FR-MSG semantics, the design reflex is to reformulate it as a read-time concern. Even the application stack bends to this theme: the framework serves the wide read-and-CRUD surface and stops at the gateway's door (ADR-15), and the data layer was chosen so the write path's invariants stay written in visible SQL rather than behind an abstraction (ADR-16).

Isolation is structural, never procedural. The single writer (ADR-04), the repository layer that cannot express an unscoped query (ADR-16), and the refusal to merge the two user populations (ADR-18) are one argument in three places: tenant safety is a property of the shapes, not of anyone's attention. Each of those decisions rejected a cheaper option whose only flaw was needing someone to be careful.

Every decision names its own undoing. Reversal triggers are part of each record — not decoration, but the discipline that keeps a solo-built system honest: the day a trigger fires, the argument for change is already written. ADR-19 is the first record to test that claim, and it complicates it: the change arrived, the prepared argument fit, and the trigger had not fired. A written remedy can be reached by a road it did not anticipate — which is worth knowing, because the trigger it was attached to is then still owed.

ADR-25 — A measured threshold for consolidating the subject grammars

Problem

Five subject grammars carry real-time events between gateway instances, and each arrived by the same argument: a kind that cannot share a payload type cannot share a subject. ADR-19 took presence:{channel_id}, ADR-20 took member:{channel_id} and member:{env}:{user}, ADR-21 took typing:{channel_id}, ADR-24 took revision:{channel_id} alongside the original chan:{channel_id}.

Each was right on its own terms. The implementation review of 2026-09-03 raised the shape rather than any instance:

One subject grammar per real-time kind is becoming a default rather than a measured decision. ADR-19→24 preserve payload compatibility well, but add subscriptions, Redis clients, recovery paths, and operational complexity per event kind. NFR-SCL-01 remains unmeasured. Define a threshold for consolidating typed envelopes or introducing a versioned event bus before adding another fabric.

The rule was sound and unbounded. Applied indefinitely it ends somewhere nobody chose, and there was no number at which anyone would notice. NFR-SCL-01 — 10,000 concurrent connections per gateway instance, P1 — had never been verified, so the cost of a grammar could not be stated even approximately.

Options

A. Consolidate now onto a typed envelope. One subject per channel; the payload carries kind. Collapses the per-channel subscription count from five to one.

B. Introduce a versioned event bus. Replace Redis pub/sub for real-time with a broker that multiplexes kinds over one connection.

C. Set a threshold and keep the rule below it. Measure the cost, write down the number at which A or B becomes correct, and leave the rule in force until then.

D. Leave it. Continue applying the rule; revisit when something hurts.

Analysis

The measurement is docs/11-scalability-measurement-2026-09-06.md, run because NFR-SCL-01's own verification method is A — analysis including load testing — and because a threshold is a number that nobody had.

One api and one gateway, real client sockets, the gateway's RSS from /proc, and the broker's subscription count from PUBSUB CHANNELS — the subscriptions are visible only from Redis's side, so reading them off the source would not have been a measurement.

connections  channels  survived  gateway RSS  redis subjects  redis mem
    10,000       200    10,000        160 MB          11,000       7 MB
    10,000    10,000    10,000        157 MB          60,000      18 MB
    20,000       200    20,000        371 MB          11,000       9 MB

NFR-SCL-01 is met, and the clause's verb is sustain: the 10,000 were held 90 seconds, past two of the gateway's 30-second ping intervals with MAX_MISSED_PINGS = 2, with zero closures. Twice the clause also holds.

The subscription cost is exactly 5 × channels + 1 × connected users, satisfied to the unit in every row. The five per channel are chan, revision, presence, typing and member; the one per user is member:{env}:{user}.

Three things follow, and the third is the one that decides this.

The grammars are not what costs. Redis holds 60,000 subscriptions in 18 MB — roughly 300 bytes each — and the gateway's RSS is indistinguishable between 11,000 and 60,000 subjects, 157 MB against 160 MB. A sixth grammar costs about 10,000 subjects and 3 MB in the worst ratio.

Consolidating would trade a real property for a saving nobody needs. Under a typed envelope a receiver takes every kind on a channel whether it wants them or not, and the five grammars exist precisely so that it does not. Option A buys 4 SUBSCRIBEs per channel and spends the property ADR-19 through ADR-24 were each written to protect.

Which term dominates is a property of the customer's data, not of the platform. At 200 channels the five grammars are 9% of subjects; at one channel per user they are 83%. A threshold expressed as a raw subject count would therefore be wrong for half of all deployments — it has to name the ratio, or name a projection that includes it.

Option B is Option A's cost plus a new operational dependency, for the same saving. Option D is what the review objected to, and correctly.

Decision

Option C. The rule stands; the threshold is written down.

A new real-time kind takes its own subject when it cannot share an existing payload type — ADR-19's rule, unchanged.

Consolidate onto a typed envelope when either holds:

• per-channel SUBSCRIBEs would exceed six, or
• a gateway instance's projected subject count exceeds 250,000, computed as 5 × channels + 1 × connected users at the deployment's own channel-to-user ratio.

250,000 is 10,000 connections at one channel each under six grammars, plus a 50% margin. It is chosen so that the worst ratio at NFR-SCL-01's stated scale sits below it and a seventh grammar at that ratio does not.

The projection is the operative half. Anyone proposing a sixth or seventh grammar computes 5 × channels + 1 × connected users for the largest deployment in view. That is one query and a multiplication, and it is the step the review found missing.

Consequences

The rule is now falsifiable. "One grammar per kind" was unbounded and is bounded, by a number derived from a measurement rather than from taste.

A sixth grammar is pre-approved and a seventh is not, at NFR-SCL-01's scale. The next kind that cannot share a payload type takes its subject without an argument; the one after that reopens this record.

NFR-SCL-01 is discharged as a side effect, recorded at SRS revision 1.9. It had been P1 and unverified since v1.0.

A finding this did not set out to make. DEFAULT_LIMITS.connect is 3,000 per minute and the gateway accepts 1,125-1,675 per second — so filling an instance from cold takes 3 minutes 20 seconds by policy where capacity would take 7.7. The limiter's own comment derives it from NFR-SCL-01, so this is deliberate; the two numbers had simply never been placed side by side. It bears on rolling restarts and on ADR-20's revocation window, and it is recorded rather than changed here.

The measurement left 10,604 channels and 24,020 users in the test lane, prefixed scale-c-* and scale-u-*. Identifiable, removable in one statement, and worth knowing before the next measurement is taken on that lane.

Revisit when

• a proposed kind would make per-channel SUBSCRIBEs seven;
• a deployment's projected subject count exceeds 250,000 by the formula above;
• NFR-SCL-01 is raised beyond 10,000 per instance, which moves the projection;
• Redis pub/sub is replaced for a reason unrelated to this record, at which point the threshold is about the replacement's costs and not these.

ADR-26 — The api serves a customer request from the analytical store

Problem

FR-ANL-07 asks for a queryable API request log per tenant, and FR-DSH-03 asks the dashboard to search it — which EIR-DSH-02 then binds to "only the same public API available to customers." So the log has to be a public route, and the rows are in ClickHouse.

That is new in a way the three earlier analytical chapters were not. Chapter 4.2 loaded the store from Postgres offline. Chapter 4.4 wrote to it from a fire-and-forget producer that blocks nothing. Chapter 4.7 read from it in a nightly job. This chapter puts it on the request path, with a person waiting, and neither ClickHouse client in the repository had a timeout: both call fetch with no signal, so a hung store held the caller until the operating system gave up.

Constitution III's second sentence is a MUST about exactly this: "failure or backlog of the analytical pipeline MUST NOT affect message delivery, real-time fan-out, or API availability." A route that hangs on a slow store is the coupling that clause forbids, and until this chapter nothing in the platform could produce it.

Options

A. Serve it from the analytical store, with a deadline on both sides. The store the producer already writes to, read with a client-side abort and a server-side execution limit.

B. Serve it from the analytical store with a client-side deadline only. One change, in one place, no SQL involved.

C. Deduplicate with LIMIT BY rather than FINAL. Cheaper on paper, since FINAL merges parts at read time.

D. Mirror the log into Postgres and serve it from there. The operational store is already on the request path and already has a deadline story.

What was measured

B fails on its own terms, and the measurement is one line. Aborting a fetch stops the client waiting; ClickHouse keeps executing. A tenant retrying a slow page therefore accumulates server-side work — the opposite of what a deadline is for. The server half needs no interface change:

SETTINGS max_execution_time = 1
→ HTTP 408 · Code: 159. DB::Exception: Timeout exceeded: elapsed 1000.343075 ms,
  maximum: 1000 ms. (TIMEOUT_EXCEEDED)    returned in 1.002 s

And the order of the two limits decides what the caller learns. With the server's limit shorter, its refusal wins the race and the route receives Code: 159 and an HTTP status to map. With the client's shorter, the route receives an AbortError carrying nothing, and a refusal that names no cause is the empty page this surface refuses to send. The route sets 2 seconds on the server and 3 on the client.

C changes what LIMIT counts. LIMIT n BY key returns up to n rows per key, so a page's size stops meaning the number of requests the caller asked for. And the problem it solves is real: measured on the lane before the reader was written, api_requests held 11,684 rows against 11,683 distinct keys — one duplicate, across two active parts — so a read without FINAL returns that request twice and the repeat disappears whenever a merge happens to run. the surface's promise that no row appears in two consecutive pages would then pass or fail on merge timing.

What FINAL costs was measured against a table with parts, because the obvious measurement is taken on a table that has just been merged and proves nothing. With merges stopped and six inserts planted:

7 parts, 12,895 rows     FINAL 3 ms     plain 2 ms     read_rows 12,895 both

Three runs a side, identical every time.

D has nothing to serve. The producer writes to ClickHouse and nowhere else; mirroring means a second producer on the request path, which is the coupling FR-ANL-02 forbids — "never synchronously on the request path" — and constitution III's first sentence forbids the analytical query against Postgres in any case.

Decision

A. The route reads ClickHouse with FINAL, a server-side max_execution_time of 2 seconds and a client-side abort at 3, and refuses with 503 analytics_unavailable when the store does not answer. The refusal is explicit rather than an empty page because an empty page is a claim about the tenant — that they made no requests — where the platform is what failed.

Consequences

The platform now has a failure mode it did not have: a customer-facing route that can be unavailable while the rest of the API is fine. That is the point of the 503 rather than a side effect of it — constitution III's second clause is about exactly this distinction, and a surface that hid the difference would satisfy the clause by being unable to report on it.

The store's error message never travels. Code: 159. DB::Exception: … elapsed 1000.343075 ms in a customer's support ticket is infrastructure detail, which is the argument codes.ts already makes about credentials (NFR-SEC-06).

And the distinction the mapping turns on is whose fault it is. A timeout or no answer at all is the pipeline being unavailable, and the client should retry. A 404 or a syntax error from ClickHouse is the platform's statement being wrong, and telling a customer to retry a query that will never work is worse than telling them nothing — those stay internal_error.

Reversal condition

Reverse when the analytical store's availability appears in the API's error budget — when analytics_unavailable stops being a rare answer on one route and becomes a number somebody reports. The 503 analytics_unavailable refusal is the instrument that makes that visible, and it shipped in the same chapter as the route so the signal exists before the decision needs reviewing.

The options at that point, in the order they cost least: a cache in front of the read; a Postgres mirror of the log, which requires amending constitution III rather than reinterpreting it; or removing the surface and with it FR-DSH-03.

Not a reversal: the store being slow. A page that takes a second and a half inside the deadline is the trade this record accepts, and the deadline is what bounds it.

ADR-27 — The integration gate reports which suites ran, and every lane runs

Problem

docs/12 §2.3 asks movement IV to close with "a CI gate that catches a planted drift on every run". The planted-drift suite was written at chapter 4.7 and the gate around it does not carry the signal that sentence assumes.

Three separate facts, each measured at this feature's phase 1:

The gate was red, and had been on every run since chapter 4.4. Six suites failed: five in request-log.itest.ts, which polls for a row only the ingester can write while compose.yaml ships no ingester (gaps.md 050-8), and one in limits.itest.ts, which asserts the lane has a platform credential configured and nothing configures one. A planted drift makes the run red; the run was already red. The signal was not absent, it was indistinguishable.

A run stopped at the first failure. --concurrency=1 means turbo stops scheduling when a task fails, so the gateway's 12 suites, the e2e journey, the ingester, the dispatcher and the harness went unexecuted whenever the api lane failed — which was every run.

And the line the gate printed counts tasks. Tasks: 8 successful, 10 total for a run that planned 18: nine builds, nine test tasks, three of which are packages with no test:integration script and complete as successful no-ops. Worse, the total is not reproducible — three chapters' close-out runs against the same tree printed 7 of 9, 8 of 10 and 7 of 11, because which tasks were already in flight when the first failure arrived decides it. A requirement cannot be written against a number that changes when nothing does.

Options

A. Report suites, run every lane. A small script in front of turbo: derive the suite count from the tree, pass --continue, parse each lane's own vitest summary, and refuse a run where a lane reported nothing or fewer suites ran than exist.

B. Raise --concurrency. Lanes in parallel would also reach every lane. Feature 045 measured the cost: the api lane's knee is two workers and the gateway's is four, against one PostgreSQL — raising turbo's concurrency stacks whole lanes on top of that.

C. Exclude the failing suites. package.json already carries --filter=!@relay/outsider, so the shape has a precedent.

D. Leave it, and read the failures by eye. What four chapters did.

What was measured

C cannot do the job, and the precedent is what misled three analysis passes. --filter selects packages. @relay/outsider is its own package; the five red suites sit inside @relay/api beside reconcile.itest.ts — the suite this gate exists to reach. No filter expression keeps one and drops the other. Separating them needs a vitest exclude, a second config, or moving the file, each of which changes what pnpm --filter @relay/api test:integration means for everyone who runs it.

The ingester did not need a compose service, and the obvious fix would have been worse. services/ingester has no Dockerfile, so it is a new image rather than a service definition; and api, gateway and dispatcher all carry profiles: ["services"], so docker compose up -d starts the stores and nothing else. An ingester added beside them would not be running when the lane runs, and one added to the default profile would drain the analytics stream on every developer's machine for as long as the stack is up — changing the opening state of every analytical suite in the repository. The suite spawns the process it needs and kills it afterwards, which consumer.itest.ts and outbox.itest.ts already do for a Node child. Measured on the first run: the ingester drained 8 batches and 1,038 records, which is the backlog the stream had been holding because nothing had ever consumed it; the second run drained 12.

One unset environment variable was costing more than the test that reported it. limits.itest.ts fails loudly without RELAY_INTERNAL_CREDENTIAL. Three attacks in isolation/gauntlet.itest.ts — the suite constitution VI names as gating releases — read if (dispatcher === undefined) return; and reported green without running. The suite's own accounting test cannot catch it, because attacked.add(...) happens before the early return, so the check that exists to find unattacked routes is satisfied by the route that was skipped. Run with and without the variable, the same three green ticks:

expand reaches only the endpoints of the environment it names   1ms → 27ms
expand naming an environment that exists nowhere                0ms →  6ms
a connection billed to one environment cannot be re-billed      1ms → 33ms

And the gateway lane held a test that had never delivered the frame it published. typing.itest.ts's four-kind test publishes a presence payload of five fields; presenceFabricSchema is a z.strictObject of three, so safeParse fails and presence.ts:284 logs presence.invalid_payload. The assertion passed anyway, at whatever rate the previous run's Redis key had expired — the gateway publishes a presence transition of its own when a connection opens, elected across instances by SET … NX, and that is the frame the test was counting. Alone it was 0 of 3 green. It also carried await settle(400) before publishing, a bet that four Redis SUBSCRIBEs finish in 400 ms; PUBSUB NUMSUB answers the same question as a condition. Both fixed, 23 of 23 three times, and the file got 10 seconds faster because a test was no longer burning its arrival deadline.

Decision

A. pnpm test:integration runs scripts/integration-gate.mjs, which derives the lane list and suite count from the tree, runs turbo with --continue, and prints what ran against what exists. It exits non-zero on three conditions: a lane failed, a lane reported no summary at all, or fewer suites executed than the tree holds.

The six reds are resolved rather than excluded: the credential is configured in the lane's own config beside feature 030's relay flags, and the request-log suite starts its own ingester.

Consequences

The gate is green on a clean tree for the first time since chapter 4.4, so a planted drift is now a change of colour rather than a continuation of one. Every lane runs on every push, which costs the wall-clock time of the lanes that used to be skipped.

The suite count is a number a requirement can name, and it is checked in both directions: a lane that silently stops reporting fails the gate as loudly as a lane that fails.

What this does not fix is one level up. ci.yml's tutorial job ends with pnpm check:fences, which exits 1 at the standing 110 problems and has on every push since feature 045. The workflow is red whatever this gate says. Nothing is stranded behind it — it is the last step of its job — but the workflow's own colour has not been a signal for nine chapters, and this ADR does not change that. A milestone about a gate that cannot fail for its own reason should not stand inside one without saying so: the series owns that decision, the options are a recorded baseline the checker compares against, continue-on-error with the count published, or a job of its own, and this chapter records the fact rather than choosing for the series.

Reversal condition

If the lanes' combined wall-clock time under --continue becomes the reason people stop running the gate locally, split it: the api and gateway lanes on every push, the rest nightly. The suite-count report survives that split unchanged, because it reads the tree rather than the schedule.

If vitest's summary line changes shape, the parser refuses the run rather than reporting a smaller number — that is deliberate, and the fix is the parser rather than the gate.

ADR-28 — FR-ANL-06's daily job has no runner, and that is recorded rather than built

Problem

Constitution III's fourth bullet: "Metered totals MUST reconcile against operational counts to within 0.1%, verified by a daily job that alerts on breach." SRS FR-ANL-06 says the same in its own words.

That sentence is three requirements and the platform has one. The comparison exists — chapter 4.7 built it, and scripts/reconcile-usage.mjs runs it for one tenant and one period. The alert has no mechanism, recorded at SRS revision 1.14: the job exits non-zero, and the two mail paths share a transport that defaults to a local catcher. And the daily job has no runner of any kind, which nothing had written down until this feature looked:

reconcile-usage in relay-platform/package.json     0 occurrences
                   relay-platform/turbo.json        0
                   .github/workflows/ci.yml         0
                   relay-tutorial/package.json      0
                   any *.sh in either repository    0
ci.yml triggers    push, pull_request — no `schedule:`

Five corpora, named because a zero from a grep is a claim about the corpus only if the corpus is named. Meanwhile the platform runs five background loops — RELAY_OUTBOX_RELAY, RELAY_DELIVERY_RELAY, RELAY_QUOTA_RELAY, RELAY_NOTIFICATION_RELAY, RELAY_EVENT_CONSUMER — so the pattern for recurring work exists and the reconciler is the one recurring job built as a script somebody runs by hand.

Options

A. A sixth background relay. RELAY_RECONCILE_RELAY, beside the five the api already starts, sweeping tenants on a daily tick.

B. A schedule: trigger in ci.yml. A cron in the workflow that runs the script against a deployment.

C. Record the absence, and state what the lane's gate actually verifies.

What was measured

A's cost is already on the record, from the feature that measured the other five. Feature 030 found that nine suites booting AppModule in process started four background loops that swept the whole database while every other suite's fixtures sat in it — and a relay catches and logs its own errors, so the damage was a log line and a green lane. Every analytical suite would gain another flag to set. That is the price of a sixth loop, and it buys a daily sweep of every tenant on a platform where every tenant is one-sided: 2,440 usage_periods rows over 2,330 environments, 7 daily_usage_billing rows over 4 environment ids, and zero environments in both. A daily job over today's data would report no-data for every tenant, every day.

B needs a deployment to run against, and ci.yml has none: its three jobs build and test a checkout. A scheduled workflow would reconcile a corpus it had just built, which is what the lane already does per push.

And the gate is not the clause's "daily", which docs/12 §2.3 substituted without saying so. §2.3 offered "the lane, every run" in place of "daily". A per-push check on a planted fixture runs more often than daily and reads no real tenant — it is a different claim, not a more frequent one. The planted-drift suite verifies that the comparison detects a drift; only a scheduled job against real data verifies that no drift exists.

Decision

C. The schedule is recorded as absent, and the milestone's claims are scoped to what runs: the arithmetic is verified by the planted-drift suite on every push (FR-ANL-06's verification letter is T), and the 0.1% figure is published once, at a volume where 0.1% resolves, in docs/13.

The constitution amendment this feature proposes decomposes the sentence, so that which of the three parts the platform has is a matter of record rather than of inference.

Consequences

The clause is not discharged, and the platform says so in the two documents that govern it rather than letting a per-push fixture check stand in for a daily reconciliation of real tenants. A milestone that claimed otherwise would be claiming the weaker of two different things.

Reversal condition

Build A the first time a tenant exists with both sides of the comparison populated by the platform itself rather than by a harness — that is, once the analytical path has a producer writing message_events in normal operation. Until then a daily sweep would report no-data for every tenant on the platform, and a check that always fires stops being read is the same argument in the other direction.

ADR-29 — A fence claims a file or declares it is not one, and the checker can print its own state

Status: accepted (2026-09-18) · feature 055

What was actually wrong

pnpm check:fences reported 110 problems and had done since feature 045. The tutorial job in ci.yml ends with that command, so the workflow was red on every push for nine chapters and its result carried no information: a chapter that broke something and a chapter that broke nothing produced the same colour.

The number was not 110 defects. Measured at the opening: 47 targets, of which 36 are platform files and 11 are phrases naming none; the Vietnamese half mirrors the English half exactly with zero vi-only problems; and one file held 16 of the 110 with one hunk explaining nine of them.

The two changes, which are one decision

A titled fence is a claim that the file is these bytes. Everything here follows from taking that literally.

Eleven fences claimed a file and named none. "the ladder against the registry", "the typo, now", "42P01", "run 11 of 20" — all eleven lang=text, all eleven command output: two checker reports, a tsc TS2345, two Postgres errors, two catalogue refusals, a vitest assertion, three source locations, a JSON gauntlet report. They were read before they were declared, and there was no twelfth candidate.

The repair is (excerpt) in the title, which NOT_A_FILE at check-fence-chain.mjs:42 already skips in both the chapter loop and the appendix loop, and which 222 titles in this series already carried — 111 in each locale. It is applying a convention, not inventing one.

And the chain's state is not the repository's, so a hunk cannot be written against the tree. vitest.coverage.config.mts replays to 318 lines where the tree holds 1,182 and has no env block at all. turbo.json replays to 62 lines at chapter 3.22 and 74 after the appendix. A hunk generated with git diff against the working tree carries context that does not exist where the hunk lives, and is rejected for a reason neither tool explains.

So the checker gained --dump <dir> [--at <page>]:

There is no --locale flag: a page path begins app/(en)/ or app/(vi)/vi/, so the chain is inside the argument copied off the problem line, and a flag could only disagree with it. A bare --dump writes the English chain and prints that it did, because the appendix mutates en.state only and the two chains end 9 paths and 13 lines of turbo.json apart.

This is an output mode. No threshold, no exemption, no change to any exit code, and --at runs a separate replay with its own throwaway problem list so the dump cannot add to, remove from or reorder what the check reports. Verified: the count is 110 with the flag, without it, and with the -- that pnpm check:fences -- --dump X forwards.

Rejected alternatives

Teach the checker that lang=text is never a file. One line, and it exempts a class without anybody reading its members. The eleven were read; a twelfth that looked like a listing would have been caught by reading and missed by a predicate.

Retitle the eleven to real paths. It makes the claim false rather than absent — there is no file whose contents are a Postgres error message.

Remove the titles. That moves them into the 360 fences no gate reads at all, which is one step further out than the class they are in now.

A separate script importing the checker's internals. check-fence-chain.mjs is a top-level program with no exports, so this means refactoring it into a module: more change for the same result.

Keep making throwaway copies, as fence-chain rule 1a says. Feature 054 did this twice and deleted it each time, which is right for a single use. Fifty regenerations is a different shape — and the copy has a failure mode that argument alone would have missed. The checker resolves relay-platform from its own file location, so the identical file run from another directory prints relay-platform not found — skipping and exits 0 having replayed nothing. The copy rule 1a asks for is one cp away from being a silent no-op, which is check-lane-scope.py's failure inside the rule cited to justify it.

What the repair found, and one of it was in the instrument

The applier corrupted $$. applyHunks used text.replace(pre, post) with a string replacement, and String.prototype.replace reads $$, $&, $`, $' and $<name> in a string replacement as substitution patterns. packages/test-harness/src/sentinel.sql is DO $$ … END $$; twice over, so the chain replayed DO $ … END $; — and that difference was filed for two features as a typo in the published listing. The listing says END $$;. A reader copying it is fine.

Fixed with a function replacement, text.replace(p, () => post.join("\n")), which is not subject to substitution. It was causing APPLY failures as well: with the applier corrected and the original published hunk restored, chapter 3.23's fence applies untouched, so two of the 42 "bad hunks" were never bad. This is a bug fix rather than a decision, and it is recorded here because it changed a shared gate.

A count of 110 was neither 110 defects nor an upper bound on them. 42 bad hunks were cleared by 24 repair operations, so 18 were shadows — one appendix hunk cleared five at once. And ten files could not be compared to the repository at all, because a checker reports the first failure per file and a file with a broken hunk never reached its HEAD comparison. session.itest.ts turned out to be 1,322 lines behind; turbo.json and packages/e2e/src/harness.ts had been diverging invisibly.

Reversal condition

If a later feature teaches the checker to distinguish command output from a file by something other than the title — a lang allow-list read from a declared set, say, with its members listed and checked — the eleven declarations become redundant and should be removed in that change.

If --dump is used fewer than five times in a year, fold it back into a throwaway copy and delete the flag. The argument for it is fifty uses, not elegance.

ADR-30 — Relay signs its own presigned URLs

Status: accepted · 2026-09-19 · chapter 4.10, feature 056

The decision ADR-13 left open

ADR-13 settled the shape: a client uploads straight to object storage against a URL Relay signs, and media bytes never transit Relay compute. It is the reason a 100 MB video costs the api a request and a row rather than a request, a buffer and a second request. What it did not settle is what produces the signature, because no chapter needed one until this one.

Three options, and the dependency count is not the whole of the argument.

Option A — @aws-sdk/client-s3 and @aws-sdk/s3-request-presigner

The default answer, and the right one for most codebases. Two packages, a transitive tree in the dozens, and a maintained implementation of an algorithm that is genuinely unforgiving.

What argues against it here is the ratio. This platform's use of object storage, today and for the two chapters that follow, is: create one bucket at boot, sign a PUT, sign a HEAD. It does not list, copy, tag, set lifecycle rules, or multipart-upload. services/api carries a 160 MB working-set budget (SRS revision 1.9) that was measured at 157, and a dependency tree is memory before it is anything else.

Option B — minio

One package instead of two, a smaller tree, and an API shaped like the store this chapter provisions.

What argues against it is the direction the platform is trying to keep open. The whole point of choosing an S3-compatible store is that the store is replaceable; a vendor's client library is an implementation of the S3 canonical-request algorithm with that vendor's name attached, and binding the signature to it at the moment the platform is choosing a replaceable store is the wrong trade in the wrong place. MinIO is the development store, not the production one.

Option C — node:crypto, and it is what shipped

Signature Version 4 for a presigned URL is five HMAC-SHA256 rounds over a canonical request: date, region, service, aws4_request, then the string to sign. Every input is a string the api already holds. Written out with the comments the repository's style asks for, it is twenty-eight lines of logic and the dependency count moves by zero.

Two details carry more weight than their size suggests.

UNSIGNED-PAYLOAD is what makes the presigned form possible at all. The signature cannot cover a body Relay never sees, so the canonical request names that literal in the payload-hash position and the store accepts it for a presigned URL specifically.

The canonical URI differs between a bucket operation and an object operation — /{bucket} against /{bucket}/{key} — and the signature is computed over it. The chapter's first probe created a bucket with mkdir on the container's volume and therefore never signed a bucket operation, which is why four of the suite's nine assertions are about the bucket.

The acceptance is a running store, not a string

A signature that is consistently wrong passes every unit test you can write about it. The URL has a stable shape, the same inputs give the same output, the query parameters are in canonical order — and none of that is a claim the store will accept it. The failure mode is a bare 403 SignatureDoesNotMatch with no indication of which field was wrong.

So the signer's acceptance is nine questions asked of a running MinIO, as an integration suite rather than a probe run once:

create the bucket                          created, then exists
signed HEAD on the bucket                  200
unsigned LIST of the bucket                403
signed PUT of an object, no credentials    200
signed GET, same object                    200, same bytes
unsigned GET of that object                403
expired URL                                403 · Request has expired
one character of the signature changed     403 · SignatureDoesNotMatch
signed with the wrong secret               403

The two unsigned 403s are the ones the design rests on, and the expiry is refused by the store's own clock — which is what lets the api publish expires_at without keeping a second record of it.

And the tampered-signature assertion was a flake before it was evidence. It replaced the signature's first character with f, a no-op whenever that character already was f: measured over 4,096 signings, 255 of them, 6.23%. On those runs the probe sent a valid URL and the store's honest 200 read as a tampered signature being accepted. A probe that may not have altered anything has to assert that it did.

The cost that is not the dependency

A presigned URL needs no contact with the store. That is the design's whole economy and it is also the reason feature 056's FR-017 needed something built for it: the api never opens a socket, so it never learns the store is down, and a slot issued into an outage is byte-identical to a good one.

docs/05-sad.md:1062 requires the opposite — "Object storage lost … Upload slots return a specific error" — so storeReachable() performs a signed HEAD on the bucket before anything is written, with a two-second timeout because cannot be reached includes does not answer.

slot request, probe on     min 6.266   p50 7.859   p95 9.007   max 10.135 ms
slot request, probe off    min 4.849   p50 6.335   p95 7.260   max 10.350 ms
the probe alone, n=200     min 0.944   p50 1.062   p95 1.194   max  1.958 ms

+1.524 ms at p50, +24.1%, 200 samples a side, the same binary minutes apart. Two thirds of it is the round trip and the rest is the second signing. That is the price of the clause and it is published rather than described, because the alternative design — issue the slot and let the client discover the outage — costs the client a wasted upload and a URL it cannot distinguish from one it mistyped.

Reversal condition

If a later chapter needs listing, copying, lifecycle rules, server-side copy or multipart uploads, the SDK's tree stops being overhead for one function. At that point this decision should be re-taken as a whole rather than extended one hand-written signer at a time: a codebase with one signed operation and a codebase with eight are different arguments, and the second one is the SDK's.

If the production store turns out not to be S3-compatible, neither option B nor option C survives, and that is a store decision rather than a signing one.

ADR-35 — The audit log is operational, its immutability is a trigger, and its timestamp is declared

Problem

FR-MOD-03: "Every moderation action shall be recorded in an immutable audit log with actor, action, target, timestamp, and request ID, retained for 1 year."

Three decisions are hidden in that sentence and none of them is the obvious one. Which store holds it, because this platform has two and a log is the shape that usually goes in the analytical one. What makes it immutable, because the mechanism a reader would reach for does nothing on this deployment. And what precision its timestamp has, which looks like a schema detail and decides whether a reader can page through the log without losing rows.

Constitution VII requires the record: "Every architecture decision is recorded as an ADR stating its drivers, rejected alternatives, and reversal condition." Nine analysis passes read past these three because no requirement captured them.

Drivers

• FR-005 of feature 064: the entry is written as part of the action it records, so an action that succeeded with no entry is not reachable.
• FR-006 of feature 064: a tenant reads its own history, paged, newest first.
• Constitution I: the read is scoped to the caller's environment and no parameter names one.
• Constitution III: the two data paths are independent, and a log is normally the analytical path's business.
• NFR-SEC-10 asks for an immutable audit trail for a different actor, and this decision is what determines whether that clause is reachable from here.

Decision 1 — Postgres, and ADR-26 is the same driver running the other way

The audit log is an operational table. FR-005 of feature 064 is the whole argument: the entry must commit with the action, and the action is a Postgres transaction. Writing it anywhere else makes the guarantee probabilistic — an entry published to a broker and consumed into ClickHouse can be lost after the ban succeeded, which is the case the clause exists to prevent.

This is the direct mirror of ADR-26, whose problem statement is "The api serves a customer request from the analytical store" and whose rejected option D was "Mirror the log into Postgres and serve it from there." That option was refused for the request log because the rows were already in ClickHouse and mirroring them would have been a second copy with a second retention story. The audit log's rows are written by the api inside a transaction, so Postgres is where they already are, and the same driver — serve the log from the store that already holds it — selects the other store. Two ADRs reaching opposite conclusions from one driver is a register working; reaching them without reference to each other is how it stops being one.

The consequence is one this chapter can state rather than fear: a moderation action records successfully while the analytical pipeline is down. Measured — docker compose stop nats clickhouse, then a ban and an unban, then two entries read back with the right actor, target and order.

Rejected: ClickHouse, with the entry published on the action's success. It is the request log's shape and it is wrong here for a reason the request log does not have: an API request is observed, and a moderation action is performed. Losing the record of an observation degrades a diagnostic; losing the record of a performance breaks the clause.

Decision 2 — a trigger, because the obvious mechanism was measured inert

REVOKE UPDATE, DELETE ON audit_log FROM relay is what a reader expects, and it does nothing:

select usesuper from pg_user where usename = 'relay';   →  t

revoke update, delete on probe_immutable from relay;
update probe_immutable set v='b' where id=1;            →  UPDATE 1   the value changed

A superuser bypasses privilege checks, and this api connects as one. Nothing in the schema would have looked wrong — the grant would be in the migration and the table would be mutable.

A BEFORE UPDATE OR DELETE trigger fires for a superuser and refuses both verbs, which is why it is the mechanism:

update audit_log set actor_id='tampered' …   →  ERROR: audit entries are append-only (FR-MOD-03)
delete from audit_log where …                →  ERROR: audit entries are append-only (FR-MOD-03)
                                                 and the row is still there

And the trigger looked unprecedented for a reason that does not apply here. No migration in services/api/migrations/ had ever created one, and that zero is enforced: packages/test-harness/src/no-trigger-in-migrations.test.ts asserts it over every .sql in that directory. Its own comment scopes the rule to the test-database sentinel guard — "the api ships a trigger whose only purpose is to reject its own legitimate sweeps, in production" — so the rule was wider than its reason, and the audit trigger's purpose is the opposite: it refuses writes the api must never make. The first assertion was narrowed to the guard by name, the sentinel check left alone, and the narrowing itself asserted so a later edit cannot widen it back in silence.

Rejected: row-level security. A superuser bypasses it too unless FORCE ROW LEVEL SECURITY is set, and it answers a different question — who may see which rows, not who may change them.

Rejected: hash-chaining each entry to its predecessor. It detects tampering rather than preventing it, which is a larger clause than FR-MOD-03 asks for. It is the thing to reach for if the reversal condition below is ever met.

Decision 3 — timestamptz(3), because the cursor compares it to a value from the wire

Postgres defaults timestamptz to microsecond precision and the platform serialises with toIso, which emits milliseconds. A keyset cursor minted from the transmitted value and compared against an undeclared column sits before every row inside the lost fraction, so in DESC order every row between .083000 and .083489 is skipped at the page boundary.

This is the platform's first keyset cursor over a Postgres timestamp, which is why nothing had met it: the request log's is exact because its column is DateTime64(3, 'UTC') and its wire format is millisecond, and the message history pages on an integer. The constitution already asks for millisecond precision in as many words; all 40 other timestamptz columns are precision 6, and that platform-wide deviation is not this chapter's to repair. This column is declared because it is the one place the deviation costs a reader rows — and it is unfixable once entries exist, since changing a column's precision rewrites every row.

Consequences

• The claim is scoped and the chapter publishes the scope. One SET session_replication_role = replica disables every trigger in the session, and DROP TRIGGER is one statement. Both measured, both succeed. The log is immutable to the application and to accident; it is not immutable to somebody holding the database password.
• NFR-SEC-10 is not satisfied by this and asks for the same words. "Administrative access to production data shall require multi-factor authentication and shall be logged to an immutable audit trail." Its actor is an operator with a database password — exactly the actor this trigger cannot refuse — and nothing records a psql session at all. The two clauses are different artifacts, and a reader who sees "immutable audit log, shipped" would reasonably assume otherwise.
• The retention FR-MOD-03 names is not enforced. Nothing prunes this table, on ADR-28's precedent: there is no scheduler. The read deliberately publishes no retention_edge, where the request log publishes one because its table has a TTL — announcing a boundary the platform does not enforce is worse than announcing none.
• Four actions gained a transaction they did not have, and it costs more than the entry does. Measured over 200 samples a side: the entry is 0.431 ms at p50 and the transaction about as much again, so an archive is 94.6% slower than before this chapter.

Reversal condition

A separate, non-superuser role for the application. That one change makes REVOKE UPDATE, DELETE real for the role the api actually uses, demotes the trigger from the only line of defence to the second, and turns psql as a superuser from this platform's ordinary posture into an exceptional act somebody can require a reason for. It is the single change that would move both FR-MOD-03's scope and NFR-SEC-10's, and it is a deployment decision rather than a chapter.

When it is taken, hash-chaining becomes worth its cost too: detection is a reasonable second layer once prevention is real, and it is not a substitute for prevention that does not exist.

Decision 3 reverses only if the platform's other 40 timestamptz columns are brought to millisecond precision, at which point this column stops being a special case and the declaration is redundant rather than wrong.

Applied to a second table 2026-10-03 (chapter 4.19). The decision is unchanged. message_edits is now append-only by the same mechanism, because FR-MSG-07 has said "an immutable edit history" since chapter 3.23 and nothing enforced it — measured before the migration: an UPDATE and a DELETE both succeeded. Both bypasses were re-measured on this table rather than assumed to transfer — SET session_replication_role = replica and DROP TRIGGER still succeed — so the scope published here holds for two tables and not one. Nothing in the Decision, the rejected alternatives or the reversal condition is rewritten: the reversal condition is a separate non-superuser role for the application, which is a deployment decision about this api and not a statement about which tables it guards. Recorded as a dated note rather than an edit, because constitution VII makes an accepted ADR immutable.

ADR-36 — A retention sweep is a compliance path, and the trigger names its one deleter

Status: accepted (feature 066, chapter 4.20, 2026-10-04)

Two decisions. The first is a reading of three documents that disagreed with a fourth; the second is a mechanism chosen after the obvious three were measured and refused.

Decision 1 — a retention sweep IS a compliance path

The conflict. FR-MOD-06 requires expired messages to be hard-deleted by a scheduled job. Three documents reserved that verb:

constitution II   hard deletion exists ONLY ON THE COMPLIANCE PATH
FR-MSG-08         Hard deletion shall occur ONLY VIA THE COMPLIANCE DELETION ENDPOINT
DR-06             Deleted messages shall RETAIN THEIR ROW

No artifact in the feature noticed until its fifth analysis pass, and the chapter cannot be built under a literal reading of all three.

The decision. A retention sweep is a compliance path. The constitution's own word is path and not endpoint — and the purpose test it implies is met: a retention policy exists because a customer promised an auditor that data would not outlive a period. That is the same kind of obligation FR-MOD-04's erasure endpoint discharges, arriving on a schedule instead of on a request.

Why this reading and not the others. The constitution is the rule hardest to change and the one a feature has no authority to amend; FR-MSG-08 and DR-06 are narrower, are amendable through the SRS's own governance clause, and are the two that used the stricter word. Changing the narrow clauses to match the broad one is the direction that loses least. There are exactly two compliance paths after this and both are named in FR-MSG-08, so the category still refuses everything else.

Rejected alternatives.

• Read constitution II as FR-MOD-04's endpoint and nothing else. Then FR-MOD-06 is unbuildable and should be withdrawn rather than left as a P3 nobody can satisfy. That is a product decision and a feature cannot take it.
• Amend the constitution. Outside a feature's authority, and unnecessary: the principle's text already admits this reading.
• Soft expiry — clear text and attachments, keep the row. This is the one worth naming at length, because it is the trap. It satisfies all three clauses word for word, needs no ADR, needs no migration beyond an UPDATE, and breaks nothing. It is also the only option that leaves a compliance team believing data is gone while the row is still in the table. FR-MOD-06 says hard-deleted, and a tombstone is precisely what expiry is not.

Reversal condition. If a later chapter needs hard deletion on a third path, compliance path has stopped being a category that distinguishes anything, and the clause needs rewriting rather than reinterpreting.

Decision 2 — the exception is a named flag in the trigger

The refusal is a pincer, and it is chapter 4.19's. Exactly one table references messages, and 4.19 began putting a row in it for every deletion as well as every edit — 5,495 messages owned one when this chapter opened. Measured in rolled-back transactions:

delete the message            ERROR  message_edits_message_id_fkey
delete the versions first     ERROR  message versions are append-only (FR-MSG-07)
ON DELETE CASCADE             ERROR  the same trigger, and the error names the
                                     statement the cascade generated:
                                     DELETE FROM ONLY "public"."message_edits"
control: no version rows      DELETE 1

The third line is the one that shaped the mechanism. A cascade is not a privileged path — it issues an ordinary DELETE, and message_edits_append_only is a row trigger, so it fires. The cascade alone deletes nothing and the exception alone leaves the foreign key refusing the parent. Both, or neither.

The decision. 0025 makes the foreign key ON DELETE CASCADE and gives the trigger one condition:

IF TG_OP = 'DELETE' AND current_setting('relay.expiring', true) = 'on'
  THEN RETURN OLD; END IF;

TG_OP = 'DELETE' is part of it because an UPDATE must stay refused with the flag set: expiry destroys a row and never rewrites one, so the immutability of a version's content is not what this chapter gives up. Measured in all three directions rather than the one that passes — UPDATE with the flag refused, DELETE with the flag unset refused, DELETE of the message with the flag set permitted and the child rows going 1 to 0.

Rejected alternatives.

• SET session_replication_role = replica. It works, which is the problem: it is the hole ADR-35 published as the limit of its own guarantee, so using it would make this platform's retention sweep the first caller of a bypass the previous chapter documented as the reason its claim is scoped. It also disables every trigger in the session, which is wider than one table and one verb.
• Drop the foreign key. A version row could then outlive its message, which is what makes the history trustworthy.
• An ordered two-step delete in application code. The invariant a version cannot outlive its message moves from the schema into a procedure somebody maintains — the distinction chapter 4.15 drew when it gave a rendition's reachability to a composite foreign key rather than to a predicate.

What it costs, and it is a published guarantee. ADR-35 scoped the audit log's immutability as to the application and to accident, and not to somebody holding the database password, and chapter 4.19 applied that scope to message_edits. After this chapter the scope for that table is to the application except one named path, and to accident. audit_log's scope is untouched. Constitution VII makes an accepted ADR immutable, so this supersedes ADR-35's scope clause for message_edits rather than editing it.

And the guarantee is one keyword wide. SET LOCAL holds for one transaction. Measured both ways, because both are silent: with plain SET the flag outlives its transaction on a pooled connection and every later request can delete version rows; with SET LOCAL outside a transaction block Postgres emits a warning, the flag is never set, and every cascade is refused in a way indistinguishable from the trigger doing its job. Repository.destroyMessages is the only setter, and retention.itest.ts asserts the flag's value on both sides of a real destroy rather than asserting the statement was issued.

Reversal condition. A separate non-superuser role for the api makes privilege the mechanism and this exception unnecessary — ADR-35's own reversal condition, unchanged by this ADR.

ADR-37 — A key into an erased row is not personal data

Status: accepted (feature 067, chapter 4.21, 2026-10-04)

Context

FR-MOD-04 requires an endpoint that permanently erases all data for an end user, analytical records among them. Seven stores name a user. Two of them name a user in a form from which no single user can be removed:

usage_active_users    (environment_id, period, user_id, first_seen_at)   22,298 rows
daily_usage_*         seven AggregateFunction(uniq, Nullable(UUID))      880 rows

The sketch has no subtract operation and recomputing it needs message_events, which holds 0 rows and has no producer. usage_active_users could be deleted from, and deleting from it would change a figure a customer has already been invoiced for.

The obvious readings both cost something real. Delete the billing rows and an invoice loses its basis. Keep them and FR-MOD-04's analytical records is partly unmet, which has to be written into the SRS as a non-conformance.

Decision

Neither. A record whose only identifier is a key into an erased row names nobody, and counts as erased.

The chapter's other decision makes this available. FR-USR-05 preserves a deleted user's messages, so messages.user_id stays; all five foreign keys to users are NO ACTION; so the row cannot be deleted and erasure leaves it standing and empty — profile cleared, external_id replaced with erased:<users.id>. Four measurements then settle it:

the table's columns     one could name anybody, and it is a uuid
both FKs' delete rule   NO ACTION — the tombstone keeps them satisfied
every reader            a count(*) scoped by (environment_id, period), and a
                        membership probe keyed on the uuid. NEITHER JOINS TO `users`
the count afterwards    unchanged, to the row

The table never produced a name. It produced a number, and the uuid was only ever what made the number distinct.

The line this draws, and the test that it is a real one

Keys against contents. A uuid pointing at an erased row names nobody; that says nothing about the VALUE on the row. messages and media_objects also reference the user by key and both hold contents, so both were decided on their own clauses — and they went opposite ways: the messages stay under FR-USR-05, the attributed media objects are destroyed under FR-MED-10. A rule that decided those two the same way would not be a line, it would be a convenience.

What it costs is one sentence in FR-MOD-04 rather than a recorded non-conformance, and that sentence is the rule: an analytical record whose only identifier is a key into an erased row counts as erased.

Alternatives considered

Delete the usage_active_users rows. Satisfies the clause literally and breaks FR-RTL-05's accounting: a customer who deleted a user in March would stop owing for March, retroactively, because somebody exercised a privacy right. No clause asks for that and the code's own comment argues against it.

Record FR-MOD-04's analytical records as partly unmet by decision. Honest, and the fallback this project has used for FR-MED-07, FR-MED-09's rendering half, FR-MOD-03's year and FR-ANL-06's job. Refused because it is not true: after the erasure nothing in those rows identifies anybody, so recording them as un-erased would understate the platform rather than overstate it — the rarer and more confusing direction.

Anonymise in place with a placeholder user. A synthetic anonymous row per environment, with the billing rows repointed at it. Costs a row nobody created, a primary-key collision the moment two erased users share a period, and a second kind of user every reader has to know about. The tombstone already is the placeholder, there is exactly one per user, and (environment_id, period, user_id) wants exactly that.

Hash the external id onto the tombstone so a second erasure can still find it. Refused for the reason it is tempting: u-4821 and an email address are both brute-forceable, so a hashed identifier is the identity wearing a disguise. The cost of refusing is that a second erasure answers 404 and a support tool retrying after a timeout cannot tell already erased from never existed; the operator's proof is the audit entry instead.

Reversal condition

If users.id ever becomes resolvable to a person by a party outside the platform, this rule fails and every uuid-keyed store is decided on its own merits again.

The live edge this ADR named does not exist (feature 068, chapter 4.22). It said users.schema.ts's listing cursor is base64 of {a: last_activity_at, id: users.id} and that a customer who paged GET /v1/users before an erasure holds the uuid. Three things measured against a running api:

GET /v1/users                            404     there is no such route
listingQuerySchema, consumers            one: GET /v1/users/{externalId}/channels
the cursor it issues, decoded            {"a": <timestamp>, "id": <a CHANNEL uuid>}

The cursor is real and it carries a channel uuid — a value the same response already returns as a top-level field and every channel route accepts. The paragraph was right that opacity is not security and wrong about which id was inside, and it stood for one chapter because every reader checked the comment rather than the route.

What was live instead was a write route. POST /v1/channels/{channelId}/members returned members[].user_id — the row's users.id — on every member added, for every user, erased or not. No route accepts that value: GET /v1/users/{a users.id} answers 404 while GET /v1/users/{external_id} answers 200, so a caller holding it held a key to nothing, and this ADR's opening sentence was false. Nothing asserted the field, nothing documented it and no tutorial page showed it, which is why six analysis passes and a chapter went past it. 4.22 removed it, and the sentence is true now.

AND THE METHOD IS THE PART WORTH KEEPING. The phantom came from reading; the real one came from enumerating every v1 response shape and asking of each value does a route accept this? Three siblings turned up and all three are kept with reasons — audit_log[].id and audit_log[].actor.id are record references a customer quotes back to support, and request_id is constitution V's requirement in every error body. Only members[].user_id had no answer to the question.

The exception that remains is this ADR's own argument working. An erased user's external_id is erased:<users.id>, and that string surfaces wherever an identity does — message.user on 290 retained messages across 367 tombstones, and member.external_id. It is a users.id crossing the boundary and it names nobody: the row it keys is empty, and no route accepts it either, because a second erasure answers 404. The handle survives and points at an empty room — which is what the previous paragraph claimed about the wrong handle. If that ever stops being true, so does this ADR.

And one thing this ADR does not cover. messages.user_id still links a person's retained messages to each other, so "these were written by the same erased person" stays computable. If the text names them, the text is the re-identification and not the key — which is the same position Slack and Microsoft Teams take, and the reason the receipt reports retained_anonymous rather than erased.

ADR-38 — A noun with a customer-supplied identifier is addressed by it

Status: accepted (feature 068, chapter 4.22, 2026-10-06)

Context

FR-USR-01 says "Relay shall not generate end-user identities" and ADR-18 says an end user's identity "is whatever external_id the customer already had for them." Both are about users. Nothing said what to do with the next noun that has one, and the next noun was channels: FR-CHN-01 took a customer-supplied identifier at creation, and all thirteen routes beneath /v1/channels/{channelId} then required a uuid Relay minted. A customer had to store Relay's key to post to a channel they had named themselves — the lookup table FR-CHN-01 exists to remove, and the one docs/03-journey-map.md Stage 1 promises Mai will not need.

Measured, 2026-10-05. Exactly two tables carry a customer-supplied identifier, from information_schema rather than from judgement: users and channels. Users: 8 of 8 routes take the identity, the upsert returns no uuid. Channels: 0 of 13, and GET /v1/channels/order-88412 answered 500 — 'order-88412'::uuid raises in Postgres before the OR beside it can short-circuit, so the natural first attempt told an integrating developer the platform was broken rather than that they had used the wrong key.

Decision

A noun with a customer-supplied identifier is addressed by that identifier. A noun with only a Relay identifier is addressed by that. Where both could name the same thing, the customer's wins.

The four nouns the second half covers have no external_id column, measured: messages, media_objects, webhook_endpoints, environments. A Relay identifier is the only identifier they have, so a uuid on their routes is correct and not an oversight.

What a key is still for

A uuid remains the foreign key, the ordering key and the primary key, and this ADR does not propose otherwise. A text primary key across 216,922 messages and 182,434 memberships costs more than it is worth. There is one identity and one internal key, and only one of them belongs on the wire — which is the sentence the whole decision reduces to.

Keys may also stay ON the wire where a route accepts them: the thirteen channel routes keep taking a uuid, because 173 call sites and every published client hold one, and removing it needs a window, a warning and a version. What a response may not carry is a key no route accepts. members[].user_id was one and is gone: chapter 4.22 swept every v1 response shape and asked of each uuid whether a route accepts it.

Alternatives considered

Resolve in middleware, one registration instead of thirteen. Nest runs middleware before guards, so it has no principal and cannot scope the lookup to an environment — and an unscoped resolution is a cross-tenant read. Disqualifying rather than inconvenient.

Resolve in each service method. Thirteen call sites instead of thirteen decorators, and it spreads the ambiguity into ten repository queries that currently take a key and nothing else.

Key-first on a uuid-shaped value. Simplest to describe, and it strands a customer who names a channel with another channel's uuid: they could never reach their own, with no error they could act on. 0 of 41,772 channels have a uuid-shaped identifier today, so this is a case the chapter constructs rather than waits for.

Retire the uuid. 173 call sites and every published client hold one. A deprecation needs a window, a warning and a version, none of which belongs in a chapter whose subject is making the identity work.

Reversal condition

If a noun ever needs an identifier the customer supplies and cannot be relied on to be unique within a tenant, the first half fails for that noun and it is decided on its own merits. Uniqueness is what makes the identity addressable at all: channels_environment_id_external_id_unique and users_environment_id_external_id_unique are the constraints this ADR rests on.

And what it did not cover, which chapter 4.23 closed. The real-time surface addressed channels by uuid — every gateway frame carried channel: <uuid>, the session response handed a connecting client a list of channel uuids, and a socket send went to a door typed z.string().uuid(). packages/protocol/src/internal.ts stated this ADR's own principle two lines above that field — "user is the EXTERNAL id, as everywhere else on this contract: internal uuids are the api's business" — and applied it to one field of two.

FR-RTM-11 now says which identifier names a channel on the socket, and the translation sits at the gateway's client edge: the session response carries pairs, the connection holds both directions of the map, and one function — send — renames on the way out. The subjects, the resume cursors' internal keying and the api-facing door keep the Relay identifier, which is this ADR's division applied one service over rather than a new one.

TWO THINGS THAT CHAPTER MEASURED AND THIS ADR SHOULD CARRY. The surface was larger than a count of channel fields showed: connection.ack names channels three more times without the word, in revisions and cursor (both keyed by channel) and in truncated (a list of them). And the tie-break this ADR fixed for REST is now load-bearing on the socket too — 19 of 44,574 channels carry an identifier that is itself another channel's Relay identifier (measured 2026-10-08, against 0 of 41,772 four days earlier), so no shape test separates the two forms in a resume cursor and the identifier wins, as it does on a path segment.

AND A THIRD SURFACE IS STILL OPEN. Webhook payloads carry channel_id as a Relay identifier — MessageCreatedData, MessageDeletedData and MembershipChangedData in services/api/src/outbox/event.ts, delivered to the customer's own endpoint — on a boundary whose own comment says "Consumers are customers: they get external ids and the field names the REST surface uses." Recorded rather than closed: it belongs to no chapter yet.