Building Relay

SAD — Software Architecture Document

Relay — Software Architecture Document

Version: 1.5 (draft) Status: For review Companion documents: 01-product-vision.md · 02-personas.md · 03-journey-map.md · 04-srs.md Structure: views-based (C4-influenced), with Architecture Decision Records


Table of contents

  1. Introduction
  2. Architectural drivers
  3. Context view
  4. Service view
  5. Runtime view — key scenarios
  6. Data view
  7. Deployment view
  8. Cross-cutting concerns
  9. Architecture Decision Records
  10. Risks and technical debt register

1. Introduction

1.1 Purpose

This document describes the architecture of Relay: the services, their responsibilities and interactions, the data model, the deployment topology, and — most importantly — the decisions and their rationale. Where the SRS says what the system shall do, this document says how, and defends the choices against alternatives.

1.2 Scope

Covers all four delivery phases of the SRS. Phase-specific elements are marked. The SDK's internal design is summarised only where it constrains the server (protocol semantics); its full design is deferred to an SDK design note.

1.3 How to read this document

Sections 3–7 are descriptive views. Section 9 (ADRs) contains the reasoning; every non-obvious choice in the views links to an ADR. If reviewing, read §2 and §9 first — the rest follows from them.

1.4 Conventions

Requirement references use SRS identifiers (FR-MSG-04, NFR-REL-02). Decisions are numbered ADR-nn and are immutable once accepted; superseding requires a new ADR.


2. Architectural drivers

The handful of requirements that actually shape the architecture. Everything else is implementation.

#DriverSourceArchitectural consequence
D1No acknowledged message may be lostFR-MSG-05/06, NFR-REL-02Ack only after durable commit; ordering and persistence in one transactional store
D2Correct delivery across gateway instances, no sticky sessionsFR-RTM-02, CON-02, NFR-SCL-02A pub/sub fabric between gateways; connection registry outside instance memory
D3Reconnect must resume exactly — no loss, no duplicatesFR-RTM-03, FR-MSG-04, Tuan's journeyServer-assigned per-channel sequences; idempotency keys enforced at the storage layer
D4Tenant isolation is a correctness propertyFR-TEN-05/06, NFR-SEC-09Tenant ID threaded through every layer; enforced in data access, not in handlers
D5Analytics must never touch the operational pathCON-01, FR-ANL-02/03, NFR-REL-05Fire-and-forget event emission to a durable queue; ClickHouse strictly downstream
D6Webhooks must not block deliveryFR-WHK-05Webhook dispatch consumes from the same queue, fully asynchronous
D710 min to first messageNFR-USE-01, journey Stage 4Dev-mode token endpoint; zero-config defaults; dashboard live stream fed from the event queue
D8One engineer must be able to run and reason about itPortfolio reality; NFR-MNT-03Few services with sharp boundaries, not many with fuzzy ones; boring technology

D8 deserves emphasis. The correct number of services is the smallest number that still demonstrates real distributed-systems boundaries. Resume-driven microservice sprawl — fifteen services where five suffice — is itself an architectural smell, and reviewers know it.


3. Context view

flowchart TB
    endusers["Customer's end users<br/>(browser / RN app — Tuan)"]
    backend["Customer's backend<br/>(mints user JWTs, admin API calls)"]
    webhook["Customer's webhook endpoint"]
    dashuser["Dashboard users<br/>(Mai, David — browser)"]

    subgraph relay [RELAY]
        rest["public REST /v1"]
        ws["WebSocket /ws"]
        dash["Relay dashboard (web app)"]
    end

    endusers -- "user token · REST + WS" --> rest
    endusers <-- "events, messages (WebSocket)" --> ws
    backend -- "API key (server-to-server)" --> rest
    relay -- "signed webhooks" --> webhook
    dashuser -- "OAuth session" --> dash

Three trust domains, three credential types:

CallerCredentialMay do
Customer backendAPI key (rk_live_…)Everything within its environment
End-user clientUser JWT (customer-signed)Act as one user: send, read own channels, presence
Dashboard userSession (OAuth)Manage org, view analytics; no message-send path

The customer's backend is the trust anchor (SRS §2.1): Relay verifies JWTs with the environment's signing secret but never authenticates end users itself (CON-06, ASM-01).


4. Service view

Six deployable services, one queue, three datastores. Phase in brackets.

flowchart TB
    edge["EDGE / INGRESS<br/>TLS, routing, IP rate limit"]

    api["API service [1]<br/>REST /v1: tenancy, channels,<br/>messages, history, keys,<br/>moderation, emoji packs"]
    gw["Gateway service [1]<br/>WS termination, fan-out,<br/>presence, backfill, typing"]

    pg[("PostgreSQL<br/>(OLTP)")]
    redis[("Redis<br/>pub/sub · conn registry ·<br/>presence · rate limits")]
    nats[["NATS JetStream<br/>subjects: events.>"]]
    ch[("ClickHouse<br/>(OLAP)")]

    ingester["Analytics ingester [3]<br/>batch insert to ClickHouse"]
    whd["Webhook dispatcher [2]<br/>retry, DLQ, HMAC signing"]
    sse["Dashboard live-stream [2]<br/>SSE relay"]
    mediaw["Media worker [3]<br/>scan, probe, thumbnail<br/>(ADR-14)"]
    obj[("Object storage<br/>S3-compatible — media bytes<br/>(ADR-13)")]
    dashapp["Dashboard app [2,3]<br/>Next.js — public API + SSE"]
    cust["Customer HTTPS endpoints"]

    edge -- REST --> api
    edge -- WebSocket --> gw
    gw -- "internal HTTP:<br/>writes + backfill (ADR-04/05)" --> api
    api -- "reads / writes + outbox" --> pg
    api -- "publish fan-out" --> redis
    api -. "presigned upload/download<br/>URLs (metadata only)" .-> obj
    gw <-- "subscribe chan:{id} ·<br/>conn registry, presence" --> redis
    pg -- "outbox relay (ADR-06)" --> nats
    nats --> ingester
    nats --> whd
    nats --> sse
    nats -- "media.uploaded" --> mediaw
    mediaw -- "scan/probe bytes,<br/>write derived objects" --> obj
    mediaw -- "status transitions via<br/>internal API (ADR-04)" --> api
    ingester --> ch
    whd --> cust
    sse --> dashapp
    dashapp -- "public API" --> api

4.1 Service responsibilities

API service [Phase 1] Owns all REST semantics: tenancy, auth verification, channels, membership, message write path, history reads, moderation, keys, and — from Phase 3 — emoji pack management, the user-facing pack catalog (browse/search/install), and resolution-map assembly on message reads (→ ADR-11, ADR-12). Stateless. The only service that writes to PostgreSQL — a deliberate single-writer discipline that keeps invariants (sequence assignment, idempotency, isolation) in one codebase (→ ADR-04). Emits an event to the queue after every state change.

Gateway service [Phase 1] Terminates WebSockets. Validates the JWT on connect, claims one of five slot keys in Redis (conn:{env}:{user}:{slot}, TTL-refreshed while the connection lives) and refuses a sixth connection with close 4004 (FR-RTM-09, → ADR-23), subscribes to the Redis pub/sub subjects for the user's channels, pushes frames. Handles resume: on connect-with-cursor it reads backfill through the API service's internal history endpoint, not from Postgres directly (single-writer/single-reader discipline, → ADR-04). Sends message frames received from clients to the API service over internal HTTP — the gateway never writes to the database (→ ADR-05).

Meters connection-minutes and reports them to the API service every 60 seconds over POST /internal/usage/connections, for FR-RTL-05's monthly quota (chapter 3.24) — a synchronous counter that must be able to refuse a connection, which is why it cannot live downstream of a lossy stream. Records every connection open and close as an analytical event (FR-ANL-01), buffered and published to ANALYTICS on a five-second tick (chapter 4.5) — a second counter of the same quantity, kept apart on purpose, with the reconciler in movement IV. That is the gateway's first broker client and its sixth dependency; see ADR-07's third amendment for what it costs that record.

Webhook dispatcher [Phase 2] Consumes events.> from JetStream with a durable consumer per environment shard. Filters by endpoint subscriptions, signs (HMAC, FR-WHK-02), posts, retries on the FR-WHK-03 schedule using JetStream redelivery + a scheduled-retry stream, dead-letters after six attempts. Records every attempt as an analytical event (FR-WHK-06).

Analytics ingester [Phase 3] Consumes everything, buffers, batch-inserts to ClickHouse every 2 s or 10k rows (DR-11). Deliberately dumb: no transformation beyond shaping, no business logic. If ClickHouse is down it stops consuming and the stream absorbs the backlog (NFR-REL-05). How long it absorbs is a function of the record rate, not a constant, and the stream is configured at seven days and 1 GiB rather than at 24 h. Measured at 320 bytes a record: 1 GiB holds 3,355,443 of them, so seven days fits only up to 5.5 records/second and 24 hours only up to 38.8. Above those rates max_bytes binds before max_age does.

Media worker [Phase 3] Consumes media.uploaded events. Fetches the object, verifies size/type against the declaration (FR-MED-03), virus-scans (ClamAV sidecar), probes dimensions/duration, generates thumbnails and poster frames (FR-MED-05), writes derived objects, then transitions pending → ready | rejected via an internal API endpoint — never touching Postgres directly, per ADR-04. The only Relay component that ever reads media bytes, and it does so off the request path entirely (→ ADR-14). Scales on JetStream consumer lag; CPU-bound (scanning, image ops), so it is the one service where ADR-01's worker-thread posture matters from day one.

Dashboard live-stream service [Phase 2] Thin SSE bridge: subscribes to a tenant's events on the queue, relays to the dashboard browser session. Exists so the dashboard's live view (FR-DSH-02, driver D7) needs no WebSocket infrastructure of its own and no polling.

Dashboard app [Phases 2–3] Next.js. Consumes the public API (EIR-DSH-02) plus the SSE stream and internal billing endpoints. Server-side sessions via OAuth (FR-TEN-01).

4.2 What is deliberately not a separate service

CandidateWhy it stays mergedRevisit when
"User service", "Channel service"Same datastore, same transactions, same team. Splitting would turn local transactions into distributed ones for zero scaling benefit — users and channels do not scale independently of messages.Never, realistically
"Auth service"JWT verification is a library concern (a middleware verifying HS256 with the env secret). A network hop per request to verify a token is pure latency.If asymmetric multi-issuer auth arrives
"Presence service"Presence is connection state, which lives in the gateway + Redis already (ADR-10, ADR-19).If presence fan-out dominates gateway CPU — ADR-19's revisit trigger, which is undischarged
Rate limiterRedis token buckets called in-process from API and gateway.If limits need to be enforced at edge before TLS termination

This table is the answer to "why only six services?" — each merge is a decision with a stated reversal condition, which is more defensible than either extreme.

And it has answered a candidate the other way once. Chapter 4.13's media worker fails all three merge criteria — a different datastore, no shared transactions, CPU-bound work off the request path — so it ships as a service rather than as a module. ADR-31 is that argument, written against this table's own columns and carrying its own reversal condition, which is what makes the table load-bearing rather than decorative: a row that can only ever say "stays merged" is not a test anything could fail.


5. Runtime view

Three scenarios, chosen because they are the journeys: Tuan's send-through-a-tunnel (Phase 1 exit criterion), the cross-instance delivery that makes it work, and Priya's moderation action.

5.1 Message send with idempotency (D1, D3)

sequenceDiagram
    participant C as Client
    participant G as Gateway
    participant A as API service
    participant P as PostgreSQL
    participant J as JetStream

    C->>G: frame message.send {idem_key, channel, text}
    G->>A: POST /internal/messages
    A->>P: BEGIN
    A->>P: SELECT channel FOR UPDATE
    Note over A,P: seq = last_sequence + 1
    A->>P: INSERT message ON CONFLICT (channel, idem_key) DO NOTHING
    alt idempotency conflict
        A->>P: SELECT existing row
        Note over A: return original message (200)
    end
    A->>P: INSERT outbox event
    A->>P: COMMIT
    A-->>G: 201 {message, seq}
    G-->>C: frame message.ack {seq}
    G->>G: publish to Redis chan:{channel_id}
    Note over G: all gateway instances fan out to members
    P->>J: outbox relay drains event (after commit)

A message a customer's backend sends takes the same write path and a different ordering (chapter 3.18):

sequenceDiagram
    participant B as Customer backend
    participant A as API service
    participant P as PostgreSQL
    participant R as Redis
    B->>A: POST /v1/channels/:id/messages
    A->>P: INSERT message (seq under row lock)
    A->>P: INSERT outbox event
    A->>P: COMMIT
    A->>R: publish to chan:{channel_id}
    A-->>B: 201 {message, seq, user}
    Note over R: every gateway instance holding a member delivers

Decisions visible here:

  • Ack after commit, never before (FR-MSG-05). The ordering of the fan-out relative to the ack depends on the transport, and only durability is invariant. On the socket path the gateway writes the ack frame and then publishes, because it holds two channels. On the REST path the response is the ack, so a handler cannot publish after it without detaching the failure from anywhere a test or an operator can see it — the api publishes before responding. A recipient may therefore see a REST-sent message slightly before the sender's 201, and can never see any message before it is durable, which is what this clause protects.

    One measurement follows and is worth stating: NFR-PRF-01's interval — "send acknowledged to recipient receipt" — is not measurable on the REST path, because it can be negative there. It remains measurable on the socket path. The publish instead falls inside NFR-PRF-02's budget, where it was measured at p95 0.226 ms against 150 ms.

  • Idempotency at the storage layer via partial unique index (DR-03), not application memory — it survives restarts and works across instances (FR-MSG-04).

  • Sequence assignment under row lock on the channel (DR-04, → ADR-03). Contention scope is one channel; busy channels serialise their own sends, which is precisely the ordering guarantee FR-MSG-03 requires.

  • Event publication via transactional outbox (→ ADR-06): the event row commits with the message; a relay drains the outbox to JetStream. Publish-after-commit without the outbox would silently drop events on a crash in the gap — and metering (FR-ANL-06) would drift.

5.2 Cross-instance delivery and Tuan's resume (D2, D3)

Normal delivery: gateway instances all subscribe to Redis pub/sub subject chan:{channel_id} for channels their connected users belong to. The publishing side is instance-agnostic — no registry lookup on the hot path, no sticky routing (FR-RTM-02).

Resume, per Tuan's journey Stage 3:

sequenceDiagram
    participant T as Client (Tuan)
    participant G as Gateway
    participant A as API service

    T->>G: WS connect {token, cursor: ch1=412, ch2=87}
    G->>G: verify JWT · register conn in Redis
    G->>G: subscribe Redis subjects FIRST,<br/>buffer incoming live frames
    G->>A: GET /internal/backfill {user, cursors}
    A-->>G: messages where seq > cursor,<br/>per channel, cap 500
    G-->>T: connection.ack {resume_ok, revisions}
    G-->>T: backfilled frames, sequence order per channel
    G->>G: flush buffer, discard seq ≤ backfill high-water mark
    G-->>T: live frames resume

The subtle bug this design closes: subscribe-then-backfill can deliver a live frame that is also in the backfill (duplicate); backfill-then-subscribe can drop a message that lands in the gap. The gateway subscribes first, buffers live frames, serves backfill, then flushes the buffer discarding anything with seq ≤ the backfill's high-water mark. Sequence numbers make the deduplication trivial — which is a large part of why they exist (→ ADR-03). Backfill beyond 500 messages per channel returns truncated: true and the client refetches history instead (FR-RTM-04).

The one thing a cursor cannot recover, and the field that reports it. Resume is ordered by the channel sequence, and an edit or a deletion carries the sequence of the message it changes rather than a new one. So a message revised below a client's cursor appears in neither the backfill nor the live stream, and consumes no sequence — no gap appears for the deduplication above to notice, because there is nothing out of order. Gap detection is the mechanism this document names as the recovery, and here it sees nothing to recover.

The ack therefore carries revisions: a per-channel count of how many revisions that channel's messages have received, for every channel the user belongs to.

connection.ack.payload.revisions = { "<channel_id>": 7, … }

A client compares each count with the one it stored at its last connection, and there are three outcomes:

ComparisonWhat it meansWhat the client does
reported > storedone or more messages it already holds were edited or deleted while it was awayre-read that channel's history (FR-MSG-09), which returns current state — the corrected text for an edit, a null text for a deletion
reported = storednothing was missednothing
reported < storedits stored number is stale, restored from a backup, or wrongnothing — treat it as nothing missed, and store the reported count

Store the reported counts in every case, including the ones that needed no repair. A client that only stores counts when it repairs never establishes a baseline for the channels it did not repair, and its next reconnect is the first one again.

The difference counts revisions, not messages. Three revisions may be three edits of one message or one edit each of three, so the number bounds how much changed and does not identify what: the repair is a history re-read for the channel, not a fetch of three messages.

The platform reports and never compares — it is not told what a client holds, so a stale or invented client-side number cannot make it refuse a connection or send anybody on a repair. That is why the third row above is the client's rule rather than the server's: refusing a connection over a number the client supplied would be a denial of service the client controls.

A client holding no count for a channel — a first connection, a client built before the field existed, or a channel joined during the absence — repairs nothing and stores what it was told. It holds nothing in that channel that a count could show to be stale, and its messages arrive by the ordinary backfill above.

Counting revisions and not messages. A send does not raise the count: a new message is delivered by the ordinary backfill above, and counting sends here would make every active channel report a repair after every absence — the same herd this design avoids, arriving from the other direction.

Why this is documented here. There is no public protocol reference in these repositories to put it in; the implementation review lists one as absent and it belongs to Part 4. §5.2 is the runtime view that already carries connection.ack in its sequence diagram, so it is the nearest published home rather than the obvious one, and that choice is recorded rather than left for a reader to reconstruct.

5.3 Priya's moderation delete (journey 3, stage 5)

sequenceDiagram
    participant S as Support tool
    participant A as API service
    participant P as PostgreSQL
    participant J as JetStream
    participant G as Gateways
    participant W as Webhook dispatcher
    participant I as Analytics ingester

    S->>A: DELETE /v1/channels/{channelId}/messages/{messageId}<br/>(API key, or the author's token)
    A->>A: verify key scope
    A->>P: UPDATE message SET text=NULL, attachments=NULL,<br/>deleted_at=now(),<br/>metadata.deleted_by (FR-MSG-08, FR-006a of 3.23)
    A->>P: INSERT outbox event (same transaction)
    A->>P: INSERT message_edits (the final text, 4.19)
    A-->>S: 204
    P->>J: outbox relay drains event
    J->>G: message.deleted → push to connected members (FR-RTM-05)
    J->>W: message.deleted → customer webhook
    J->>I: analytical event

One write path serves four consumers — Priya's real-time removal, the audit trail (FR-MOD-03), the customer's webhook, and metering — without any of them coupling to the others. This scenario is the clearest illustration of why the outbox/queue spine exists.

Amended 2026-09-03 (chapter 3.23), and three things in the diagram above were wrong. The route is on the channel's message resource, not /v1/messages/{id}, and it accepts the message's author's token as well as a tenant API key (FR-MOD-02 grants the key deletion of any message; FR-013 of chapter 3.23 grants the author their own).

There was no audit_log table when this diagram was drawn, so the second INSERT in the original wrote to something that did not exist and the paragraph above counted a consumer that was not built. What the deletion recorded instead is metadata.deleted_by on the message row — the actor's KIND, and their external id when there is one — which is FR-MSG-08's "deletion metadata" and not FR-MOD-03's log. The distinction is real and narrow: a single mutable column on the row it describes carries no request id, cannot be appended to, and says nothing about moderation actions that leave no row.

Amended again 2026-10-02 (chapter 4.18): the table exists. 0021_audit_log.sql creates it and a BEFORE UPDATE OR DELETE trigger makes it append-only, and a deletion by an application credential now writes an entry inside the same transaction as the tombstone (ADR-35, §6.1). The sentence above was true of the schema for eight chapters and the SRS's own §6.1 drew AuditLogEntry throughout — the data model was ahead of the schema and this document recorded only the absence, which is what a reader checking whether the table was intended would have found first.

metadata.deleted_by stays and is still not the log. It is written on the row the deletion changes and the audit entry is written beside it; the paragraph above is the reason both exist rather than one. Note also that metadata is not published on any read path — the API answers metadata: null — so the fragment was invisible to the tenant whose log it would have been.

And the entry is written only when an application credential deletes. A user deleting their own message is chapter 3.23's FR-013, not FR-MOD-02, and a compliance log that recorded it would fill with ordinary user activity: DELETE /v1/channels/:channelId/messages/:messageId is the one route in the moderation set whose classification the credential decides.

And the deletion is idempotent (FR-009 of chapter 3.23): the second DELETE answers 204, changes nothing — deleted_at in particular does not move — and emits no second event, so nothing downstream in this diagram fires twice for one deletion.

Amended again 2026-10-03 (chapter 4.19): the transaction has a third write. The text the message held at the moment of removal goes into message_edits, because until then an edit recorded the text it replaced and a deletion recorded nothing — so FR-MOD-01's "complete history" lost exactly one version per deletion, which FR-MSG-08 reserves for the compliance endpoint. The row is written on the branch that actually deletes, so a repeated DELETE records no second version for the same reason it emits no second event. It costs 0.331 ms at p50 on a 3.2 ms deletion, 200 samples a side interleaved — against chapter 4.18's audit entry at 0.43 ms, which cost 94.6% because four actions had to GAIN a transaction and this one already had hers.

And the paragraph above still counts right. One write path, four consumers: the version row is a fourth WRITE and not a fifth consumer — nothing subscribes to it, and GET …/messages/{id}/edits reads it on demand.


6. Data view

6.1 PostgreSQL — operational schema (core tables)

CREATE TABLE environments (
    id              UUID PRIMARY KEY,
    application_id  UUID NOT NULL REFERENCES applications(id),
    kind            TEXT NOT NULL CHECK (kind IN ('development','production')),
    signing_secret  TEXT NOT NULL,          -- envelope-encrypted (NFR-SEC-02)
    retention_days  INT
        CHECK (retention_days IS NULL OR retention_days IN (30, 90, 365)),
        -- FR-MOD-06, bounded in 4.20. The clause ENUMERATES four options rather
        -- than describing a range, so 45 is not a stricter policy somebody chose.
        -- NULL is indefinite. Declared in 2.1, read by nothing for seventeen
        -- chapters, set on 0 of 33,051 rows when the sweep was finally written.
    quota_config    JSONB NOT NULL DEFAULT '{}'
);

CREATE TABLE users (
    id              UUID PRIMARY KEY,
    environment_id  UUID NOT NULL REFERENCES environments(id),
    external_id     TEXT NOT NULL,
    display_name    TEXT,
    avatar_url      TEXT,
    metadata        JSONB NOT NULL DEFAULT '{}',
    banned_at       TIMESTAMPTZ,
    deleted_at      TIMESTAMPTZ,                          -- FR-USR-05: the row survives
    kind            TEXT NOT NULL DEFAULT 'person'
                    CHECK (kind IN ('person','bot')),     -- FR-USR-07 (chapter 3.17)
    description     TEXT,                                 -- what the software is, and why
    UNIQUE (environment_id, external_id),                 -- DR-02, and FR-CHN-11:
                                                          -- uniqueness per tenant is what
                                                          -- lets a route take the identity
    CHECK (kind <> 'bot' OR description IS NOT NULL)      -- a bot without one is not a bot
);

CREATE TABLE channels (
    id              UUID PRIMARY KEY,
    environment_id  UUID NOT NULL REFERENCES environments(id),
    external_id     TEXT NOT NULL,
    type            TEXT NOT NULL CHECK (type IN ('public','private')),
    name            TEXT,
    metadata        JSONB NOT NULL DEFAULT '{}',
    last_sequence   BIGINT NOT NULL DEFAULT 0,             -- ADR-03
    archived_at     TIMESTAMPTZ,
    last_activity_at TIMESTAMPTZ NOT NULL DEFAULT now(),   -- FR-CHN-08's ordering
    UNIQUE (environment_id, external_id)                   -- DR-02, and FR-CHN-11:
);                                                         -- the constraint that makes
                                                           -- the identity ADDRESSABLE

-- FR-CHN-09's unread count. Per user, per channel, the sequence up to which that
-- user has read — and no counter column: unread is
-- `greatest(channels.last_sequence - sequence, 0)`, because the write path already
-- maintains `last_sequence` (ADR-03).
--
-- `environment_id` is denormalised — `channel_id` determines it — so feature 030's
-- global-operation guard can watch the table. `members` carries none and is
-- classified as a foreign-key hop instead.
CREATE TABLE read_positions (
    environment_id  UUID NOT NULL REFERENCES environments(id),
    channel_id      UUID NOT NULL REFERENCES channels(id),
    user_id         UUID NOT NULL REFERENCES users(id),
    sequence        BIGINT NOT NULL,                       -- forwards only; <= last_sequence
    updated_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
    PRIMARY KEY (channel_id, user_id)                      -- no `id` column
);

CREATE TABLE messages (
    id              UUID PRIMARY KEY,
    channel_id      UUID NOT NULL REFERENCES channels(id),
    sequence        BIGINT NOT NULL,
    user_id         UUID REFERENCES users(id),
    text            TEXT,                                   -- NULL ⇒ tombstone
    metadata        JSONB NOT NULL DEFAULT '{}',            -- see below
    attachments     JSONB,
    idempotency_key TEXT,
    created_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
    edited_at       TIMESTAMPTZ,
    deleted_at      TIMESTAMPTZ,
    UNIQUE (channel_id, sequence)                           -- DR-01
);
CREATE UNIQUE INDEX messages_idem
    ON messages (channel_id, idempotency_key)
    WHERE idempotency_key IS NOT NULL;                      -- DR-03

CREATE TABLE message_edits (                                -- built in 3.23
    message_id  UUID NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
                                                            -- 4.20: a version row
                                                            -- must not outlive its
                                                            -- message, and the cascade
                                                            -- alone changes NOTHING
    edited_at   TIMESTAMPTZ NOT NULL,
    prior_text  TEXT NOT NULL,                              -- FR-MSG-07
    ended_by    TEXT NOT NULL,                              -- 4.19: 'edit' | 'deletion'
    PRIMARY KEY (message_id, edited_at),
    CONSTRAINT message_edits_ended_by_check
        CHECK (ended_by IN ('edit', 'deletion'))
);
-- APPEND-ONLY SINCE CHAPTER 4.19 (FR-MSG-07's `immutable`, ADR-35's mechanism applied
-- to a second table). `REVOKE` is inert because the api connects as a superuser, so a
-- `BEFORE UPDATE OR DELETE` trigger refuses both verbs. Two bypasses remain and are
-- published rather than omitted: `SET session_replication_role = replica` and
-- `DROP TRIGGER`.
--
-- AND ONE NAMED EXCEPTION SINCE CHAPTER 4.20 (ADR-36). The function returns OLD when
-- `TG_OP = 'DELETE' AND current_setting('relay.expiring', true) = 'on'`, which the
-- retention sweep sets with `SET LOCAL` inside the transaction that deletes. The
-- cascade above needs it: a cascade issues an ordinary DELETE and a row trigger fires
-- on it, so `ON DELETE CASCADE` was refused by this very trigger until the exception
-- existed. An UPDATE stays refused with the flag set.
CREATE TRIGGER message_edits_append_only
  BEFORE UPDATE OR DELETE ON message_edits
  FOR EACH ROW EXECUTE FUNCTION message_edits_refuse_write();

CREATE TABLE outbox (
    id          BIGSERIAL PRIMARY KEY,
    subject     TEXT NOT NULL,             -- e.g. events.msg.created.{env}
    payload     JSONB NOT NULL,
    created_at  TIMESTAMPTZ NOT NULL DEFAULT now(),
    published_at TIMESTAMPTZ                                -- ADR-06
);

CREATE TABLE emoji_packs (
    id              UUID PRIMARY KEY,
    environment_id  UUID NOT NULL REFERENCES environments(id),
    external_id     TEXT NOT NULL,
    name            TEXT NOT NULL,
    description     TEXT,
    cover_url       TEXT,
    visibility      TEXT NOT NULL DEFAULT 'listed'
                    CHECK (visibility IN ('listed','unlisted')),
    emoji_version   BIGINT NOT NULL DEFAULT 0,              -- DR-13, ADR-12
    deleted_at      TIMESTAMPTZ,
    UNIQUE (environment_id, external_id)
);

CREATE TABLE emojis (
    id          UUID PRIMARY KEY,
    pack_id     UUID NOT NULL REFERENCES emoji_packs(id),
    environment_id UUID NOT NULL,          -- denormalised for the unique index
    shortcode   TEXT NOT NULL CHECK (shortcode ~ '^[a-z0-9_]{2,64}$'),
    image_url   TEXT NOT NULL,
    tags        TEXT[] NOT NULL DEFAULT '{}',
    deleted_at  TIMESTAMPTZ
);
CREATE UNIQUE INDEX emojis_shortcode
    ON emojis (environment_id, shortcode)
    WHERE deleted_at IS NULL;                               -- DR-12

CREATE TABLE user_emoji_packs (
    user_id      UUID NOT NULL REFERENCES users(id),
    pack_id      UUID NOT NULL REFERENCES emoji_packs(id),
    installed_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    PRIMARY KEY (user_id, pack_id)                          -- FR-EMJ-07
);

CREATE TABLE media_objects (
    id               UUID PRIMARY KEY,                      -- doubles as object key suffix (DR-15)
    environment_id   UUID NOT NULL REFERENCES environments(id),
    uploader_user_id UUID REFERENCES users(id),
    kind             TEXT NOT NULL CHECK (kind IN ('image','audio','video')),
    mime             TEXT NOT NULL,
    declared_bytes   BIGINT NOT NULL,
    actual_bytes     BIGINT,
    status           TEXT NOT NULL DEFAULT 'pending'
                     CHECK (status IN ('pending','ready','rejected')),
    probe            JSONB,                                 -- dims / duration
    created_at       TIMESTAMPTZ NOT NULL DEFAULT now(),
    deleted_at       TIMESTAMPTZ
);
CREATE INDEX media_unreferenced
    ON media_objects (created_at)
    WHERE status = 'pending';                               -- the 24 h reaper's scan (FR-MED-10)

-- FR-MOD-03 (chapter 4.18, ADR-35). The SRS's §6.1 has drawn `AuditLogEntry` since before
-- Part 4; this is the table.
CREATE TABLE audit_log (
    id              UUID PRIMARY KEY,
    environment_id  UUID NOT NULL REFERENCES environments(id),   -- no ON DELETE: the
                                                                 -- convention, and it
                                                                 -- refuses to orphan
    occurred_at     TIMESTAMPTZ(3) NOT NULL,                     -- (3) decides whether the
                                                                 -- cursor loses rows
    actor_kind      TEXT NOT NULL CHECK (actor_kind IN ('application','user','platform')),
    actor_id        TEXT,                                        -- NULL for a platform
                                                                 -- principal: no tenant,
                                                                 -- no readable identifier
    action          TEXT NOT NULL,                               -- `METHOD /path`, the
                                                                 -- route's own key
    target_kind     TEXT NOT NULL
                    CHECK (target_kind IN ('user','message','membership','channel')),
    target_id       TEXT NOT NULL,                               -- what a CUSTOMER calls it
    request_id      UUID NOT NULL                                -- the join to the request log
);
CREATE INDEX audit_log_read_idx
    ON audit_log (environment_id, occurred_at DESC, id DESC);    -- the read's only path;
                                                                 -- `id` is the cursor's
                                                                 -- tiebreaker and is not
                                                                 -- optional

-- WHAT MAKES IT IMMUTABLE, AND WHAT THAT DOES NOT COVER. `REVOKE UPDATE, DELETE` was
-- measured inert: the api connects as a superuser. The trigger fires for one.
CREATE FUNCTION audit_log_refuse_write() RETURNS trigger AS $$
BEGIN
    RAISE EXCEPTION 'audit entries are append-only (FR-MOD-03)'
      USING ERRCODE = 'restrict_violation';
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER audit_log_append_only
    BEFORE UPDATE OR DELETE ON audit_log
    FOR EACH ROW EXECUTE FUNCTION audit_log_refuse_write();

What the trigger enforces, and what it does not (ADR-35). It refuses UPDATE and DELETE from the application and from accident, including from a superuser, which is the whole reason it is a trigger rather than a grant. It does not survive SET session_replication_role = replica, which is one statement and disables every trigger in the session, nor DROP TRIGGER. Both were measured. So the log is immutable to the application and not to somebody holding the database password — and NFR-SEC-10 asks for an immutable trail for exactly that actor, so that clause is not satisfied by this table. A separate, non-superuser role for the application is the one change that would move both; it is ADR-35's reversal condition.

And it guards two tables since chapter 4.19, with the decision unchanged. message_edits carries the same BEFORE UPDATE OR DELETE trigger, for FR-MSG-07's immutable — a word that had been a description since 3.23. Both bypasses were re-measured on that table rather than assumed to transfer, and both still succeed. The argument, with its measurements and its reversal condition, is in docs/06-adr-deep-dives.md; this is the summary and that is the ADR.

THE SECOND TABLE'S SCOPE NARROWED ONCE, IN CHAPTER 4.20, AND THE FIRST'S DID NOT. FR-MOD-06 requires expired messages to be hard-deleted and the trigger refused it — not only directly but through ON DELETE CASCADE, because a cascade issues an ordinary DELETE that a row trigger fires on. So message_edits now carries one named exception (ADR-36): TG_OP = 'DELETE' with a SET LOCAL flag the retention sweep is the only setter of. The scope for that table is immutable to the application except one named path, and to accident; audit_log's is unchanged. ADR-36 supersedes ADR-35's scope clause for message_edits rather than editing it, because constitution VII makes an accepted ADR immutable.

And nothing prunes it. FR-MOD-03 says one year; this platform has no scheduler (ADR-28), so the retention is unmet by decision and the read publishes no retention_edge — announcing a boundary nothing enforces is worse than announcing none.

AND THE LOG IS WHAT COMPLIANCE ERASURE CANNOT REACH (chapter 4.21). audit_log.target_id holds the user's EXTERNAL id — measured at 1,357 of 1,357 user-target rows, not one of them a uuid — so FR-MOD-04's erasure, which destroys every other trace of a person, writes one more entry naming them. That is deliberate: the entry is the operator's only proof the erasure happened, and FR-MOD-03's log exists to demonstrate exactly that. The one place the name survives is the record that the name was erased. The receipt reports it as cannot_erase rather than omitting it, and narrowing ADR-35 a second time was refused — a guarantee with two exceptions three chapters apart is a list, not a guarantee (gaps.md 067-1).

ADR-37 — A key into an erased row is not personal data

Summary. users.id is an internal uuid that the platform exposes nowhere a caller can act on — true since chapter 4.22 and not before it, see the reversal condition below. When FR-MOD-04's erasure empties a user's row — profile cleared, external_id replaced with erased:<users.id> — every store that references that user by key alone stops naming anybody, without being touched. So usage_active_users keeps all its rows with its count(*) unchanged to the row, and the seven AggregateFunction(uniq, Nullable(UUID)) sketch columns need no subtract operation, which is as well because none exists.

The line is keys against contents, and it is testable rather than rhetorical: messages and media_objects also reference the user by key, hold CONTENTS, and were decided on their own clauses — and they went opposite ways. A photo is personal data whatever key it hangs from.

Why the tombstone is available at all. FR-USR-05 preserves the messages, so messages.user_id stays, so all five NO ACTION foreign keys make the row undeletable. The constraint and the decision are the same fact.

Reversal condition. If users.id ever becomes resolvable to a person by a party outside the platform, this rule fails and every uuid-keyed store is decided on its own merits again.

THE LIVE EDGE THIS ADR NAMED DOES NOT EXIST, AND A REAL ONE DID (feature 068, chapter 4.22). It said the one bounded exception was "the GET /v1/users listing cursor". There is no such route. listingQuerySchema has exactly one consumer, GET /v1/users/{externalId}/channels, and the cursor it issues decodes to {"a": <timestamp>, "id": <a CHANNEL uuid>} — a value the same response already returns as a top-level field and every channel route accepts. Measured against a running api, not read.

What was live instead was POST /v1/channels/{channelId}/members, which returned members[].user_id — the row's users.id — on every member added, for every user. No route accepts that value: GET /v1/users/{a users.id} answers 404 while GET /v1/users/{external_id} answers 200. So the summary sentence above was false on a write route everyone uses, and it was found by sweeping every v1 response shape rather than by reading, because reading is what produced the phantom. Chapter 4.22 removed the field, which is what makes the first paragraph true.

The one exception that remains is this ADR's own argument working. An erased user's external_id is erased:<users.id>, and that string surfaces wherever an identity does — in message.user on 290 retained messages across 367 tombstones, and in member.external_id. It is a users.id crossing the boundary, and it names nobody: the row it keys is empty, and no route accepts it either, since a second erasure answers 404. An exception a reader can check in one query beats a claim of none.

The argument is in docs/06-adr-deep-dives.md; this is the summary and that is the ADR.

Hot-path indexes: history pagination (FR-MSG-09) is a pure index-order scan over (channel_id, sequence) — the composite index's leftmost-prefix behaviour is exactly what cursor pagination wants. That ordering is already supplied by DR-01's UNIQUE (channel_id, sequence), which the planner walks backward for newest-first pages; a separate DESC index is therefore not created. members (user_id, channel_id) serves the resume path's "which channels am I in".

Amended 2026-08-02 (v1.1). This section previously specified a dedicated messages (channel_id, sequence DESC) index. Measured against 50,000 rows, the planner ignored it in favour of a backward scan of DR-01's unique index, and dropping it changed neither the plan nor the cost estimate (0.41..5.04 either way). A btree is bidirectional, so a DESC twin of an existing ASC index adds no ordering — it only adds write amplification on the send path and storage. Mixed-direction multi-column ordering would justify one; this query orders by a single column after an equality predicate, so it never can. The index was created by the schema chapter and removed by a forward-only migration in the pagination chapter, where the evidence appeared.

Isolation enforcement (D4): every query goes through a repository layer whose constructors require an environment_id; raw connection access is lint-forbidden outside that layer. The cross-tenant test suite (NFR-SEC-09) attacks every endpoint with foreign IDs on every build.

Growth management: messages is the only unbounded table. Partitioning by created_at (monthly, pg_partman) keeps retention deletion (FR-MOD-06) as partition drops rather than bulk DELETEs. Under ASM-04 (≤10 M messages/day) this holds to v2.

What messages.metadata holds (added 2026-09-03, chapter 3.23). Three tables in §6.1 declare a metadata JSONB NOT NULL DEFAULT '{}' and this document said what none of them holds. For messages there is now exactly one key, and it arrived with FR-MSG-08's deletion metadata:

metadata.deleted_by = { "kind": "user", "user": "<external id>" }
metadata.deleted_by = { "kind": "application" }

The kind is always recorded; the external id exists only for a user principal, because an application credential has no user of its own. This is a different fact from user_id — a tenant API key may delete any message in its environment (FR-MOD-02), so who removed a message and who wrote it are two people, and FR-MSG-08 itemises "sequence number, author, timestamps, and deletion metadata" with timestamps listed separately, so the last item has to mean more than deleted_at.

Writers merge rather than replace. Chapter 3.23 is this column's first writer anywhere in the platform, so every row that predates it carries '{}'; a later writer of a second key must not erase this one.

Which credential an application deletion presented is NOT here. That is FR-MOD-03's audit log — actor, action, target, timestamp, request id, retained a year, across every moderation action — and a single mutable column on the row it describes is not one.

What messages.attachments holds (added 2026-09-04, chapter 3.24). §6.1 declares attachments JSONB and this document said nothing about it — the same omission chapter 3.23 filled for messages.metadata above, in the same table, one column across.

It holds an array of objects, or NULL:

[ { "type": "url", "kind": "image" | "audio" | "video", "url": "https://…" } ]

NULL and [] are different values, and only one of them is written. A message with no attachments stores NULL; [] is what every read path returns for it (FR-MSG-11's external-URL half, chapter 3.24 FR-007), so a client needs no special case and the platform keeps the distinction between no attachments and a list that happens to be empty. Exactly one place converts — the map in listMessages — and three tests hold it there.

The platform stores a reference and never fetches it. No bytes cross Relay compute; the URL is a string it hands back. That is §4.14's whole premise, and it is why a data: or javascript: URL is refused at the boundary rather than sanitised somewhere later: the allowed schemes are http and https, checked with a URL parser rather than a prefix match.

Bounds. Ten attachments per message and 2,048 characters per URL — the second takes the platform's only precedent for a stored URL, users.avatar_url, capped at 2,048 since chapter 3.16. Multiplied, that is about 20 KB of attachment on a message whose text may be 8,000 characters and whose metadata may be 4 KB.

What each read path does with the column (SC-006, chapter 3.24). Six shapes return a message and they do not agree, which is a fact about this codebase rather than a design:

ReadCarries attachments
listMessages — history, and the resume replayyes, [] when the column is NULL
getMessageByIdempotencyKey — the retry replayyes
editMessage's internal readyes, so the edit event carries what the message has
deleteMessage's internal readno — a tombstone's attachments are unlinked
listMessagesRawno — a test-only helper returning id, seq, text
listChannelsForUser.last_messageno — a preview shows what was said

An edit does not change them (FR-MSG-07 changes message text), and message_edits says nothing about attachments either way: a version records what the text WAS, because nothing about the attachments changed. The table is four columns since chapter 4.19 — ended_by joined the three this document published — and the new one says why a text stopped being current, not what was attached to it. What a deleted message HAD attached is not recoverable at all: FR-MED-10 unlinks it.

A deletion unlinks them. The tombstone's column becomes NULL alongside its text, and the message.deleted event carries no attachment field at all — for the reason it carries no text: a payload with the field can carry what somebody asked to have removed, and a URL is exactly as recoverable.

What reads it: nothing. Not the history route, not the channel listing, not the message.deleted frame, not the webhook event. The frame and the event carry the message's AUTHOR, deliberately, because a client already holds that name beside the message. So the actor is recorded and answerable only by a database query today; a read surface for it would decide, with no requirement asking, who may learn that an operator removed somebody's message.

What every read path does with a tombstone (added 2026-09-03, chapter 3.23, FR-017). Derived by reading the four code paths rather than from a requirement list, because a list goes stale and the code does not:

PathA deleted message
REST history (GET /v1/channels/:id/messages)returned, in its original position, text: null, and carrying deleted_at since chapter 4.19 — the instant, where a live message's row carries null for the same key. listMessages has never had a predicate on messages.text. Before 4.19 a client catching up through history could see the tombstone in place and could not say when it was removed; the real-time frame and the webhook had both carried the instant since 3.23, and the DELETE itself answers 204 with an empty body and carries nothing
Resume backfill (POST /internal/backfill)dropped. A tombstone is not a message.created and there is no truthful text to invent, so toFrame returns nothing for it and the client sees a gap it repairs through history
Channel listing (GET /v1/users/:id/channels)previewed with a null text at its own sequence, and still counted as one unread — unread is last_sequence - read_position, so a tombstone keeps its place in the arithmetic
Backfill truncation flagcomputed from rows read, not frames delivered. A page at the cap containing tombstones returns fewer frames and still reports truncated: true: dropping an unrenderable row is not a reason to tell the client to go page history, and hiding a real cap would be

The fourth row is not a per-state answer like the first three, which is why it is easy to miss — the requirement counted three paths until a fourth was measured.

And the edit's equivalent is one line: every read path returns the CURRENT text, because the superseded text lives in message_edits and no read path but GET …/:messageId/edits touches that table. An edit keeps its sequence number, so a resuming client receives the corrected text as a message.created under the sequence it always had.

6.2 ClickHouse — analytical schema (representative table)

CREATE TABLE message_events (
    environment_id  UUID,
    channel_id      UUID,
    user_id         UUID,
    ts              DateTime64(3, 'UTC'),
    event           LowCardinality(String),   -- created|edited|deleted
    text_length     UInt32,
    attachment_count UInt8,
    delivery_latency_ms UInt32
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(ts)                      -- DR-07
ORDER BY (environment_id, ts)                  -- tenant-scoped range scans
TTL toDateTime(ts) + INTERVAL 90 DAY;          -- DR-09, see the note below

CREATE MATERIALIZED VIEW daily_usage
ENGINE = SummingMergeTree
PARTITION BY toYYYYMM(day)
ORDER BY (environment_id, day)
AS SELECT
    environment_id,
    toDate(ts)            AS day,
    count()               AS messages,
    uniqState(user_id)    AS active_users_state
FROM message_events
WHERE event = 'created'
GROUP BY environment_id, day;                  -- DR-10: billing never scans raw events

AMENDED IN REVISION 1.2, BY MEASUREMENT. The TTL above read TTL ts + INTERVAL 90 DAY from the first draft until chapter 4.2 tried to create the table. ClickHouse 25.3 refuses it:

Code: 450. DB::Exception: TTL expression result column should have DateTime or
Date type, but has DateTime64(3, 'UTC'). (BAD_TTL_EXPRESSION)

A TTL expression must yield DateTime or Date, and ts is DateTime64(3, 'UTC'). toDateTime(ts) is the narrowing the server wants. Amended rather than diverged from silently, per the governance clause.

AND FOUR COLUMNS ARE NULLABLE IN THE IMPLEMENTED SCHEMA. user_id, text_length, attachment_count and delivery_latency_ms are published here as non-nullable and are Nullable in relay-platform/analytics/0000_message_events.sql, each with a comment saying why. The argument is one argument four times: zero is a measurement and NULL is an absence. A non-nullable user_id turns a deleted author's message into the zero UUID and invents an active user; a non-nullable text_length claims an empty message was sent where a tombstone preserved nothing; and delivery_latency_ms has no producer at all and still has none after FR-ANL-10's chapter, so every row would assert a delivery took 0 ms. That sentence read "until FR-ANL-10" and the prediction was wrong: chapter 4.8 defined the quantity the clause was missing — commit to the frame written to a subscriber's socket — and recorded that building the producer means carrying both instants across a service boundary, the commit being the api's and the socket write the gateway's, on the busiest path in the platform. The clause names its quantity now; the column is still empty (gaps.md 048-2). The column list above is kept as the shape; chapter 4.2 carries the nullability and the measurements behind it.

No message text anywhere in this store (DR-08 / FR-ANL-11) — the compliance erasure endpoint (FR-MOD-04) deletes analytical rows, which is tolerable precisely because it is rare and content-free.

TWO THINGS IN THAT SENTENCE WERE WRONG BY CHAPTER 4.21 AND BOTH MATTER. It is not a mutation: ALTER TABLE … DELETE returns with its rows still countable, so a receipt built on it reports an intention, and the erasure uses the lightweight DELETE FROM, which the next SELECT sees. And it is not by user_id: the one analytical table that names a user names them by user_external_id, and the statement must carry environment_id as well — an external id is unique per environment, not globally, so an unscoped delete of the lane's worst id takes 152 rows from 110 other tenants against 4 correctly. Both predicates are bound parameters, because a free-text external id interpolated into that statement defeats the tenant scope outright.

AND THE API REQUEST LOG, ADDED IN REVISION 1.4. relay_analytics.api_requests carries FR-ANL-07's 30-day retention beside webhook_attempts' 90 — a second table rather than more columns, because one TTL clause cannot express two retentions. Its environment_id is nullable, which is the one decision in it worth arguing: a large share of API requests resolve to no tenant, and the platform principal the dispatcher and gateway present carries no environment by design. Revision 1.4 also corrected this document's claim that the analytics stream absorbs 24 h; that window is a function of the record rate, and the same chapter made the rate constant by publishing a record for every request rather than for every webhook attempt.

AND IT HAS A READER NOW, WHICH IS WHAT MAKES IT A SURFACE. GET /v1/request-log (chapter 4.8, ADR-26) is the first customer-facing read of this store: a tenant-scoped page over api_requests, filtered by endpoint, status and a half-open time window, paged by an opaque cursor over (ts, request_id) and read with FINAL. The cursor cannot key on a sequence the way messages does — this table has no server-assigned ordinal — and 42 (environment_id, ts) pairs in the lane hold more than one row, so the pair is what makes a page boundary exact.

THREE THINGS THE READ HAS TO DO THAT THE SCHEMA DOES NOT SHOW. FINAL, because a ReplacingMergeTree returns a redelivered request twice until a merge it cannot be made to wait for. A presence column per nullable column, because the HTTP interface writes NULL as the two characters \N and a reader taking the value alone reports an endpoint of "\N" — endpoint and limited_operation both, the second being NULL on 11,660 of 11,683 rows. And a deadline in two halves, because aborting a fetch leaves the query running (ADR-26).

AND THE PER-TENANT FILTER SKIPS NOTHING AT THIS SIZE. Measured at chapter 4.8: a 50-row page reads the whole table, because the part is Compact and a compact part is one granule whatever index_granularity declares — built two ways, 11,695 rows in a Wide part taking two granules and the same rows in a Compact part taking one. The key is right and there is nothing for it to skip until the table crosses min_bytes_for_wide_part.

AND THE DELIVERY-ATTEMPT TABLE, ADDED IN REVISION 1.3. message_events above is labelled representative and this document named no table for FR-ANL-01's webhook delivery attempts, while services/api/src/webhooks/analytics.ts has been publishing one record per attempt since chapter 3.20. Chapter 4.3 built the consumer and this is the table it writes:

CREATE TABLE webhook_attempts (
    environment_id  UUID,
    ts              DateTime64(3, 'UTC'),   -- the publisher's `attempted_at`
    delivery_id     UUID,
    endpoint_id     UUID,
    event_id        UUID,
    attempt         UInt8,                  -- 1..7
    status          Nullable(UInt16),       -- absent means nothing answered
    error           Nullable(String),       -- absent means there was no error
    latency_ms      UInt32,                 -- how long the ENDPOINT took
    outcome         LowCardinality(String), -- delivered|rescheduled|dead_lettered
    CONSTRAINT ts_is_real CHECK ts > toDateTime64('2020-01-01 00:00:00', 3, 'UTC')
)
ENGINE = ReplacingMergeTree
PARTITION BY toYYYYMM(ts)                            -- DR-07
ORDER BY (environment_id, ts, delivery_id, attempt)  -- ordering AND identity
TTL toDateTime(ts) + INTERVAL 90 DAY;                -- DR-09

The sorting key is also the deduplication key, which is what makes the engine ReplacingMergeTree rather than MergeTree: JetStream delivers at least once and this store has no uniqueness constraint, so a redelivered record must collapse rather than double-count. (delivery_id, attempt) is the same pair the publisher treats as identity. Reads take FINAL — the duplicate is physically present until a merge.

latency_ms here is not message_events.delivery_latency_ms. This is how long an endpoint took to answer a webhook; that is how long a message took to reach a client, and it still has no producer.

AND THE CONNECTION LOG, ADDED IN REVISION 1.5 (chapter 4.5). message_events above is labelled representative and FR-ANL-01 names four arms — message send, connection open and close, API request, webhook delivery attempt. Three had tables; connection open and close was the last one without a producer, and the gateway is the only service that can see it:

CREATE TABLE connection_events (
    environment_id   UUID,                   -- NOT nullable, unlike api_requests
    ts               DateTime64(3, 'UTC'),
    connection_id    UUID,
    event            LowCardinality(String), -- opened | closed
    close_code       Nullable(UInt16),       -- close only; a number, not a reason string
    duration_ms      Nullable(UInt64),       -- close only; UInt32 ms wraps at 49.7 days
    user_external_id String,
    CONSTRAINT ts_is_real CHECK ts > toDateTime64('2020-01-01 00:00:00', 3, 'UTC')
)
ENGINE = ReplacingMergeTree
PARTITION BY toYYYYMM(ts)                                 -- DR-07
ORDER BY (environment_id, ts, connection_id, event)       -- ordering AND identity
TTL toDateTime(ts) + INTERVAL 90 DAY;                     -- DR-09

event IS IN THE SORTING KEY AND THAT IS THE ONE DECISION HERE WORTH ARGUING. One connection produces TWO rows under one connection_id; without it the engine collapses the open into the close and a reader sees half the story. Verified against the server: count() FINAL is 2, not 1.

environment_id is NOT nullable here, where api_requests made it nullable, and the difference is the two chapters' subjects. A request can be made by nobody — every 404, every 401, and every call the internal seam makes under a platform principal. A connection cannot: the only function that builds one takes a non-optional identity, and every refusal path (429 on the upgrade, 4001, 1011, 4003, 4008, and the 4004 connection cap) returns before it. An unauthenticated socket exists; an unauthenticated connection does not.

The producer buffers and publishes on a tick, and that is a contract rather than an optimisation. A publish per close costs 2.87 s of awaited acks for the 10,000 sockets NFR-SCL-01 allows, which is chapter 3.24's burst argument on a new transport; one message per record, pipelined, costs 0.026 ms each. A record whose publish fails is retained and retried rather than dropped — meter.ts drops a report that cannot be delivered except for a closed connection, which has no next report to repair the loss, and every connection event is in that exception.

AND THERE ARE TWO ROLLUPS, WHICH ONE TABLE COULD NOT HAVE BEEN (chapter 4.6). DR-10 asks for daily per-tenant rollups "so billing never scans raw events", and FR-ANL-09 asks for usage attributable by application, environment, channel and day. Putting channel in a single rollup's key satisfies the second and destroys the first, measured over a 248,155-row corpus across 91 days and 2,400 channels:

keyedrowsa tenant's 91-day bill reads
(environment_id, channel_id, day)147,53432,778 rows · 4 ms
(environment_id, day)281—
the raw table, same question248,15532,768 rows · 3 ms

The channel-keyed table is fully merged — every row is a distinct key and OPTIMIZE FINAL changes nothing — so the 525× is the dimension's cost rather than a compaction artefact. A billing read against it touches more rows than the raw events it was supposed to replace. daily_usage_billing is keyed (environment_id, day) and is what billing reads; daily_usage_v2 keeps the channel key and answers the attribution question. Neither is a compromise of the other.

AND A MATERIALISED VIEW IS A TRIGGER ON FUTURE INSERTS, NOT A QUERY OVER HISTORY. On the day the first of these views was created the rollup answered 0 connection-minutes while the same expansion run directly against connection_events answered 56, over records already in the table. Every deployment that applies these statements to a store holding data needs the backfill that follows them, and it will read as working without one, because every new record appears. A backfill recovers only what still exists: measured against the same corpus, the view counted 242,667 messages over 92 days and a backfill taken minutes later found 239,997 over 91 — the difference is one day the 90-day TTL had already removed. The rollups outlive the raw events on purpose — DR-09's 25 months against the raw tables' 90 days, applied as TTL toDateTime(day) + INTERVAL 25 MONTH in chapter 4.6 — which means a rollup created late is permanently short by whatever has already expired. This paragraph said "the rollups carry no TTL (FR-003a)" until chapter 4.7. Both halves were wrong: 4.6 gave them one in the same feature this paragraph was written in, and FR-003a is a feature-local id four features use to mean four different things, not a clause in the SRS.

AND daily_usage_billing NOW CARRIES TWO COLUMNS WHOSE NAMES READ AS THE SAME QUANTITY AND ARE NOT (chapter 4.16). stored_delta counts stored messages — it is sum(multiIf(event = 'created', 1, event = 'deleted', -1, 0)) from mv_billing_messages, and it has counted nothing since it was written because message_events has no producer. stored_bytes_delta counts stored bytes, from mv_billing_storage over media_events. A reader meeting them side by side will assume the second is the first measured differently, and this chapter's own premise check nearly concluded the work was half done for exactly that reason. The name stored_bytes was rejected on the same grounds: it would have read as the byte-valued twin of a message count. A column name is not a unit, and where two of them share a stem the difference belongs in the schema comment and here, not in whoever is reading.

AND THE SECOND READER THAT CROSSES THE FENCE IS DR-17's, WHICH CROSSES IT TWICE (chapter 4.16). reconcileStorage reads daily_usage_billing over HTTP, the object store over its own signed URL, and Postgres through the repository layer — three stores in one function, where FR-ANL-06's reconciler reads two. The third is not a convenience: the analytical side records a reserved event and no uploaded event, so nothing in it can tell an outstanding slot from a delivered object, and the term that explains a gap has to come from the operational side. Which of those reservations are still outstanding can only be answered by the inventory, so all three are load-bearing. It compares every tenant at once rather than one at a time, because its instrument is a listing of a bucket and that arrives whole. It writes nothing. The constitution III reading below is unchanged and now has a fourth item standing against it.

AND THE ONE READER THAT CROSSES THE FENCE IS FR-ANL-06's RECONCILER (chapter 4.7). It runs in the api, reads daily_usage_billing over HTTP and usage_periods/usage_active_users through the repository layer, compares one tenant and one period at a time, and writes nothing. It is the only component in this architecture that reads both stores, which constitution III appears to forbid and ADR-06 below has always assumed — "the only strict consumer (metering) reconciles daily against Postgres" is the trade-off that makes choosing NATS over Kafka acceptable. The reading that holds is that an auditor is not billing, metering or dashboard analytics; an auditor confined to one side of a fence cannot check the fence. SRS revision 1.14 records it, and the amendment is the constitution's.

AND THE COMPARISON IT CAN ACTUALLY MAKE IS NARROWER THAN THE CLAUSE READS. Three of FR-ANL-05's four quantities cannot meet the 0.1% bound for reasons that are not defects — uniq is exact only to 65,536 distinct, the raw retention boundary makes the oldest day in any window disagree by up to 1.0989% depending on the time of day, and connection-minutes count different populations on the two sides. The fourth, stored message count, has no operational counterpart at all. The job reports not-comparable and no-data as verdicts of their own for exactly this reason.

A sixth table, emoji_events (DR-14), records emoji usage as (environment_id, ts, kind, identifier, pack_id) with the same partitioning and TTL. It deliberately omits channel_id and user_id: per-tenant-per-day aggregates (FR-EMJ-11) need neither, and omitting them keeps the table outside the scope of compliance erasure entirely — an aggregate that cannot identify a person needs no erasing. ADR-37 generalises that sentence: it is not only an aggregate with no identifier that needs no erasing, but any record whose only identifier is a key into a row erasure has already emptied.

6.3 Redis — ephemeral state only

Key patternPurposeTTL
conn:{env}:{user}:{slot} → connection ID, slot 0-4Connection registry (FR-RTM-09), built in chapter 3.22. One key per place, and this row used to prescribe a sorted set — correct about the defect and wrong about the fix. A Redis TTL is per key and not per set member, so a set keeps a dead instance's entry alive for ever; making each member its own key makes the TTL per member by construction. SET NX PX claims, SET IFEQ PX renews, a conditional one-millisecond tombstone releases, and the TTL is the only unconditional way a place is freed. The sorted set with ZREMRANGEBYSCORE needs Lua to make the claim atomic, which Constitution VII will not admit without profiling evidence this lane cannot produce (→ ADR-23). Presence (ADR-19) needs none of it — it asks a yes-or-no question of the key below rather than counting members60 s, refreshed every 20 s
presence:{env}:{user}Presence with 30 s grace (FR-RTM-06). Its existence IS the state. The TTL and the grace are two different quantities and both are 30 s by coincidence: the TTL is refreshed every 10 s while a connection is open, and the closing instance re-pins it to the grace so the key dies when the grace ends rather than up to a refresh interval earlier30 s, refreshed every 10 s
presence:offline:{env}:{user}Elects one publisher when two instances' last connections close together (ADR-19). Cleared by the next online30 s
pub/sub presence:{channel_id}Presence fan-out, one subject per channel (ADR-19)—
rl:{env}:{bucket}Token buckets (FR-RTL-01)window
emoji:{env}:{version} → shortcode→URL mapResolution-map cache (→ ADR-12)24 h, version-keyed
pub/sub chan:{channel_id}Fan-out fabric (D2)—
pub/sub member:{channel_id}Membership changes to a channel's members, and to the member being removed — who is still one at the instant it publishes (ADR-20)—
pub/sub member:{env}:{user}Membership changes addressed to a principal rather than a channel (ADR-20). An addition cannot ride the channel's subject: the instance holding the new member is not subscribed to it yet. A ban rides this one alone, carrying channel: "*", which the gateway expands per channel and never sends to a client—
pub/sub typing:{channel_id}Typing signals, one subject per channel (ADR-21). The TTL column is — and that is the point: no key is created, nothing is stored, and the five-second expiry lives in the receiving client because the published frame carries no state field (ADR-22)—

Nothing in Redis is a source of truth. Total Redis loss ⇒ all clients reconnect and resume from cursors; no data loss (NFR-REL-04 analysis depends on this property).


7. Deployment view

flowchart TB
    subgraph k8s ["Kubernetes cluster — any conformant distro (NFR-MNT-06)"]
        ing["Ingress<br/>TLS 1.2+, HTTP→HTTPS"]

        subgraph deployments [Deployments]
            apis["api-svc ×3<br/>HPA: CPU + RPS"]
            gws["gateway-svc ×3<br/>HPA: connection count"]
            whds["webhook ×2"]
            ings["ingester ×2"]
            mws["media-worker ×2<br/>HPA: consumer lag<br/>+ ClamAV sidecar"]
            sses["sse-svc ×2"]
            dashs["dashboard ×2"]
        end

        subgraph stateful [StatefulSets]
            natss[("NATS ×3<br/>JetStream R3")]
            rediss[("Redis<br/>sentinel")]
        end
    end

    subgraph managed [External / managed]
        pgm[("PostgreSQL<br/>primary + replica, PITR")]
        chm[("ClickHouse<br/>single node v1 — ADR-08")]
        objm[("Object storage<br/>S3-compatible — ADR-13")]
    end

    ing -- "api.relay.dev" --> apis
    ing -- "ws.relay.dev" --> gws
    ing -- "dashboard.relay.dev" --> dashs
    ing -- "dashboard.relay.dev/events" --> sses
    apis --> pgm
    apis --> rediss
    apis -. "presigned URLs" .-> objm
    gws --> rediss
    pgm -- outbox relay --> natss
    natss --> whds
    natss --> ings
    natss --> sses
    natss --> mws
    mws --> objm
    ings --> chm

Local development (NFR-MNT-03): docker-compose up — every service plus postgres, nats, redis, clickhouse, and MinIO standing in for object storage, with a seeded demo tenant.

Gateway drain on deploy (NFR-REL-03): on SIGTERM the gateway stops accepting connections, sends a server.shutdown frame with a jittered reconnect hint, waits up to 30 s, closes with code 4009. Clients reconnect to surviving instances and resume by cursor — a deploy costs each client exactly one reconnection cycle, which is the SRS bound.

Scaling triggers: gateway on connection count (NFR-SCL-01 sets 10k/instance as the budget — measure before trusting, see risk R2); API on RPS; ingester on JetStream consumer lag; webhook dispatcher on stream depth.


8. Cross-cutting concerns

Security. Three credential middlewares (API key hash-lookup with 5 s revocation cache → FR-AUT-05; JWT verification per environment secret; dashboard session). Key rotation is dual-active by design (FR-AUT-04). Secrets envelope-encrypted at rest; TLS everywhere; schema validation with unknown-field rejection at the edge of every handler (NFR-SEC-04).

Observability. OpenTelemetry SDK in every service; trace context propagated through JetStream headers so a message's journey — REST ingress → outbox → dispatcher → webhook attempt — is one trace (NFR-OBS-02, -06). X-Request-Id = trace ID, so a customer support ticket carries its own trace handle. Structured logs with environment_id on every line; Prometheus metrics per NFR-OBS-03; the four golden alerts of NFR-OBS-04.

Two metrics systems, on purpose. ClickHouse (FR-ANL) and Prometheus (NFR-OBS) both hold "metrics," and the duplication is deliberate. ClickHouse is the product's analytics: per-tenant business events answering questions customers ask — usage, metering, request logs, delivery percentiles in their dashboard. Prometheus is operational observability: pre-aggregated time series about the system itself, answering questions the operator asks and feeding the alerts that page someone. They cannot be consolidated, because the observer must not share fate with the observed: ClickHouse outage is a row in the failure matrix above, and the entire analytical path is designed to be droppable (D5) — alerting cannot live on a component whose failure the architecture is built to tolerate. Prometheus's pull model scrapes /metrics directly from each service, touching none of the data infrastructure, so it keeps working precisely when the event pipeline does not. (This is ADR-09's observer-isolation argument applied one level up.) Secondary reasons: the data shapes differ (aggregated series + PromQL alert expressions vs. raw high-cardinality events), and the Kubernetes deployment (§7) makes Prometheus the ambient standard — every StatefulSet ships an exporter. Where one measurement serves both audiences (delivery latency: operator alerts and FR-ANL-10's customer percentiles), it is recorded twice, once per store — correct, not wasteful. A single ClickHouse-backed observability stack is a defensible design elsewhere (SigNoz et al.); it loses here on the fate-sharing argument specifically.

Backpressure, end to end. Client → gateway: per-connection send window (unacked frames cap). Gateway → API: bounded internal HTTP pool; on saturation, reject sends with a retryable error rather than queueing unboundedly. Outbox → JetStream: relay lag alarmed. JetStream → consumers: pull consumers with explicit ack; slow consumers grow the stream, which is the design (D5), bounded by 24 h retention (NFR-REL-08).

Failure matrix (summary of the analysis behind NFR-REL):

FailureBlast radiusRecovery
Gateway instance diesIts connections onlyClients reconnect + resume; zero loss (Redis registry TTLs out)
Redis lostPresence + fan-out pauseGateways buffer briefly, reconnect clients; Postgres unaffected
JetStream lostWebhooks, analytics, live dashboard pauseOutbox accumulates in Postgres; relay drains on recovery — this is why the outbox is in Postgres, not fire-and-forget
ClickHouse lostDashboards staleIngester pauses; the stream absorbs the backlog for a window set by the record rate — 24 h holds only up to 38.8 records/second at 320 bytes each (NFR-REL-05)
Object storage lostMedia uploads/downloads fail; text messaging unaffectedUpload slots return a specific error; attachments render as temporarily unavailable; no Relay-side state to recover — storage provider's durability is the recovery
Postgres lostFull write outageThe one honest SPOF: managed HA + PITR (NFR-REL-06/07); reads could survive on replica but v1 does not attempt write continuity

Scaling behaviour by scenario. The failure matrix answers "what breaks?"; this answers "what saturates?". Each load scenario stresses a different component, and each service deliberately scales on a different signal (§7).

S1 — connection growth (many users online, mostly idle). Pure gateway load: socket memory, heartbeats, registry TTL refreshes. Linear horizontal scaling on connection count; no sticky routing (D2) means new instances absorb load immediately. Idle connections cost Postgres nothing. The cheapest dimension to scale — and the reason the per-instance connection budget (NFR-SCL-01, risk R2) must be measured first: it is the fleet-sizing formula.

S2 — aggregate message throughput, spread across channels. Two regimes: horizontal at the stateless API tier until Postgres saturates, then vertical at Postgres. Per-channel locks (ADR-03) do not contend across channels, so they are irrelevant here. A tuned single primary clears ASM-04 (10 M msg/day ≈ 115/s average) and the 1,000 msg/s target (NFR-SCL-03) with roughly an order of magnitude of headroom. Beyond that lies tenant sharding — a v2 redesign, deliberately out of scope. This is the architecture's one true wall, named as R1.

S3 — one hot channel (1,000 members, rapid sends). The sequence lock serialises that channel's writes, but a row-lock cycle is sub-millisecond — hundreds of msg/s in one channel, faster than humans converse, and serialisation is the requirement (FR-MSG-03). The real cost is fan-out amplification: 100 msg/s × 1,000 members = 100,000 frames/s of gateway egress. Redis publishes once per message; each gateway multiplies to its local sockets, so gateway egress bandwidth and event-loop time saturate first, never the database. Response: more gateway instances (thinner member spread); revisit trigger: per-socket frame batching if hot channels become the norm. FR-CHN-07's 1,000-member cap exists to bound this amplification factor.

S4 — reconnection storm (deploy, or network blip recovery). Tuan's car park at fleet scale, and the likeliest real incident. The herd hits JWT verification (gateway CPU), the registry (Redis write burst), and — the expensive part — backfill reads fanning through the API into Postgres. Layered defence: SDK jittered backoff (FR-SDK-04) spreads the herd; the deploy drain protocol (§7) pre-spreads it; the 500-message backfill cap (FR-RTM-04) bounds per-user read cost. If backfill still dominates, ADR-04's stated escape hatch fires: route gateway backfill to a read replica — reads do not threaten invariants. Reconnection is a read-scaling problem, and reads have a cheap answer (replicas) that writes do not.

S5 — webhook trouble (event burst, or one slow customer endpoint). The dispatcher scales on stream depth, but the governing property is isolation, not throughput: per-endpoint concurrency limits stop one endpoint timing out at 10 s from occupying the worker pool. The stream absorbs backlog (D5/D6 by design), retries decay to the 2 h tier, auto-disable (FR-WHK-07) amputates dead endpoints. Nothing on this path can touch message delivery — the queue is a one-way valve.

S6 — analytics load (event bursts, heavy dashboard queries). The ingester scales on consumer lag, and batching means more throughput arrives as bigger batches before it means more instances — which is what ClickHouse prefers. Query load lands on a single node (ADR-08) that scales vertically for a long time because rollups (DR-10) keep billing off the raw tables. Total failure of this path degrades to stale dashboards and a draining backlog; messaging never feels it.

S7 — tenant growth (100 → 10,000 customers). Nearly free: tenancy is rows, not infrastructure — no per-tenant processes, topics, or schemas. What does scale with tenant count: rate-limit key cardinality (trivial), webhook endpoint count (dispatcher shards by environment), and noisy-neighbour pressure, which quotas (FR-RTL) contain. The watch item is skew: one tenant at 50% of traffic is fine for the shared-schema model until their retention deletes (R6) or exports bully the shared tables.

S8 — media-heavy tenants (photo/voice-note dominant traffic). The design's payoff scenario: bytes flow client → object storage and storage → client directly (ADR-13), so media volume stresses Relay only at three metadata points — slot issuance and signed-URL minting on the API service (a hash + HMAC, thousands/s per instance), scan throughput on the media worker (CPU-bound; scales horizontally on consumer lag, and a backlog degrades only time-to-ready, never message delivery per FR-MED design note 1), and the storage bill (metered per tenant, FR-MED-12, with DR-17's inventory reconciliation as the tripwire). Object-storage bandwidth itself is the provider's scaling problem — which is precisely why ADR-13 buys it rather than builds it.

Saturation summary:

ComponentScales onRegimeReal ceiling
Gatewayconnection counthorizontal, linearfan-out amplification (S3)
API serviceRPS / CPUhorizontalPostgres write throughput behind it
PostgreSQL—vertical, then shardthe wall (R1) — ~10× above v1 targets
Redispub/sub throughputvertical; shard by subject if everfar beyond v1 scale
Webhook dispatcherstream depthhorizontalcustomer endpoint slowness (isolated, not fixed)
Analytics ingesterconsumer laghorizontal + larger batchesClickHouse insert rate (huge)
Media workerconsumer laghorizontalscan CPU; backlog degrades time-to-ready only (S8)
Object storage—provider's problem (ADR-13)the invoice, not the throughput
SSE / dashboardsessionshorizontalnone relevant

The design converts every scaling problem into either "add stateless instances" (cheap, automatic) or "Postgres writes" (the single named, bounded, monitored wall) — a clarity bought by refusing distributed writes in v1. A trade, not a triumph; but one wall with a name beats three without.


9. Architecture Decision Records

Each ADR below is deliberately terse. The full rationale — problem framing, all options considered, quantitative analysis, and consequences — lives in the companion document 06-adr-deep-dives.md, one deep dive per ADR.

ADR-01 — Runtime: TypeScript/Node.js for all services

Status: accepted · Drivers: D7, D8

One language across services, SDK, and dashboard; the SDK (FR-SDK-01) must be JS anyway, and sharing protocol types between server and SDK eliminates an entire class of drift bugs. Node's event-loop model suits I/O-bound gateway work. Trade-off accepted: CPU-bound work (HMAC signing at volume) needs care; NFR-SCL-01's 10k connections/instance must be validated early (→ R2). Rejected: Go (better gateway fit, but splits the codebase into two languages for a solo builder); a polyglot showcase (violates D8 — depth over surface).

ADR-02 — Queue: NATS JetStream over Kafka

Status: accepted · Drivers: D5, D6, D8

JetStream provides durable streams, consumer groups, redelivery, and DLQ-adjacent patterns at a fraction of Kafka's operational mass, and doubles as the transport the SSE service subscribes to. Volume ceiling (ASM-04, NFR-SCL-05: 10k events/s) is comfortably within JetStream's envelope. Trade-off: weaker ecosystem for exactly-once sinks; mitigated because the only strict consumer (metering) reconciles daily against Postgres (FR-ANL-06). Rejected: Kafka (operational overkill at this scale; the reconciliation job is needed regardless); Redis Streams (couples the durability spine to the ephemeral-state store — one Redis incident would then have two blast radii).

ADR-03 — Ordering: per-channel sequence via last_sequence row lock

Status: accepted · resolves SRS Open Question 1 · Drivers: D1, D3

Per-channel sequences, assigned by incrementing channels.last_sequence under SELECT … FOR UPDATE in the message-insert transaction. Contention is scoped to a single channel — and serialising a channel's writes is the requirement (FR-MSG-03), so the lock is not a cost, it is the mechanism. Resume cursors are per-channel maps {channel_id: seq}, capped in practice by FR-RTM-04's truncation. Rejected: per-tenant sequence (single hot row per tenant — a real bottleneck for zero benefit at v1 scale); Postgres sequences per channel (unbounded object count, non-transactional gaps break dedup reasoning); Snowflake-style IDs (globally unique but not gap-free per channel, which complicates the client's "did I miss something?" logic).

ADR-04 — Single writer: only the API service touches PostgreSQL

Status: accepted · Drivers: D1, D4, D8

All invariants — sequence assignment, idempotency, tenant scoping, tombstone semantics — live in one codebase behind one repository layer. The gateway calls internal HTTP for writes and backfill reads. Trade-off: one extra intra-cluster hop on the send path (~1–2 ms) and on resume; accepted against the alternative of duplicating isolation logic in two services and testing it twice. Revisit: if backfill volume ever dominates API-service load, grant the gateway a read-only replica path — reads don't threaten invariants.

AND media_objects.state HAS TWO WRITERS FROM CHAPTER 4.13, WHICH IS NOT A BREACH AND IS WORTH WRITING DOWN — constitution IV names exactly this shape, so the argument goes here rather than being inferred from the fact that nothing broke. The worker does not touch Postgres: it posts a verdict to /internal/media/:mediaId/verdict and the api performs the write, so ADR-04's letter is untouched. What it is second writer of is the column's meaning.

The api writes pending at slot time and never writes that column again. The worker's verdict writes ready or rejected and can only move a row out of pending — the statement is UPDATE … WHERE state = 'pending', so a second verdict for a resolved object updates nothing and is told so. The transitions are disjoint: nothing but the slot route creates a row, nothing but a verdict leaves pending, and no path returns to it. That is what makes two writers safe here, and it is a property of the statements rather than of anybody's discipline — which is why the predicate is in the UPDATE and not in a comment above it.

ADR-05 — Sends travel through the WebSocket, writes through the API

Status: accepted · Drivers: D3, journey 4

Clients send over the socket they already hold (lower latency, and the SDK's offline queue flushes through one channel), but the gateway forwards to the API service rather than writing. REST send (FR-MSG-13) uses the identical API path — one write path, two entrances. Rejected: REST-only sends (breaks the offline-flush ordering story and doubles connection overhead on mobile); gateway-direct DB writes (violates ADR-04).

ADR-06 — Transactional outbox between Postgres and JetStream

Status: accepted · Drivers: D1, D5, FR-ANL-06

State change and its event commit atomically; a relay (polling, FOR UPDATE SKIP LOCKED, batch-publish, mark published) drains to JetStream. At-least-once by design — consumers dedupe on event id (EIR-WHK-04 pushes the same discipline to customers). Rejected: publish-after-commit (drops events on crash in the gap; metering drift would violate FR-ANL-06 undetectably); CDC/Debezium (operational mass, D8); publish-before-commit (phantom events, worse).

ADR-07 — Fan-out fabric: Redis pub/sub, at-most-once, by design

Status: accepted · Drivers: D2, D3 · extended by ADR-20 (chapter 3.20), which carries a payload this record's loss argument does not cover — a revocation has no cursor · and by ADR-22 (chapter 3.21), which carries the opposite case: a dropped typing frame self-corrects within one renewal interval, so this record's argument holds for it exactly

Live fan-out uses fire-and-forget Redis pub/sub. A dropped pub/sub frame is not a lost message: durability lives in Postgres, and the client's cursor + sequence-gap detection recovers anything missed (the SDK refetches on gap). This is the architectural payoff of ADR-03 — once ordering and resume are correct, the fan-out fabric is allowed to be lossy, and therefore simple and fast. Rejected: JetStream for live fan-out (durable, but per-channel consumer management for ephemeral delivery is heavy machinery to avoid a problem the cursor already solves); core NATS pub/sub (technically apt — at-most-once, subject-based, comparable latency — and refused on dependency shape rather than mechanism: Redis is mandatory for the gateway regardless, since ADR-10 puts presence in Redis with TTLs, so fan-out on NATS would leave that service holding two broker clients and remove none); gateway-to-gateway mesh (O(n²) connections, discovery complexity).

Amended 2026-08-04 (v1.1). Core NATS pub/sub was added to the rejected list. The original record refused JetStream for live fan-out but was silent on core NATS — the closest competitor, and the one the "we already run NATS" argument actually reaches for. That refusal is deliberately weaker than the others: it is an argument about how many client libraries the gateway holds, not about whether the mechanism fits. A NATS-only proposal that also moves presence off Redis (NATS KV) would reopen it legitimately, and would be a larger decision than this ADR — it would delete a store from the deployment, not swap a fabric.

Amended 2026-09-03 (chapter 3.23) — THE CURSOR RECOVERS CREATIONS AND NOTHING ELSE, and this record's loss argument rests on it. "The client's cursor + sequence-gap detection recovers anything missed" is true of every payload this fabric carried when the sentence was written, and chapter 3.20 already narrowed it once for a revocation, which has no sequence. A message revision narrows it a second way, and differently: an edit and a deletion both have a sequence, and it is the sequence of a message the client may already hold.

So a cursor cannot address them. A cursor is a position in a channel's sequence and a sequence orders creations; an edit creates nothing, so there is no position at which it happened. Concretely (FR-016a and FR-016b of chapter 3.23): a message older than a client's cursor that changed during a disconnect produces no frame AND no sequence gap. The absent gap is the half that matters — gap detection is the mechanism this record names as the recovery, and it sees nothing to recover.

The repair is a history re-read, which returns current state: the corrected text for an edit, and the row with a null text for a deletion. That is documented as a property of a cursor rather than as a limitation, because the alternative is a different design rather than a fix. Slack does exactly this — conversations.history returns current state and replays no event stream, with message_changed and message_deleted existing only as live events. Matrix takes the other shape: an append-only timeline where a redaction is an event of its own, so a resuming client receives it — at the cost of a timeline that grows with edits rather than with messages. IMAP's CONDSTORE/QRESYNC takes a third: a MODSEQ beside the sequence, so a client asks what changed since modseq N — a second monotonic counter per mailbox that every mutation has to maintain. Both alternatives add a per-channel counter; this platform is already the first shape, and chapter 3.23 chose to say so rather than to add one.

Amended 2026-09-15 (chapter 4.5) — THE PRICE THIS RECORD PUT ON CORE NATS IS NOW ZERO, AND THE CLEAN MAPPING IS BROKEN ON BOTH SIDES.

The Decision is untouched and so are the Revisit when clauses; this amends the selection argument only, in the shape chapter 3.18 used when it did the same from the api's side.

Two lines above stop being true. The rejected list says core NATS was "refused on dependency shape rather than mechanism … so fan-out on NATS would leave that service holding two broker clients and remove none", and v1.1 restates that as "an argument about how many client libraries the gateway holds". Chapter 4.5 gives the gateway a NATS client for FR-ANL-01's connection records — 5 dependencies to 6 — so it holds two brokers already. Fan-out on NATS would now add none and remove none: the cost side of the refusal is zero, and what is left is that Redis does not leave, which ADR-10 decides and this record has always said.

And the clean mapping is the thing that actually went. "Gateway to Redis, api and workers to NATS" lost its api half in 3.8 and 3.18 — the amendment below records that cost as "relocated rather than avoided" — and this chapter takes the gateway half, which is the half the mapping is named for. Afterwards it describes no service in this platform. A reader comparing it against §4's component diagram deserves to be told so rather than left to reconcile them.

AMENDED RATHER THAN SUPERSEDED, AND THE ALTERNATIVE IS NAMED. Constitution VII says ADRs are immutable once accepted and that superseding requires a new ADR, and ADR-07 carries both forms already: two dated amendments, and extended by ADR-20 and ADR-22. The rejected alternative here was a new extending ADR. It was declined because the decision has not changed — ADR-20 and ADR-22 extend this record because they carry new payloads whose recovery properties differ, which is a new decision each time, and this chapter carries none. VII's closing line, "disagreement attacks the driver, not the choice", describes a proposal to change the choice; nobody is proposing one. A fact a driver rested on became false, which is what a dated amendment is for.

The full argument is in docs/06-adr-deep-dives.md, amended with this.

ADR-08 — ClickHouse single-node in v1, schema designed for cluster

Status: accepted · Drivers: D5, D8, NFR-SCL-05

One node with backups meets 10k inserts/s and the 2 s/90-day query bound with margin. The analytical store's unavailability is survivable by design (24 h stream buffer), so HA here buys little. Partitioning and ORDER BY are already cluster-shaped; moving to a replicated setup is a data migration, not a redesign. Rejected: ClickHouse Cloud from day one (cost + a cloud dependency against NFR-MNT-06); reusing Postgres for analytics (CON-01 exists precisely to forbid this).

ADR-09 — Dashboard live view over SSE, not WebSocket

Status: accepted · Drivers: D7

The dashboard's live stream (FR-DSH-02) is one-directional; SSE gives it with plain HTTP, automatic reconnection, and zero protocol design. Keeps the real WebSocket gateway dedicated to end-user traffic with its own scaling signal. Rejected: sharing the gateway (couples dashboard load to end-user delivery paths — the one thing that must not degrade); polling (2 s latency bound of FR-DSH-02 makes it ugly).

ADR-10 — Presence in Redis with TTL, no dedicated service

Status: accepted · resolves SRS Open Question 3 (provisionally) · Drivers: D8 · superseded in part by ADR-19 (chapter 3.19), which replaces the subject-grammar clause below

Presence = connection-registry keys with a 30 s grace TTL (FR-RTM-06); transitions publish on the affected channels' subjects only (FR-RTM-07). Presence loss (Redis incident) is cosmetic and self-heals — the correct amount of durability for typing dots and green circles is none. Revisit trigger: presence fan-out exceeding ~30% of gateway publish volume in load tests.

ADR-11 — Custom emoji as shortcodes in plain text, with a read-time resolution map

Status: accepted · Drivers: D1, D3, D8 · implements FR-EMJ-08/10

Message text remains a plain string; custom emoji are written as :shortcode: and resolved at read time — history and send responses carry a sidecar map {shortcode → {image_url, pack_id} | unresolved} covering shortcodes present in the payload. Why: every invariant the architecture bleeds for — byte-exact storage (FR-EMJ-01), tombstones, edit history, idempotent retries — is defined over an opaque string. Structured message entities (an AST of text runs and emoji nodes, Slack-style) would entangle emoji lifecycle with FR-MSG semantics: what does an edit diff of an entity tree mean? What does a tombstone preserve? Plain text keeps the write path emoji-ignorant; the emoji system becomes a pure read-side concern, which is also what makes FR-EMJ-10 trivial — deleting a pack changes future resolutions, never stored messages (Priya's journey 3 depends on exactly this). Trade-off: literal :text: that was never an emoji may resolve accidentally if a matching shortcode is later created; accepted — the client fallback and the shortcode grammar make collisions benign. Rejected: entity trees (above); Unicode Private Use Area code points (uninspectable, breaks the "text is honest" property, hostile to export FR-MOD-05); server-side rendering to image URLs inside text (mutates content — violates FR-EMJ-01's spirit and Priya's record).

ADR-12 — Environment-scoped resolution map, cached whole, invalidated by version

Status: accepted · Drivers: D8, NFR-PRF-03 · implements DR-13

History reads are the hottest read path (NFR-PRF-03: p95 < 100 ms) and now must attach a resolution map. Per-message shortcode lookups against Postgres would put a join on every history page. Instead: the entire environment's shortcode map is cached in Redis under a version-stamped key (emoji:{env}:{version}, DR-13); any pack mutation bumps emoji_version, making stale keys unreachable rather than requiring explicit invalidation — the same version-key pattern as CDN cache-busting. Map size is bounded by construction: FR-EMJ caps at 200 emoji/pack and shortcodes at 64 chars, so even a hundred packs is a sub-megabyte map, loadable in one round trip and memoised in-process per API instance. Resolution-map assembly is then a pure in-memory scan of the returned page's text. Trade-off: a pack mutation cold-starts the cache for its environment once; acceptable — pack edits are rare, reads are constant. Rejected: per-message resolution joins (hot path); pushing resolution to the SDK only (leaves REST-only consumers unresolved and duplicates logic across clients — the map costs the server almost nothing given the cache).

ADR-13 — Media bytes never transit Relay compute: presigned direct-to-storage

Status: accepted · reverses the v1.0 file-storage exclusion · Drivers: D5, D8, NFR-PRF-08

Uploads: the API service validates declaration and quota, records a pending row, and returns a presigned PUT URL; the client uploads straight to S3-compatible storage. Downloads: signed GET URLs minted at read time, authorised by channel membership (FR-MED-08), never persisted (DR-16). Relay's services handle metadata only — the original exclusion's cost argument (bandwidth, CDN, storage ops) is answered by not building any of it: object storage's durability, bandwidth, and lifecycle rules are bought, not rebuilt. Tenant-prefixed keys (DR-15) make tenant erasure and export prefix operations — and USER erasure is not one of them, which chapter 4.21 had to measure rather than assume. The key is {environment_id}/{media_id} and carries no user segment, so FR-MOD-04's erasure of one person's uploads is a lookup in Postgres followed by one store request per object. DR-15's prefix is for tenants. Trade-off: presigned-URL auth is coarser than per-request auth (a leaked signed URL is valid until expiry — bounded at 1 h, unguessable, and never stored); upload success is observed asynchronously (the confirm/scan event), not synchronously. Rejected: proxied uploads through the API service (every media byte on Relay's network path — the exact cost the exclusion feared, now voluntary); public-read bucket with obscure keys (violates FR-MED-08; membership changes must revoke access, and time-limited URLs are the mechanism).

ADR-14 — Async scan pipeline gates bytes, never messages

Status: accepted · Drivers: D5, D6, journey 4 · implements FR-MED-04/06/07

A message may reference pending media and ship immediately; recipients render a placeholder; the media worker's pending → ready transition fans out media.updated on the referencing channels. The scan gates byte delivery (no signed URL until ready), never message delivery. Why: the alternative — scan-before-send — puts a 2–10 s CPU-bound pipeline inside the send path, violating the latency budget (NFR-PRF-01) and coupling message availability to scanner availability, exactly the coupling D5 exists to forbid. The state machine mirrors the message state machine the SDK already implements (sending/sent/failed ↔ pending/ready/rejected) — one mental model, twice applied. Rejection is a first-class terminal state rendered explicitly (FR-MED-09): Priya must distinguish "the upload was rejected" from "the message was deleted." Trade-off: a recipient can see a placeholder for a file that is subsequently rejected — accepted, since consumer messengers set exactly this expectation. Rejected: synchronous scanning (above); no scanning (hosting unscanned user uploads is a liability no metering revenue covers); scan-on-first-download (moves the latency to the recipient's tap and re-scans per CDN miss).

ADR-15 — API service application framework: NestJS

Status: accepted · Drivers: D7, D8 · serves EIR-API-04/07, NFR-USE

The API service's REST surface grows to dozens of endpoints across Phases 2–4 (tenancy, channels, messages, keys, moderation, emoji packs, dashboard reads). NestJS supplies the module/DI/guard/pipe conventions that keep per-endpoint variance low for a solo builder, schema validation at the boundary as a framework primitive, uniform error shaping (EIR-API-04), and generated OpenAPI 3.1 (EIR-API-07) instead of a hand-maintained spec. Scope: the API service only. The gateway stays frameworkless — its work is raw socket mechanics (resume buffering, backfill ordering), and a framework between that code and the socket is surface without benefit. Workers remain plain consumers. Isolation is unaffected: the repository layer (D4) sits beneath the framework, constructed per request with the authenticated environment_id — guards authenticate, the data layer isolates. Trade-off: decorators require a build/transform step, and the dependency tree grows a DI container; accepted against hand-rolling (and hand-documenting) the same conventions across the Phase 2–4 surface. Rejected: hand-rolled node:http routing (right for a walking skeleton, quadratic pain at platform scale); bare Express/Fastify (routing without the validation/OpenAPI/DI conventions this surface needs — Fastify is the named fallback). Revisit: if framework overhead shows up on latency-budget paths (NFR-PRF-01), or if the abstraction starts fighting a core-loop mechanism the way it would on the gateway.

ADR-16 — API service data layer: Drizzle over raw pg

Status: accepted · Drivers: D1, D4, D8 · serves NFR-MNT-02

Repository-layer queries are written with Drizzle: SQL-shaped, fully typed, and — the deciding test — the core loop's mechanisms are first-class, not escape hatches: SELECT … FOR UPDATE (ADR-03's row lock), ON CONFLICT DO NOTHING (DR-03), partial unique indexes and CHECK constraints expressible in the schema definition. Migrations remain versioned, forward-only SQL files: drizzle-kit generates them, they are reviewed and applied as SQL, and the applied SQL — not the TS schema — is what runs; generated migrations are diffed against this document's §6.1 definitions so the two cannot drift silently. The repository discipline is unchanged: Drizzle is the query engine inside the layer, never a client handed to handlers; raw SQL islands remain permitted inside the layer where the builder falls short. Trade-off: the schema gains a TS definition alongside §6.1's SQL — a second artifact to keep honest, checked rather than assumed. Rejected: Prisma (cannot express DR-03's partial unique index in its schema, and lock semantics exist only as raw escape hatches — the abstraction fails exactly where correctness lives); TypeORM (weak result typing where NFR-MNT-02 wants confidence; legacy trajectory); kysely (the closest call — typed SQL with zero schema ownership; declined for Drizzle's schema-level constraint definitions and relational ergonomics, and it is the named fallback). Revisit: if the TS schema and applied SQL drift in practice, or if builder coverage forces raw SQL for a majority of new queries — at that point kysely's schema-free model wins.

ADR-17 — Monorepo build orchestration: Turborepo

Status: accepted · Drivers: D8 · serves NFR-MNT-02/03

ADR-15 gives the workspace its first real build step (NestJS's decorators compile), and Phases 2–4 grow it to six services plus shared packages. Plain pnpm -r re-runs every task in every package on every change; with a build step and a task order (protocol builds before the services that import it) that cost becomes quadratic exactly where D8 wants it flat. Turborepo runs the existing package scripts through a declared task graph with content-hash caching: the gate's cost scales with the change, not with the workspace. Scope: a task runner only. pnpm remains the package manager and workspace protocol; every turbo run target is an ordinary package script, so removing Turborepo degrades to pnpm -r, not to a rewrite. Remote caching is deferred until CI exists. Trade-off: a wrong cache is a false green — correctness now depends on turbo.json declaring task inputs and outputs honestly. Accepted with discipline: input/output declarations are reviewed like code, and tagged checkpoints run clean-cache. Rejected: staying on plain pnpm -r scripts (correct until a build step existed; the wrong default once one does — and it remains the degradation path); Nx (more capable, but its plugin ecosystem, daemon, and generators are surface a solo builder doesn't need — the named fallback if Turborepo's caching proves untrustworthy); Bazel (a build team's tool, priced in a currency D8 doesn't hold). Revisit: if stale-cache incidents survive the input-declaration discipline, or if the task graph starts encoding knowledge that belongs in package.json dependencies.


ADR-18 — Two user populations: platform humans and tenant end users, never merged

Status: accepted (chapter 3.1) · Drivers: D4 · serves FR-TEN-01/06/07, NFR-SEC-09

Relay has two kinds of person and they share nothing but the word "user". A customer's end users (users, §6.1) belong to an environment, arrive through the API, are identified by the customer's own external_id, and never sign in to Relay. The humans who run a Relay account belong to organisations, arrive by OAuth, are identified by a provider account, and may own several organisations at once. They live in separate tables — users below the tenant boundary, humans above it — and no row ever moves between them.

The instinct to merge them is strong, and the cost is precise: a single table needs a nullable environment_id, because a platform human belongs to no tenant. That is the one column shape Principle I forbids, and FR-TEN-06 states as a requirement — every operational record carries a non-null tenant identifier. Once nullable, the repository's mandatory scoping cannot be enforced by construction, and every isolation guarantee in the system becomes a code review rather than a type. Trade-off: two identity tables, and "who is this?" is answered differently on either side of the boundary — accepted, because that difference is exactly what the boundary means. Consequence: a person's provider account is their identity (UNIQUE (provider, provider_account_id)); account linking across providers needs a verified-email or add-a-login flow and is deferred with its own chapter. Rejected: one users table with a nullable tenant (breaks FR-TEN-06 and Principle I); a platform_users view over the same table (same nullable column, now hidden); storing organisation membership as an array on the organisation (not queryable, no room for roles, no foreign key).

ADR-19 — Presence on its own subject grammar, superseding ADR-10's

Status: accepted (chapter 3.19) · supersedes ADR-10's subject clause · closes SRS Open Question 3 · Drivers: D8 · extended by ADR-21 (chapter 3.21), which re-derived this record's count of the message path's typed points and found seven where this says three — the argument is unchanged and stronger

ADR-10 said presence "transitions publish on the affected channels' subjects only". They publish on presence:{channel_id} instead — a subject derived from each affected channel rather than the channel's own — and the audience is unchanged: the same members, reached the same way. The grammar and the payload schema live in their own protocol module, the gateway half lives in its own module beside fanout.ts, and fanout.ts itself is not edited. The event spine already keeps its subjectFor in its own file, so each fabric owning its subject grammar is the precedent here rather than a concession.

Why not the channel's own subject. The message path is typed to messages at three points, and they are not all in one file: publish(message: Message) and a messageCreatedSchema parse in services/gateway/src/fanout.ts, and the literal message.created send inside session.ts's deliver, a function ten chapters fence. Carrying two kinds on chan:{channel_id} means editing the highest-volume path in the system to serve the lowest-volume traffic on it, and it makes cross-kind mis-delivery a property tests must defend rather than one the topology guarantees.

Rejected: an enveloped payload on chan:{channel_id}. Halves the subscription count and reads closer to ADR-10's letter. It also puts a discriminated-union parse on every message every instance receives, and during a rolling deploy an old instance emits fanout.invalid_payload for every transition on every channel until it drains.

The declared cost, measured. A channel now carries two subscriptions instead of one — one SUBSCRIBE per channel per instance, confirmed at 6 and 6 for two instances over three channels. ioredis takes a variadic subscribe, so the round trips do not double.

This is half of ADR-10's own revisit remedy, taken before its trigger fired. ADR-10 says that above ~30% of gateway publish volume "presence subjects get their own fabric or channels opt in". Nothing here measured publish volume at scale; the reason is the typed fan-out above, not the trigger. The trigger remains undischarged and so does NFR-SCL-01.

Revisit when: presence fan-out exceeds ~30% of gateway publish volume in load tests, or the doubled subscription count becomes the constraint on connections per instance.

ADR-20 — Membership on a third subject grammar, with a periodic re-read behind it

Status: accepted (chapter 3.20) · extends ADR-07's loss argument to a payload that has no cursor · Drivers: D2, D8 · the count it borrows from ADR-19 is corrected in ADR-21 (chapter 3.21): seven typed points, not three. The argument it carries is unchanged

Membership changes publish on member:{channel_id} and member:{env}:{user}, and every connection re-reads its own memberships from the api on a sixty-second timer.

ADR-07 permits a lossy fabric with a stated reason, and this payload falls outside it. That record says a dropped frame "is not a lost message: durability lives in Postgres, and the client's cursor + sequence-gap detection recovers anything missed". A revocation has no sequence and no cursor. It is not in a stream, a client cannot detect a gap in it, and there is nothing to refetch — a dropped one is a client that keeps receiving a channel it was removed from, indefinitely, which is FR-RTM-10's failure and not a delivery delay. Constitution IV requires any new mechanism to preserve the recovery property; the periodic re-read is what preserves it. It is this payload's cursor.

Two subject shapes, and the second is the first event addressed to a principal. A removal rides member:{channel_id} and reaches the removed user and the remaining members in one publish, because the removed user is still a member at the instant it goes out. An addition cannot: the instance holding the new member is not subscribed to that channel — that is precisely what is changing — so member:{env}:{user} exists. Every other subject in this system names a thing to hear about; this one names someone to tell.

Why not presence:{channel_id}'s grammar, or chan:'s. ADR-19's argument transfers unchanged: the message path is typed to messages at three points, and a membership frame riding it would mean editing the highest-volume path in the system for the lowest-volume traffic on it. Presence's subject carries a payload with a different schema and a different audience rule. Each fabric owning its grammar is now the pattern rather than the exception.

The interval is sixty seconds because of the connection budget, not because of a clause. NFR-SCL-01 budgets 10,000 connections per instance and the re-read is per connection, so sixty seconds is 167 requests per second per instance and five seconds would be 2,000. No SRS clause bounds a post-loss revocation. FR-RTM-10's five seconds is the budget for a working mechanism, met by the publish at 34–88 ms measured; sixty seconds bounds a mechanism that did not run at all. Reading the two as one number turns a backstop into a poll.

The honest consequence: under fabric loss a revocation lands within sixty seconds rather than five, exceeding FR-RTM-10 by 55 seconds. The revocation is guaranteed; what is bounded is how late it can be.

Rejected: publishing revocations through JetStream for durability. It is ADR-07's own rejected alternative with a stronger case — per-channel consumer management for a payload that is administrative and rare — and it would give the gateway a second broker client for one event kind. Rejected: holding the api's publish until every gateway acknowledges the subscription. That makes an administrative route wait on the fabric and invents a failure mode for news the reader can already fetch.

Rejected: reusing POST /internal/session for the re-read. It answers identity, limits and a connect policy that can refuse with a 402 when an environment is over its monthly allowance — so a routine refresh could fail for a reason unrelated to membership. GET /internal/memberships, described in the protocol package since chapter 3.2 and served by nothing since, asks the one question the backstop has.

Revisit when: the re-read's request rate becomes a constraint on connections per instance, or a clause is written that bounds a post-loss revocation — in which case sixty seconds is the number it has to argue with.

ADR-21 — Typing on a fourth subject grammar

Status: accepted (chapter 3.21) · extends ADR-19 and ADR-20 · Drivers: D2, D8

Typing signals publish on typing:{channel_id}. One shape, not two.

The chapter's plan assumed the opposite and the grep settled it. Typing was described as the one remaining real-time kind that could reuse chan:{channel_id} — per channel, ephemeral, carrying no membership question — so ADR-19's argument for a separate grammar looked inapplicable. That argument rests on a count, and the count was wrong: ADR-19 says the message path is typed to messages at three points, and re-deriving it returns eight lines covering seven — onDelivery, publish(message: Message) and a deliver type in fanout.ts, the messageCreatedSchema parse there, and three separate literal message.created sends in session.ts. Carrying a second kind on chan: means widening a type in four places, loosening a parse that currently rejects everything that is not a message, and editing the highest-volume path in the system to serve the lowest-volume traffic on it.

Three chapters have now reached this from three starting points, so it is the pattern rather than a judgement call: a fabric owns its subject grammar, and a kind that cannot share a payload type cannot share a subject.

One shape, where ADR-20 needed two. That record's second subject exists because an addition cannot ride the channel it adds you to — the instance holding the new member is not subscribed to it yet. Typing has no such case: a signal is only ever interesting to people already in the channel, and a member who cannot hear the subject has nothing to be told.

Rejected: an enveloped payload on chan:. ADR-19 rejected the same thing for presence and the objection is stronger here — it puts a discriminated-union parse on every message every instance receives, and during a rolling deploy an old instance logs fanout.invalid_payload for every keystroke on every channel. Typing is higher frequency than presence by orders of magnitude.

Rejected: a bidirectional typing frame. Reusing the published outbound frame for the client's signal would let a client name a user, which is what chapter 3.12's direction gauntlet forbids. The inbound frame is typing.send and carries a channel and nothing else; the connection supplies the identity.

Revisit when: a fifth kind arrives whose payload the typing fabric could carry unchanged — at which point the question is whether two kinds share one grammar, not whether typing should have moved to chan:.

ADR-22 — The typing expiry belongs to the receiving client

Status: accepted (chapter 3.21) · Drivers: D2, D8 · qualifies FR-RTM-08

FR-RTM-08 reads "Typing indicators shall expire automatically after 5 seconds without renewal and shall not be persisted." The platform cannot keep the first half, and this record is where that is said rather than discovered.

typingSchema has published { channel, user } since chapter 1.3 — no state field, no deadline — so there is no frame with which to end an indicator, and nothing anywhere knows one exists. No table, no Redis key, no server timer. A server that does not know an indicator started cannot announce that it stopped. The five seconds therefore live in the receiving client, counted from the last frame for each (channel, user).

What the platform owes, and does: it emits a typing frame when a member signals, and it stops emitting when they stop. The gateway holds a two-second debounce per connection and channel so a keystroke is not a publish — 2.5 renewals per expiry window, so one dropped publish does not make an indicator flicker.

The second half of the clause is met absolutely. "Shall not be persisted" is true because nothing is stored anywhere, which is a stronger property than a TTL would have given.

Why not add a state field and a stop frame. It would edit a published schema that frames.test.ts asserts and twenty chapters of clients parse, to add a message whose loss is unrecoverable: a dropped typing.stop leaves an indicator showing for ever, where a dropped renewal self-corrects within one interval. A lost typing frame converges on the truth; a lost stop frame converges on a lie. Chapter 3.20 took the opposite decision for membership for exactly this reason, and the two records are the same argument with the inputs reversed.

The honest consequence: a customer implementing a client from the published documents alone will not expire an indicator, because FR-RTM-08's plain reading puts the timer on the server. This ADR is the correction, and the clause is unchanged — a requirement is the customer's contract and a chapter does not rewrite one to match its code.

Revisit when: an SDK exists in this repository. The timer would then have a home the platform owns, and "the client" would stop meaning "code we do not control".

ADR-23 — Five slot keys, not a sorted set

Status: accepted (chapter 3.22) · Drivers: D2, D4, D8 · supersedes §6.3's conn:{env}:{user} row

FR-RTM-09 caps a user at five concurrent connections. §6.3 has carried a remedy since the first draft — a sorted set scored by heartbeat time, pruned with ZREMRANGEBYSCORE on read — and this record rejects it. The row was right about the defect and wrong about the fix.

A cap is a claim, not a count. Feature 040's FR-013 requires that two connections arriving at the same instant cannot both take the fifth place, which needs an atomic check-and-insert. A sorted set has none. ZADD then ZCARD is check-then-act. Add-then-verify — add, count, remove if over — refuses both of two connections arriving at four held: safe, and wrong. The atomic version needs Lua.

Constitution VII permits a second language only with a superseding ADR carrying profiling evidence, and this lane cannot produce it. The largest fixture in the repository holds five channels; NFR-SCL-01 asks about ten thousand connections per instance and stands undischarged (R2). An ADR arguing for Lua from a five-channel fixture would be arguing from nothing. grep for .eval(, defineCommand and .multi( returns zero across the platform: no multi-command Redis operation has ever shipped here.

So the member becomes the key. conn:{env}:{user}:{slot} for slots 0 to 4, claimed with SET NX PX, renewed with SET IFEQ PX, released with a conditional one-millisecond tombstone. Three commands, every one of them conditional, and the TTL is the only unconditional way a place is freed. That is the defect §6.3's row recorded — a Redis TTL is per key and not per set member — repaired by construction rather than worked around: when each member is a key, the per-key TTL is per member.

The cost, stated rather than hidden. Counting a user's connections is five reads instead of one ZCARD, and the count arrives as a by-product of the walk rather than as a query. Nothing in this chapter needs the count without also claiming a place, so nothing pays it.

Revisit when: a chapter needs a user's count without claiming a slot — an admin API, a dashboard, a support tool answering "why can this person not connect". Five reads is the wrong shape for that, and by then the load test R2 has been owed since the first draft will have produced the evidence Constitution VII asks for.

ADR-24 — Message revisions take a fifth subject grammar

Status: accepted (chapter 3.23) · Drivers: D2, D8 · qualifies FR-RTM-05, extends §6.3's pub/sub rows

FR-RTM-05 names six real-time event kinds. message.updated and message.deleted have been in the published protocol since chapter 1.3 with nothing emitting them, and chapter 3.23 gives them producers. Neither can reach a socket on the subject that already exists.

chan:{channel_id} carries a wire frame's payload rather than a shape of its own — §6.3's row and the code's own comment both say so — and that payload is a Message. Two consequences:

A deletion is not a Message. messageSchema.text is z.string() and a tombstone has no text, which is the same constraint that gave message.deleted its own frame payload in the same chapter. It cannot ride chan: even in principle.

An edit is a Message and still cannot ride it. The kind was never on the fabric: the receiving gateway stamped type: "message.created" at the call site, so an edit arriving on chan: is indistinguishable from a creation. Putting the kind in chan:'s payload would widen a grammar four chapters have treated as message-shaped, and would make every existing consumer branch on a field it has never seen.

A kind that cannot share a payload type cannot share a subject. ADR-19 took presence:{channel_id} on that argument, ADR-20 took member:{channel_id} and member:{env}:{user}, ADR-21 took typing:{channel_id}. This is the fourth application of one rule, which is why it is stated as a rule rather than re-derived here.

One subject, not two. revision:{channel_id} carries both mutations with a discriminator — kind: "updated" | "deleted" — following ADR-20's change: "added" | "removed" rather than ADR-19's single-purpose subject. An edit and a deletion are two things that happen to one message; a receiver wants both or neither, and two subjects would double the subscription bookkeeping for a distinction the payload already makes.

Revisit when: a mutation arrives that is not per-channel — a moderation sweep across an environment, or a retention job deleting by age. member:{env}:{user} exists because membership found exactly that case, and this grammar would need its principal-addressed sibling for the same reason.

ADR-25 — A measured threshold for consolidating the subject grammars

Status: accepted (2026-09-06) · Drivers: D2, D8 · bounds ADR-19, ADR-20, ADR-21, ADR-24 · discharges NFR-SCL-01

Five subject grammars carry real-time events between instances, each admitted by the same rule: a kind that cannot share a payload type cannot share a subject. The rule was sound and unbounded — applied indefinitely it ends somewhere nobody chose — and the cost of a grammar could not be stated, because NFR-SCL-01 had been P1 and unverified since v1.0.

Measured (docs/11-scalability-measurement-2026-09-06.md): one gateway instance sustains 10,000 concurrent connections for 90 seconds past two ping intervals at 160 MB, and 20,000 also holds. The subscription cost is exactly 5 × channels + 1 × connected users — 60,000 subjects at one channel per user, costing Redis 18 MB, with the gateway's RSS indistinguishable from the 11,000-subject case.

So the grammars are not what costs. A sixth is about 10,000 subjects and 3 MB in the worst ratio. Consolidating onto a typed envelope would buy four SUBSCRIBEs per channel and spend the property the five grammars exist to protect: a receiver subscribing to what it wants rather than to every kind on the channel.

The rule stands, bounded. Consolidate when per-channel SUBSCRIBEs would exceed six, or when a gateway's projected subject count exceeds 250,000 by the formula above at the deployment's own channel-to-user ratio. The projection is the operative half: which term dominates is a property of the customer's data, not of the platform — the five grammars are 9% of subjects at 200 channels and 83% at one channel per user, so a threshold expressed as a raw count would be wrong for half of all deployments.

A sixth grammar is pre-approved at this scale; a seventh reopens the record. Full argument, options and the measurement table: ADR-25 in 06-adr-deep-dives.md.

ADR-26 — The api serves a customer request from the analytical store

Status: accepted (2026-09-17) · Drivers: D2, D5 · bounds ADR-06 · discharges FR-ANL-07, FR-DSH-03 · constrained by constitution III

GET /v1/request-log reads ClickHouse on the request path, with a customer waiting. Chapter 4.7 put the same store behind a nightly reconciler, where an unbounded wait costs a job that was slow anyway; this is the first time the platform's availability depends on the analytical store answering, and it invents a failure mode the platform did not have — a 503 on a customer route because a store the customer has never heard of is slow.

Constitution III is why it is the analytical store and not the operational one: "billing, metering, and dashboard analytics read only from the analytical store." A customer-facing request log is dashboard analytics, and it is the clause's central case rather than an exception to it. The producer writes nowhere else — api_requests exists in ClickHouse and has no Postgres counterpart — so there is no second source to choose.

The rejected alternatives are measurements rather than opinions.

A client-side deadline alone. AbortSignal.timeout stops the caller waiting and leaves the query running: ClickHouse keeps executing, so a tenant retrying a slow page accumulates server-side work — the amplification a deadline exists to prevent. Measured: SETTINGS max_execution_time = 1 on a long scan answers HTTP 408 in 1.002 s with Code: 159 … (TIMEOUT_EXCEEDED). Both halves ship, and the server's limit is the shorter one so its refusal wins the race and the route receives a code it can map; the other ordering yields an AbortError carrying nothing.

LIMIT BY instead of FINAL. It deduplicates without a merge and it changes what LIMIT counts, so a page's size stops meaning what the contract says. FINAL costs 3 ms against 2 ms at seven parts — three runs a side, identical every time, the same rows read — and the table without it returns a redelivered request twice, which the surface's own test drives red.

Serving the log from Postgres. Constitution III's first sentence forbids it, and there is nothing to serve: the record was never written there.

The reversal condition, which is the part an ADR is usually missing. Undo this when the analytical store's availability appears in the API's error budget — that is, when analytics_unavailable stops being a rare answer on one route and starts being a number somebody reports. The 503 analytics_unavailable refusal is the instrument that makes it visible, and it was added in the same chapter as the route precisely so the signal exists before the decision needs reviewing. At that point the options are a cache in front of the read, a Postgres mirror of the log accepting constitution III's amendment, or removing the surface.

What it does not reverse on: the store being slow. A slow page inside the deadline is the trade this record accepts.

Full argument, the three rejected options and the measurements: ADR-26 in 06-adr-deep-dives.md.

ADR-27 — The integration gate reports which suites ran, and every lane runs

Status: accepted (2026-09-17) · Drivers: D8 · discharges constitution VI's gating clause · supports FR-ANL-06's verification method (T)

pnpm test:integration has been red on every run since chapter 4.4 and stopped scheduling at the first failure, so the gateway's twelve suites, the e2e journey, the ingester, the dispatcher and the harness went unexecuted — and a planted drift in the reconciler could not change the gate's colour, because the gate was already that colour. The line it printed counts turbo tasks, not suites: eighteen planned is nine builds and nine test tasks, three of which are packages with no integration script, and the total does not reproduce run to run (7 of 9, 8 of 10, 7 of 11 across three chapters against the same tree).

The gate now runs scripts/integration-gate.mjs: the lane list and suite count come from the tree, turbo runs with --continue, and each lane's own vitest summary is read back. It exits non-zero when a lane fails, when a lane reports no summary at all, or when fewer suites execute than the tree holds.

The six failures are resolved rather than excluded, and --filter could not have excluded them anyway — it selects packages, and the five request-log.itest.ts reds sit inside @relay/api beside the reconciler's own suite. The lane configures the two platform credentials (which also turned on three isolation-gauntlet attacks that had been returning at their first line and reporting green), and request-log.itest.ts spawns the ingester it needs rather than the stack running one forever.

AND THE GATE'S ACCOUNT HAS A HOLE THE GATE CANNOT SEE (added chapter 4.17). The lane list comes from the tree, which is what makes the suite count honest — and packages/outsider is not in it. That package declares no @relay/* dependency and talks to a composed stack over HTTP and WebSocket, so it is reached by no local lane at all: not pnpm test, not pnpm test:integration, and not pnpm coverage, which excludes it at vitest.coverage.config.mts:98. Its only runner is CI's separate sealed job, which builds two Node images and starts --profile services.

That matters beyond bookkeeping, because the sealed job is the only thing anywhere that exercises the deployed media worker. Every other check of the verification path stands in for the container: media-worker's own suite runs the sweep in process, the api's state machine tests call recordMediaVerdict directly, and the delivery gate's tests set states with SQL. So the service that decides whether a customer's upload is readable has exactly one automated consumer, it runs in a job most contributors never trigger locally, and it contributes nothing to the coverage number — a file that is 100% covered by the coverage lane and a file the deployed worker runs are two different claims about two different sets of code. Chapter 4.17 is where that job gained an assertion that fails when the worker stops.

One level up is not fixed and is not claimed. ci.yml's tutorial job ends with pnpm check:fences. That exited 1 at a standing 110 problems on every push from feature 045 until feature 055 took the chain to 0, where it has stayed; the sentence here said "on every push" for eight chapters after it stopped being true, and is corrected at chapter 4.17 rather than deleted, because the decision it records — that the series owns the threshold, not this document — still stands.

Full argument, the four rejected options and the measurements: ADR-27 in 06-adr-deep-dives.md.

ADR-28 — FR-ANL-06's daily job has no runner, and that is recorded rather than built

Status: accepted (2026-09-17) · Drivers: D5 · constrained by constitution III · amends nothing in FR-ANL-06's bound

"…within 0.1%, verified by a daily job that alerts on breach" is three requirements. The comparison exists (chapter 4.7). The alert has no mechanism (SRS 1.14). And the daily job has no runner at all — zero occurrences of reconcile-usage in either repository's package.json, in turbo.json, in ci.yml or in any shell script, and ci.yml triggers on push and pull_request with no schedule:. The platform runs five background relays, so the pattern exists; this is the one recurring job built as a hand-run script.

A sixth relay was costed rather than dismissed: feature 030 measured what the other five do to a test lane, and every analytical suite would gain another flag. What it would buy today is a daily sweep reporting no-data for every tenant, because no environment in this platform has both sides of the comparison — 2,440 usage_periods rows over 2,330 environments, 7 rollup rows over 4 environment ids, zero in both.

So the absence is recorded and the milestone's claims are scoped to what runs. docs/12 §2.3 substituted "the lane, every run" for "daily" without naming the substitution, and a per-push check on a planted fixture runs more often than daily while reading no real tenant — a different claim rather than a stronger one.

Full argument, the three options and the measurements: ADR-28 in 06-adr-deep-dives.md.

ADR-29 — A fence claims a file or declares it is not one, and the checker can print its own state

Status: accepted (2026-09-18) · Drivers: D1 · constitution VI's quickstart clause · changes no threshold, no exemption and no exit code

pnpm check:fences had reported 110 problems since feature 045, and ci.yml's tutorial job ends with that command, so its result reflected nothing for nine chapters. Feature 054 costed five ways out. This is the one taken: repair the chain, and change what the checker can print rather than what it accepts.

Two changes, and they are one decision about what a titled fence claims.

A title is a whole-body claim, so a block that is not a file says so. Eleven fences carried a prose phrase — "the typo, now", "42P01" — and all eleven are lang=text command output. The checker already skips a title containing (excerpt), in both its loops, and 222 titles in the series already carried it. Declaring these eleven applies a convention rather than inventing one. Rejected: teaching the checker that lang=text is never a file, which exempts a class without anybody reading its members; retitling to a real path, which would make the claim false rather than absent; removing the titles, which moves them into the 360 fences no gate reads at all.

And a hunk must be generated against the state the checker replays, so the checker can write that state out. --dump <dir> [--at <page>] adds an output mode: no threshold, no exemption, no change to any exit code, and --at replays separately so the dump cannot alter what the check reports. It exists because the chain's state is not the repository's — vitest.coverage.config.mts replays to 318 lines where the tree holds 1,182 — so a hunk written with git diff is rejected for reasons neither tool explains. Rejected: a separate script importing the checker's internals, which is a top-level program with no exports; and the throwaway copy fence-chain rule 1a asks for, which exits 0 having replayed nothing when it is run from anywhere else, because the platform path is resolved from the script's own location.

Reversal condition. If a later feature teaches the checker to distinguish a command-output fence from a file fence by something other than its title, the eleven declarations become redundant and should be removed in that change. If --dump is used fewer than five times in a year, fold it back into a throwaway and delete the flag.

Full argument, the measurements and what the repair cost: ADR-29 in 06-adr-deep-dives.md.

ADR-30 — Relay signs its own presigned URLs

Status: accepted (2026-09-19) · Drivers: D3 · ADR-13 · NFR-SCL-01's memory budget · dependency count moves by zero

ADR-13 chose the pattern — clients upload straight to object storage against a URL Relay signs, so media bytes never transit Relay compute. It did not say what does the signing, and hosted media is the chapter that has to.

node:crypto, twenty-eight lines. AWS Signature Version 4 for a presigned URL is five HMAC-SHA256 rounds over a canonical request, and every input is a string Relay already has. The UNSIGNED-PAYLOAD literal is what makes it work without the body: the client sends bytes Relay never sees, so the signature cannot cover them.

Rejected: @aws-sdk/client-s3 + @aws-sdk/s3-request-presigner. Two packages and their transitive tree for one function, on a service with a 160 MB working-set budget that was measured at 157. The SDK is the right answer for a codebase that also lists, copies, tags and multipart-uploads; this one signs a PUT and a HEAD.

Rejected: minio. One package rather than two, and a smaller tree — but it binds the signature to one vendor's client at the moment the platform is choosing a store it can replace. The canonical-request algorithm is S3's and is what every compatible store implements; a client library is an implementation of it with a name attached.

What the decision is not. It is not a claim that signing is easy. A wrong signature is a bare 403 SignatureDoesNotMatch with no indication of which field was wrong, and a consistently wrong one passes every unit test you can write about shape. The acceptance is a running store answering nine questions — signed PUT 200, signed GET 200, unsigned GET 403, expired refused from the store's own clock, tampered 403 — and those nine are an integration suite rather than a one-off probe for that reason.

And the cost that is not the dependency. A presigned URL needs no contact with the store, so the api never learns the store is down and feature 056's FR-017 refusal needed a round trip built for it: +1.524 ms at p50, +24.1% on the happy path, 200 samples a side.

Reversal condition. If a later chapter needs listing, copying, lifecycle rules or multipart uploads, the SDK's tree stops being overhead for one function and this decision should be re-taken as a whole rather than extended one signer at a time.

Full argument, the three options and the measurements: ADR-30 in 06-adr-deep-dives.md.

ADR-31 — The media worker is a fifth service, argued against §4.2's own table

Status: Accepted · Date: 2026-09-26 · Chapter: 4.13

Context. FR-MED-03 and FR-MED-04 require every uploaded object to be verified against its declaration and virus-scanned. Constitution VII asks a new service to justify itself against §4.2's "deliberately not a separate service" table, and no artifact in this feature had named that clause until the plan's constitution check was re-read — VII was engaged twice and only the language half (§7.3) was written down.

Decision. A fifth service, services/media-worker, packaged as a container behind --profile services.

The argument, against §4.2's three merge criteria, all of which fail:

§4.2's criterionThe media worker
Same datastoreNo. It opens the object store, which no other process in this platform opens. The api signs URLs for it and never contacts it except to probe reachability and create a bucket.
Same transactionsNo. It holds none. ADR-04 keeps it off Postgres entirely: its two questions travel the internal seam as HTTP, exactly as the dispatcher's do.
A network hop is pure latencyNo. The work is CPU-bound and off the request path. Streaming 100 MB through a signature engine costs 204 ms — measured — and the client is not waiting for it.

And the merge would have a cost the table's other rows do not. The api would hold the scanner's socket and stream customer bytes through its own heap, which is the process that also serves every tenant's requests. docs/11 measures the gateway at 157 MB against a 160 MB budget; buffering one 100 MB object costs 142.7 MB of RSS — measured, 1.4× the object — so a merged verifier would put a variable, caller-controlled allocation inside the request path's process.

Revisit when: the scan is no longer the dominant cost. It is 52% of the work at 4 KiB and 97.7% at 100 MB today. If verification became a header read — a store that scanned on write, or a scanner that took a URL rather than a stream — the CPU criterion would flip and this should be re-taken rather than extended.

What this does not claim. The worker is not horizontally scaled today and needs no lease to become so: UPDATE … WHERE state = 'pending' is a compare-and-set, so a second instance loses the race and is told it lost. That is a property of the seam rather than of the packaging.

ADR-32 — A program Relay addresses over a socket is not a program Relay is implemented in

Status: Accepted · Date: 2026-09-26 · Chapter: 4.13

Context. Constitution VII says this platform is TypeScript, and docs/12 §7.3 asks whether a virus scanner breaks that. ADR-01 named the case in advance.

Decision. ClamAV is a container this platform speaks to over TCP, and the clause is not engaged. The worker that talks to it is TypeScript.

The line. Relay already addresses five programs it is not written in — Postgres (C), Redis (C), NATS (Go), MinIO (Go) and ClickHouse (C++) — in four languages. Each is reached over a socket with a documented protocol and none of them is a violation of VII, because the clause governs what Relay is implemented in and not what Relay talks to. ClamAV's INSTREAM is the same relationship: zINSTREAM\0, a four-byte length before each chunk, a zero length to end. Twenty-two lines, no client library, on ADR-30's ratio.

What VII would forbid is writing the media worker itself in Go — a second language in the build, a second test runner, a second dependency manifest and a second set of people who can change it. That is the case the clause is about, and this decision does not approach it.

Revisit when: a verification step cannot be expressed as a conversation with a separate program. Duration probing is the near case — four container parsers with MP3 VBR as a genuinely hard one — and if it ever needs ffprobe, that is a sixth such program and not a sixth language. What would engage VII is needing to link one.

ADR-33 — A media transition takes the revision fabric's third arm, not a sixth grammar

Status: Accepted · Date: 2026-09-28 · Chapter: 4.14

Context. FR-MED-07 requires a media.updated event on every channel referencing an object when it leaves pending. docs/12 §7.4 asks whether that takes a sixth subject grammar. Five exist — chan, revision, presence, typing, member — each admitted by ADR-19's rule: a kind that cannot share a payload type cannot share a subject.

Decision. A third arm on revisionFabricSchema, published on the existing revision:{channel_id}. Per-channel SUBSCRIBEs stay at five.

The arithmetic, which ADR-25 had already written. That record sets the consolidation threshold at per-channel SUBSCRIBEs exceeding six, or a projected subject count over 250,000 under 5 × channels + 1 × connected users. A sixth grammar is therefore permitted — six does not exceed six — and it would spend the entire remaining headroom on a kind whose subscriber set is identical to one that already exists. fanout.ts subscribes chan: and revision: in one call under one reference count and calls them co-extensive by construction. This arm keeps the projection unchanged.

Rejected: widen chan:{channel_id}. Refused by ADR-24 in writing — everything on that subject IS a creation, and the kind was never on the fabric.

Rejected: re-deliver the message as message.updated. The cheapest shape available: no new subject, no new frame, and FR-MED-07's state is read at serve time so a re-read message carries it for free. It dies on a field that does not exist. messageSchema is {id, channel, seq, user, text, attachments, created_at} — there is no edited_at — so a client could not tell an attachment resolving from an author editing, and every verified photo would render its message as edited by somebody who did not touch it. That is ADR-24's own objection to putting edits on chan:, one level up.

Admitted under ADR-20's test, not as an exception to ADR-19's rule. ADR-20 put two payload types on one subject with a discriminator because an edit and a deletion are two things that happen to one message, a receiver subscribes to both or neither. The same holds here: a connection that wants a channel's messages wants to know when one of their attachments resolves.

Costs, stated. The subject is named revision: and carries something that is not a message revision; its contract becomes something changed about what this channel's messages show. Renaming it is a wire change and a fence-chain change across every chapter that publishes the file, so the name stays and the module says so. And reusing a subject means reusing its publisher: publishRevision derived its subject from revision.message.channel, which this arm does not have. Eight production sites across two services reached through .message; they delegate to channelOfRevision in the module that owns the grammar, whose switch is exhaustive so a fourth arm is a type error in one place.

Reversal condition. A seventh real-time kind, or a deployment whose projected subject count approaches 250,000 — at which point ADR-25's typed envelope is the change, and this arm is already half of it.


ADR-34 — A native image library is a dependency, not a second language

Status: Accepted · Date: 2026-09-29 · Chapter: 4.15

Context. FR-MED-05 requires a thumbnail for every uploaded image. ALLOWED_TYPES admits four image formats and nothing in this platform can decode any of them: the media worker's runtime dependencies were two workspace packages, and dimensions.ts reads headers — a PNG IHDR, a GIF screen descriptor, a WebP VP8, a JPEG SOF0 — without touching a pixel. Constitution VII says one language across services, and ADR-01 requires a superseding ADR with profiling evidence to introduce a second.

Decision. sharp — libvips behind an npm package with prebuilt platform binaries — as a runtime dependency of services/media-worker. It is the workspace's first native binary in a shipped service. Clause VII is not engaged.

Why the clause is not engaged, and ADR-32's argument does not reach this. ADR-32 settled that ClamAV does not make this a polyglot platform, and its reasoning is specific: Relay addresses five programs it is not written in — Postgres, Redis, NATS, MinIO, ClickHouse — "each reached over a socket with a documented protocol", and the clause governs what Relay is implemented in and not what Relay talks to. A linked library is a different relationship and cannot borrow that sentence. The argument it takes instead: the worker is a TypeScript program, every line of it is TypeScript, and calling a native module from it is the same relationship as calling Node's own JSON parser, which is C. What VII forbids is a second language this team writes and maintains. No .c file enters this repository.

Alternatives, measured rather than argued. Installed size is the delta on /usr + /lib inside node:22-alpine, which is what the worker's Dockerfile builds FROM; time is p50 of seven runs turning one 1920x1080 JPEG into a 320-bounded WebP:

optionaddedjpeg/png/gif/webpp50
sharp, in-process30,380,799 Byes15.2 ms
ImageMagick, one subprocess per object28,936,284 Byes35.8 ms
ffmpeg, subprocess+113,994,336 Byes, badly—
a sixth containera whole servicedepends—
pure TypeScript0 BPNG only—

The size did not decide it. 30.4 MB against 28.9 MB is 5% on a choice between two fundamentally different relationships, and anyone re-running this should expect them to stay close. What decided it is the 2.4x and the shape: the subprocess pays its extra 20 ms as process spawn on every object, in a service whose whole job is a sweep over a backlog. Pure TypeScript is out on coverage rather than effort — node:zlib gives inflate, which reaches one of four formats, and JPEG needs a DCT decoder, GIF LZW and WebP VP8. A sixth container fails §4.2's "deliberately not a separate service" test on all three columns and engages ADR-31 for a job with no independent scaling story: the resizer would sit behind the only service that reads the bytes, reading the same objects, with the same lifecycle.

The video half is not built. A poster frame needs a video decoder, and the only tool for it costs 113,994,336 B — 3.75x the image half — for the harder half of a clause whose easier half (FR-MED-04's duration) this platform already declined at revision 1.20. SRS 1.22 records it as unmet by decision.

Costs, stated. The worker's image grows from 245 MB to 279 MB, and its dependency count from two workspace packages to three entries. The workspace's runtime dependency total moves for the first time in Part 4: 31 entries to 32, third-party 13 to 14. A native binary means an install that can fail per-platform — mitigated, not eliminated, by the fact that pnpm install runs inside the target image and the lockfile records all 26 platform variants including linuxmusl-x64, both verified rather than assumed.

Reversal condition. If a second clause needs video, ffmpeg's 114 MB buys two things instead of one and the subprocess option is re-priced against a sharp that would then be redundant. And if the image path ever needs a format libvips is not built with, the question is whether the binary changes or the allow-list does — which is a decision about ALLOWED_TYPES, not about this dependency.

10. Risks and technical debt register

#Risk / debtExposureMitigation / trigger
R1Postgres is the single point of failure for writesFull write outage (see failure matrix)Accepted for v1: managed HA, PITR (NFR-REL-06/07). Trigger for revisit: availability SLO tightens past 99.9%
R2Node gateway connection ceiling unvalidated — NFR-SCL-01's 10k/instance is a budget, not a measurementArchitecture sizing wrong by ×2–5Load-test in Phase 1 week 1, before any dependent decision hardens (this is the SAD's single most urgent action item)
R3Channel-list subscription churn — a user in hundreds of channels makes connect/subscribe expensiveResume latency (NFR-PRF-05) degrades for power usersCap channels-per-user in v1 (matches FR-CHN-07 spirit); wildcard-subject scheme sketched if needed
R4Outbox relay is a serial choke pointEvent lag under burstRelay is horizontally scalable via SKIP LOCKED batching; alarm on outbox depth (NFR-OBS-04)
R5At-least-once everywhere requires consumer disciplineA future consumer forgets to dedupe → double webhooks / double meteringConsumer template with dedup built in; reconciliation job (FR-ANL-06) as the tripwire
R6Retention deletion vs. partitioning — per-environment retention (FR-MOD-06) doesn't align with monthly partitionsBulk deletes on mixed partitionsv1: row deletes off-peak per environment; debt noted — env-major partitioning if tenants grow large
R7Single-language monoculture (ADR-01)CPU-bound hot spots have no escape hatch in-languageIsolate HMAC/crypto behind an interface; a Rust/Go sidecar is a contained swap if profiling demands it
R8Customer-hosted emoji images — Relay serves URLs it does not control (FR-EMJ design note 1)Broken/slow images degrade perceived quality; malicious URL swaps after moderation review alter a record's appearance (though never its text)Document CDN/caching responsibility; resolution map is versioned so Priya's tooling can pin the resolution seen at review time; revisit trigger: emoji images may now optionally use hosted media (ADR-13), which removes this class entirely for customers who opt in
R9Hosted media liability surface — Relay now stores user-uploaded bytes: illegal content, scanner misses, storage-cost runawayLegal exposure; cost growth decoupled from message volumeMandatory scan gate (ADR-14) with audit trail; per-kind size caps + per-tenant storage quotas and spend caps (FR-MED-02, FR-RTL-06); DR-17 inventory reconciliation catches metering drift; abuse-report takedown path rides the existing moderation API (FR-MOD-02 + FR-MED-10's unlink-and-reap)

ADR-35 — The audit log is operational, its immutability is a trigger, and its timestamp is declared

FR-MOD-03 hides three decisions in one sentence, and none is the obvious one. The full argument, with its measurements and its reversal condition, is in docs/06-adr-deep-dives.md.

Postgres, not ClickHouse, and ADR-26 is the same driver running the other way. FR-005 of feature 064 requires the entry to commit with the action, and the action is a Postgres transaction. ADR-26's rejected option D was "mirror the log into Postgres and serve it from there" — right to reject for the request log, whose rows were already in ClickHouse, and the audit log's rows are already in Postgres. Serve the log from the store that already holds it selects a different store each time. Measured: with nats and clickhouse stopped, a ban and an unban both commit and both leave an entry.

A trigger, because REVOKE was measured inert. The api connects as relay and relay is a superuser, so REVOKE UPDATE, DELETE changes nothing — the grant would be in the migration and the table would be mutable, with nothing in the schema looking wrong. A BEFORE UPDATE OR DELETE trigger fires for a superuser and refuses both verbs. Row-level security answers a different question and a superuser bypasses it too; hash-chaining detects rather than prevents.

And the trigger was forbidden by a test whose rule was wider than its reason. no-trigger-in-migrations.test.ts asserts no migration creates one, scoped in its own comment to the test-database sentinel guard — a trigger that rejects the api's own sweeps. This one refuses writes the api must never make. Narrowed to the guard by name, with the narrowing asserted.

occurred_at timestamptz(3), because this is the platform's first keyset cursor over a Postgres timestamp: the column defaults to microseconds and toIso emits milliseconds, so an undeclared column loses every row inside the lost fraction at each page boundary. Unfixable once entries exist.

The claim is scoped and the chapter publishes the scope. SET session_replication_role = replica and DROP TRIGGER both succeed, both measured. The log is immutable to the application and to accident, not to somebody holding the database password — and NFR-SEC-10 wants an immutable trail for exactly that actor, so the two clauses are different artifacts and only one of them ships here. The reversal condition is the single change that would move both: a separate, non-superuser role for the application.

ADR-36 — A retention sweep is a compliance path, and the trigger names its one deleter

Chapter 4.20 took two decisions and the first is the one that unblocked the chapter. The full argument, with its measurements and its reversal conditions, is in docs/06-adr-deep-dives.md.

Decision 1 — a retention sweep IS a compliance path. FR-MOD-06 requires expired messages to be hard-deleted, and three documents forbade it: constitution II reserves hard deletion for the compliance path, FR-MSG-08 said only via the compliance deletion endpoint, and DR-06 said a deleted message retains its row. The constitution's own word is path, not endpoint — and a retention policy exists to keep a promise a customer made to an auditor, which is the same kind of obligation FR-MOD-04's erasure endpoint serves. So the rule hardest to change is the one that already permits this, and the two narrower SRS clauses are amended to match rather than the constitution. There are exactly two compliance paths and both are named. Rejected: soft expiry — clear text, keep the row — which satisfies all three clauses word for word, needs no ADR, and is the only reading that keeps every internal rule and still tells a compliance team their data is gone while the row is still there.

Decision 2 — the exception is a named flag in the trigger, not a privilege. Measured: the foreign key refuses the parent, message_edits_append_only refuses the children, and ON DELETE CASCADE is refused too, because a cascade issues an ordinary DELETE and a row trigger fires on it — the error names the generated statement. The only thing that worked unchanged was SET session_replication_role = replica, which is the hole ADR-35 published as the limit of its own guarantee and which disables every trigger in the session. So 0025 adds TG_OP = 'DELETE' AND current_setting('relay.expiring', true) = 'on', auditable in a way the bypass is not: the condition is in the trigger's body, in a migration, inside the fence chain, naming one verb on one table.

THIS SUPERSEDES ADR-35's SCOPE CLAUSE FOR message_edits RATHER THAN AMENDING IT, because constitution VII makes an accepted ADR immutable. The scope was immutable to the application and to accident, and not to somebody holding the database password; it is now immutable to the application except one named path, and to accident. audit_log's scope is untouched.

Reversal. Decision 1: if a later chapter needs hard deletion on a third path, compliance path has stopped being a category and the clause needs rewriting rather than reinterpreting. Decision 2: a separate non-superuser role makes privilege the mechanism and the exception unnecessary — ADR-35's own reversal condition, unchanged.


Every ADR above states its reversal condition or rejected alternatives. If a review disagrees with a decision, the productive move is to attack the driver, not the choice — the choices follow from D1–D8 fairly mechanically.

ADR-38 — A noun with a customer-supplied identifier is addressed by it

Summary. FR-USR-01 and ADR-18 say an end user's identity is whatever external_id the customer already had. This carries that to every noun: a noun with a customer-supplied identifier is addressed by it, a noun with only a Relay identifier is addressed by that, and where both could name the same thing the customer's wins. Exactly two tables carry one, measured from information_schema — users and channels — and the four that do not (messages, media_objects, webhook_endpoints, environments) are correctly addressed by uuid rather than by oversight.

A uuid stays the foreign key, the ordering key and the primary key; a text primary key across 216,922 messages costs more than it is worth. There is one identity and one internal key, and only one of them belongs on the wire. A key may stay on the wire where a route accepts it — the thirteen channel routes keep taking one, for 173 call sites' sake — and may not where none does, which is why members[].user_id is gone — chapter 4.22 swept every v1 response shape and removed the one value no route took.

Reversal condition. If a noun ever needs a customer-supplied identifier that cannot be relied on to be unique within a tenant, the first half fails for that noun. Uniqueness is what makes an identity addressable: channels_environment_id_external_id_unique and users_environment_id_external_id_unique are the constraints this rests on.

What it did not cover, and what still does not. The real-time surface addressed channels by uuid until chapter 4.23: FR-RTM-11 now states the rule for the socket, the session response carries pairs, and one function at the gateway's client edge renames on the way out while the subjects, the cursors' internal keying and the api-facing door keep the Relay identifier. Webhook payloads still carry channel_id as a Relay identifier (services/api/src/outbox/event.ts), on a boundary whose own comment says consumers get external ids — recorded, and nobody's chapter yet.

The argument is in docs/06-adr-deep-dives.md; this is the summary and that is the ADR.

← Back to the table of contents