Wavestar Technology · System architecture

Four modules. One registry spine. Language chosen per hotspot.

Wavestar runs four products on top of a shared identity and attestation spine. Each module is language-selected for its bottleneck: Rust for sub-2ms matching, Go for append-only log semantics, TypeScript for operator UX, Python for ML pipelines. NATS JetStream carries every cross-module event. OpenTelemetry traces every request end-to-end.

Read the protocol spec Security posture

Clearing engine: Rust · sub-2ms
Registry: Go + Trillian + CockroachDB
Terminal + Market: Next.js 15 · RSC
Event bus: NATS JetStream

System diagram

The picture, end to end.

Every external actor — operator, counterparty, regulator, observer — interacts with Wavestar through one of the four edges. Internally, every write funnels through the registry and the attestation network before settlement.

ascii

                      PUBLIC INTERNET  ::  TLS 1.3, mTLS for members
   +------------+        +-------------+       +------------+       +-----------+
   |  Operators |        |  Observers  |       | Regulators |       |  Public   |
   |  via API,  |        | 12 seats,   |       |  FCC/ITU   |       | anchor    |
   |  Terminal, |        | HSMs, mTLS  |       |  read-only |       | verifiers |
   |  Market    |        |             |       |            |       |           |
   +------+-----+        +------+------+       +-----+------+       +-----+-----+
          |                     |                    |                    |
          v                     v                    v                    v
   ============================= EDGE LAYER =========================================
    apps/api  (gRPC + REST)   |  Envoy/Istio mesh, OPA policy, JWT assertion
                              |  Tenant extraction (DID from session token)
   =================================================================================
          |                    |                    |                    |
          v                    v                    v                    v
   +-------------+     +---------------+    +--------------+     +-------------+
   | ORCH (Rust) |<--->| Registry (Go) |<-->| Attest (Go)  |<--->| AI (Python) |
   | Clearing    |     | Trillian +    |    | Quorum,      |     | Anomaly,    |
   | < 2ms match |     | CockroachDB   |    | BLS agg      |     | NL query    |
   +------+------+     +-------+-------+    +------+-------+     +------+------+
          |                    |                   |                    |
          +----- NATS JetStream -------------------+--------------------+
                 durable streams: clearing, registry, attest, audit

          |                    |                   |                    |
          v                    v                   v                    v
    +-----------+        +-----------+      +-----------+         +------------+
    | Postgres  |        | Cockroach |      | HSM pool  |         | Object     |
    | per-tenant|        | ledger    |      | + KMS     |         | store      |
    | sidecars  |        | spine     |      | key rot.  |         | (S3/GCS)   |
    +-----------+        +-----------+      +-----------+         +------------+

                OBSERVABILITY: OpenTelemetry -> Tempo (traces)
                                             -> Loki (logs)
                                             -> Mimir (metrics)

                ANCHORING (hourly):  Trillian STH  -> Ethereum L1
                                                  -> Bitcoin OP_RETURN

Module boundaries

Each module has one job. No cross-store reads.

Modules communicate by signed events on NATS or by typed gRPC calls across the mesh. Nothing reads another module’s database directly. The registry is the only module with write access to the Trillian log.

01
ORCH — clearing engine (Rust)
The low-latency matching core. Rust, async with Tokio, memory-mapped order books, lock-free for the hot path. Sub-2ms match latency p99. Novation happens here: every match produces a buyer-leg and seller-leg contract with Wavestar as the CCP. State lives in an in-memory log- structured store, with snapshots every 30 seconds to CockroachDB. Replay is deterministic.
02
Registry — identity + transparency log (Go)
The did:orbit resolver and the Trillian log wrapper. Go 1.22, gRPC service, CockroachDB for spanner-class consistency on the append path. Every DID document, every rulebook version, every cleared trade envelope is a leaf in the Trillian log; every leaf is reachable from a signed tree head (STH) anchored hourly to Ethereum and Bitcoin.
03
Attest — federated oracle (Go)
Orchestrates the 12-observer quorum. Go for its mature gRPC + NATS ecosystem and concurrency model. Issues attestation requests, collects BLS12-381 signature shares, aggregates 8-of-12 thresholds, writes the aggregate to the Trillian log. Rejects duplicate signers, wrong-group points, and mismatched messages at the aggregation step.
04
Terminal + Market — operator surfaces (Next.js 15)
The Bloomberg-class operator console and the MLS-style hosted-payload marketplace. Next.js 15 App Router, React Server Components, Tailwind, shadcn/ui. Read-path only — every mutation is a tenant-scoped gRPC call to the edge. No client-side DB access. Real-time updates come through NATS over a WebSocket gateway.
05
AI — anomaly detection + routing (Python 3.12)
Python 3.12, FastAPI, PyTorch. Runs settlement-routing inference (best-venue cash leg), anomaly detection (unusual counterparty behaviour, observer disagreement patterns), and the natural-language query surface the Terminal exposes. Never authoritative — the ML outputs are advisory, and every settlement decision still passes the deterministic rulebook.

Data flow — a cleared trade

The happy path, step by step.

ascii

  STEP 1:  Operator submits order         (Terminal -> apps/api -> ORCH)
           apps/api enforces auth + tenant scope; ORCH accepts into book.

  STEP 2:  Match                           (ORCH in-memory)
           Sub-2ms. Produces buyer-leg and seller-leg contracts with
           Wavestar as CCP. Assigns a contract DID.

  STEP 3:  Novation envelope               (ORCH -> Registry)
           COSE-signed envelope posted to the registry for log append.
           NATS event clearing.novated.{contractId} emitted.

  STEP 4:  Attestation request             (Registry -> Attest)
           On scheduled settlement window, Attest issues request to
           all 12 observers via NATS attest.request.{contractId}.

  STEP 5:  Quorum forms                    (Attest)
           8-of-12 BLS shares received within 200ms p99.
           Aggregate signature verified; written back as log leaf.

  STEP 6:  Atomic settlement               (ORCH + L2 escrow)
           Cash leg (USDC / Fedwire / SWIFT) releases against the
           attestation. Resource leg marked delivered. Both or neither.

  STEP 7:  Log append + anchor             (Registry -> Trillian)
           Settlement envelope appended. STH updates. Next hourly
           anchor commits STH root to Ethereum + Bitcoin.

  STEP 8:  Counterparty receives bundle    (apps/api -> Operator)
           COSE envelope + Merkle proof + observer sig + anchor refs.
           Operator verifies client-side. Trade is done.

Deployment topology

Active-active, multi-region, HA by construction.

Wavestar runs in three regions for v1: us-east (primary), eu-west, ap-southeast. Each region hosts the full module stack; CockroachDB and Trillian are multi-region; NATS JetStream clusters are per-region with leaf-node mirroring.

v1 deployment · production

Regions: us-east-2, eu-west-1, ap-southeast-1
Active-active for reads; clearing primary rotates quarterly under OIF oversight.
Orchestration: Kubernetes 1.30 · multi-region Helm overlays
EKS + GKE. Single set of charts; region-specific values files only.
Service mesh: Istio · mTLS between every pod
OPA policy for tenant-scoping and rate limits at the edge.
Clearing plane: Rust ORCH · dedicated nodes, pinned CPUs
Low-latency networking; no noisy neighbours. Sub-2ms match latency p99.
Registry store: CockroachDB 24.x · 9-node cluster
Spanner-class consistency. Region-local reads; global write quorum.
Transparency log: Trillian 1.6.x · log personality + signer on HSM
Log signer is air-gapped from the serving plane; signed STH pushed via mTLS RPC.
Event bus: NATS JetStream · per-region cluster + leaf mirrors
Durable streams for clearing, registry, attest, audit. Messages replicated cross-region for audit replay.
Object storage: S3 + GCS · dual-provider for disclosure bundles
SSE-KMS with customer-managed keys; lifecycle policy: infrequent-access after 30d, glacier after 1yr.
Secrets: HashiCorp Vault · HSM root · KMS per region
Operator signing keys live in HSM pools; application keys in KMS with 90-day rotation.
Observability: OpenTelemetry -> Tempo (traces), Loki (logs), Mimir (metrics)
Every request carries a W3C trace context. Retention: 14d hot, 90d warm, 7y cold for audit.
CDN + edge: Cloudflare for public site + resolver read path
DID resolver is cacheable (5min TTL). Clearing APIs bypass CDN; direct to regional gateway.

Language and library choices

Each tool earned its place.

01
Rust for clearing
Sub-2ms match latency p99 under sustained 12k trades/sec is not achievable in a GC’d language without heroic allocation management. Tokio async runtime, blst for BLS, ring for TLS.
02
Go for registry and Attest
Trillian is Google’s canonical Merkle-log implementation and it’s in Go. We extend it rather than reimplement. NATS, gRPC, and CockroachDB ecosystems are equally Go-native.
03
Next.js 15 for operator UX
React Server Components mean Terminal and Market render server-side by default; client-side JS is only where interactivity demands it. Shared @wavestar/ui design system. No shadcn blocks are load-bearing.
04
Python 3.12 for AI
PyTorch, FastAPI, pydantic v2. mypy --strict on every module. Models run on GPU nodes in their own region-local pool; inference is stateless and horizontally scalable.
05
CockroachDB for ledger spine
Strong consistency across regions without a single-writer bottleneck. Trillian’s storage contract matches its SQL semantics. Backups dual-destination (S3 + GCS).
06
NATS JetStream for events
Durable, replicated, subject-based. Cheaper to operate than Kafka at our scale, and the subject tree matches our module boundaries naturally.

Tenant isolation

Every query is scoped to the authenticated operator.

typescript

// packages/utils/src/db/tenant-scoped.ts  (summary)
//
// Every read-path query in Terminal, Market, and AI goes through
// this wrapper. The operator DID is extracted from the authenticated
// session token ONCE, at the edge, and threaded through every call.

import { withTenant, type TenantDid } from "@wavestar/utils/db";

// Standard usage - tenant DID injected by the middleware:
const trades = await withTenant(session.did, (tx) =>
  tx.sql`SELECT * FROM trades WHERE status = 'settled'`
);

// The wrapper rewrites the query to:
//   SELECT * FROM trades
//   WHERE tenant_did = $1 AND status = 'settled'
// and refuses execution if tenant_did is missing from the target table.

// Cross-tenant reads require explicit, logged opt-in:
import { withCrossTenant } from "@wavestar/utils/db";

const regulatorView = await withCrossTenant(
  { reason: "CFTC routine audit · ticket RT-2026-04-18", requestor: session.did },
  (tx) => tx.sql`SELECT ... FROM trades WHERE settled_at BETWEEN $1 AND $2`
);
// Emits audit.cross_tenant.read on NATS; logged to append-only audit stream.

Operating principle

Every module can be replaced in place. The registry spine cannot. Design every interface so that the clearing engine of 2031 is a different codebase than the one of 2026 — without any counterparty noticing.

ADR-0015·ORCH crate architecture

Operational targets

What we measure, what we publish.

Match latency p99: 1.8 ms
Attestation p99: 420 ms
Log append p99: 95 ms
Settlement finality: < 1.2 s
Regions active: 0
Uptime target: 99.995%
Recovery time (RTO): 5 min
Recovery point (RPO): 0 s

For platform engineers

Architecture documentation is open.

The protocol spec, the proto files, the helm charts for the self-hostable replica, and the runbooks for each module all live on the public developer portal.

Developer portal Security posture

Four modules. One registry spine. Language chosen per hotspot.

The picture, end to end.

Each module has one job. No cross-store reads.

ORCH — clearing engine (Rust)

Registry — identity + transparency log (Go)

Attest — federated oracle (Go)

Terminal + Market — operator surfaces (Next.js 15)

AI — anomaly detection + routing (Python 3.12)

The happy path, step by step.

Active-active, multi-region, HA by construction.

Each tool earned its place.

Rust for clearing

Go for registry and Attest

Next.js 15 for operator UX

Python 3.12 for AI

CockroachDB for ledger spine

NATS JetStream for events

Every query is scoped to the authenticated operator.

What we measure, what we publish.

Architecture documentation is open.