Conduit is the fleet's **single egress point** for every external API feed. It fetches an upstream once, stores the raw response **as-is** (byte-for-byte, no normalization), and fans it out to many internal consumers. It's a rate-limit shield, a cost shield, and a single egress point — "call an API once, use it many times."
- **Consumers:** [[navi]]'s traffic tiles (`tomtom_flow_tiles`) and all 13 of [[meshai]]'s hazard adapters (migrated 2026-07-15 — see [[conduit-operations]])
Everything Conduit brokers is a **source** — an addressable API identity, whether an internet feed or an internal app's output; the model makes no distinction. The `sources` table holds ~21 rows.
- **Ingest:** **pull** (Conduit fetches the upstream on cadence/on-demand) is live. **push** (an app POSTs a raw payload in) is planned, not yet exercised.
- **Store:** the latest raw payload per `(source, request_key)`, with a per-source freshness/TTL — a cache, not by itself a historical archive (see Retention below for the opt-in exception).
- **Serve:** a uniform `GET /up/{source}/{path}` returns the raw bytes byte-for-byte, no transformation.
- **Single-flight:** concurrent cache misses for the same `(source, request_key)` coalesce into exactly one upstream fetch; every waiting caller gets the same bytes back.
- **Topologies** — all fall out of the one model: `internet → conduit → app`, `app → conduit → app`, and chained (`internet → conduit → A → conduit → B`, where A registers its own output as a new source).
**Key property:** upstream calls scale with *unique resources × refresh rate*, not with the number of consumers — the 100th reader of the same resource costs zero additional upstream calls.
- **Serve-stale** — on quota-block, upstream 429, or 5xx/transport failure, Conduit serves the last-known-good cached copy instead of failing the caller (`X-Conduit-Stale: 1`).
- **Faithful 4xx passthrough** (PR #16, 2026-07-15) — a genuine upstream 4xx *except* 429 (e.g. TomTom flow's `400 "Point too far from nearest existing segment"`) is returned to the caller with its real status + body, uncached, not wrapped as a 502. Only 429/5xx/transport failures fall back to serve-stale-or-502. This unblocked the last meshai adapter (traffic) to migrate.
- **Hot-reload of sources** — a source add/edit/delete made through the GUI calls `Broker.set_sources()` and takes effect with no restart. A direct out-of-band SQL change to `sources` still needs `systemctl restart conduit` — the GUI path is the live one.
- **Retention engine** (`poller.py` + `store/history.py`) — opt-in per source (`retain=true`). A background `Poller` fetches retained sources on their `poll_interval_seconds` and appends changed raw payloads to `payload_history` (append-on-change, sha256-dedup). Idle by construction when nothing is retained. Read back via `GET /history/{source}` and `/history/{source}/{id}/body` (unauthenticated, mesh-internal). Purpose: accumulate raw feeds for later forecast/trend models.
- **Management GUI** (`gui/`) — built in meshai's visual language. argon2 operator auth + two-tier CSRF + schema-reflection forms harvested from central. Pages: sources CRUD (incl. headers/quota/retention fields), API keys, a Cmd-K command palette. Admin operator provisioned. `/up` and `/history` stay unauthenticated (mesh-internal, tiles-trust model); the GUI is the authenticated surface.
- **Keystore** (`keystore.py`, `crypto.py`) — AES-256-GCM `api_keys` by alias, under Conduit's own master key. Holds `tomtom`, `roads511`, and `firms` keys (harvested from central / provisioned) — meshai no longer holds any of these itself.
**Central is retired and dropped (2026-07-15) — Conduit replaced it.** Central's app services were stopped and disabled 2026-07-14 (zero live consumers remained), then on 2026-07-15 its database was archived to pi-nas (sha256-verified) and dropped (`DROP DATABASE central`, ~41 GB reclaimed); the shared Postgres instance was cleaned back to plain (TimescaleDB removed from `shared_preload_libraries`). Central is recoverable only from the pi-nas archive. Conduit's own `conduit` DB shares that same Postgres instance, which was never stopped.
Conduit was born by harvesting central's proven, decoupled pieces — the AES-256-GCM encrypted key store, the GUI auth/CSRF + schema-reflection form patterns, and the aiohttp+tenacity fetch idiom — while deliberately shedding central's NATS/JetStream, CloudEvents normalization, and enrichment pipeline.
- Single egress point for the whole fleet: [[navi]]'s traffic tiles plus all 13 [[meshai]] hazard adapters (NWS, SWPC ×4, ducting, WFIGS/fires ×2, FIRMS, avalanche, USGS streams, usgs_quake, tomtom_traffic, roads511, WZDx, satpass) — see [[conduit-operations]] for the migration record.
-`sources` table holds ~21 rows.
- Quota guard, serve-stale, faithful-4xx-passthrough, hot-reload, the retention engine, and the management GUI are all live (not planned).
- Keystore holds `tomtom`, `roads511`, `firms` — meshai no longer holds any of these keys directly.
- Central is gone; it is no longer a rollback target for anything.
