matt/echo6-docs
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

auto: docs sync 2026-07-16T12:00:15+00:00

Browse source 
Files changed: engine/.embcache.json engine/changelog.md engine/lint-report.md vault/docs/hardware/environment.md vault/docs/hardware/ip-allocation.md vault/docs/software/central.md vault/docs/software/conduit.md vault/runbooks/add-peertube-channel.md vault/runbooks/central-deploy-cutover.md vault/runbooks/conduit-operations.md vault/runbooks/peertube-remote-runner.md
This commit is contained in:
echo6-autocommit 2026-07-16 12:00:15 +00:00
commit 77b4715384
11 changed files with 54 additions and 48 deletions
Download patch file Download diff file Expand all files Collapse all files

14
vault/docs/software/conduit.md
View file

@ -6,11 +6,11 @@ tags:
aliases: []
related:
- [[conduit-operations]]
- [[recon-operations]]
- [[recon]]
- [[central]]
- [[navi]]
- [[caddy]]
- [[meshai]]
updated: 2026-07-15
- [[fleet-patch-audit]]
updated: 2026-07-16
---
# Conduit — Raw-API Broker
@ -63,11 +63,11 @@ Everything Conduit brokers is a **source** — an addressable API identity, whet
- **Single-flight coalescing** — concurrent misses for the same `(source, request_key)` → one upstream call.
- **Quota guard** (`quota.py`) — per-source day/minute/month caps enforced against a durable `upstream_calls` log, atomic per-source `asyncio.Lock` (single-process). Protects free-tier upstreams (e.g. TomTom's free plan).
- **Serve-stale** — on quota-block, upstream 429, or 5xx/transport failure, Conduit serves the last-known-good cached copy instead of failing the caller (`X-Conduit-Stale: 1`).
- **Faithful 4xx passthrough** (PR #16, 2026-07-15) — a genuine upstream 4xx *except* 429 (e.g. TomTom flow's `400 "Point too far from nearest existing segment"`) is returned to the caller with its real status + body, uncached, not wrapped as a 502. Only 429/5xx/transport failures fall back to serve-stale-or-502. This unblocked the last meshai adapter (traffic) to migrate.
- **Faithful 4xx passthrough** (PR #16, 2026-07-15) — a genuine upstream 4xx *except* 429 (e.g. TomTom flow's `400 "Point too far from nearest existing segment"`) is returned to the caller with its real status + body, uncached, not wrapped as a 502. Only 429/5xx/transport failures fall back to serve-stale-or-502. This unblocked the last [[meshai]] adapter (traffic) to migrate.
- **Hot-reload of sources** — a source add/edit/delete made through the GUI calls `Broker.set_sources()` and takes effect with no restart. A direct out-of-band SQL change to `sources` still needs `systemctl restart conduit` — the GUI path is the live one.
- **Retention engine** (`poller.py` + `store/history.py`) — opt-in per source (`retain=true`). A background `Poller` fetches retained sources on their `poll_interval_seconds` and appends changed raw payloads to `payload_history` (append-on-change, sha256-dedup). Idle by construction when nothing is retained. Read back via `GET /history/{source}` and `/history/{source}/{id}/body` (unauthenticated, mesh-internal). Purpose: accumulate raw feeds for later forecast/trend models.
- **Management GUI** (`gui/`) — built in meshai's visual language. argon2 operator auth + two-tier CSRF + schema-reflection forms harvested from central. Pages: sources CRUD (incl. headers/quota/retention fields), API keys, a Cmd-K command palette. Admin operator provisioned. `/up` and `/history` stay unauthenticated (mesh-internal, tiles-trust model); the GUI is the authenticated surface.
- **Keystore** (`keystore.py`, `crypto.py`) — AES-256-GCM `api_keys` by alias, under Conduit's own master key. Holds `tomtom`, `roads511`, and `firms` keys (harvested from central / provisioned) — meshai no longer holds any of these itself.
- **Keystore** (`keystore.py`, `crypto.py`) — AES-256-GCM `api_keys` by alias, under Conduit's own master key. Holds `tomtom`, `roads511`, and `firms` keys (harvested from [[central]] / provisioned) — meshai no longer holds any of these itself.
## Modules (`src/conduit/`)
@ -99,7 +99,7 @@ Pull-based deploy: authored/pushed from a cortex clone, CT 104 pulls via a **rea
## Relationship to [[central]]
**Central is retired and dropped (2026-07-15) — Conduit replaced it.** Central's app services were stopped and disabled 2026-07-14 (zero live consumers remained), then on 2026-07-15 its database was archived to pi-nas (sha256-verified) and dropped (`DROP DATABASE central`, ~41 GB reclaimed); the shared Postgres instance was cleaned back to plain (TimescaleDB removed from `shared_preload_libraries`). Central is recoverable only from the pi-nas archive. Conduit's own `conduit` DB shares that same Postgres instance, which was never stopped.
**Central is retired and dropped (2026-07-15) — Conduit replaced it.** Central's app [[services]] were stopped and disabled 2026-07-14 (zero live consumers remained), then on 2026-07-15 its database was archived to pi-nas (sha256-verified) and dropped (`DROP DATABASE central`, ~41 GB reclaimed); the shared Postgres instance was cleaned back to plain (TimescaleDB removed from `shared_preload_libraries`). Central is recoverable only from the pi-nas archive. Conduit's own `conduit` DB shares that same Postgres instance, which was never stopped.
Conduit was born by harvesting central's proven, decoupled pieces — the AES-256-GCM encrypted key store, the GUI auth/CSRF + schema-reflection form patterns, and the aiohttp+tenacity fetch idiom — while deliberately shedding central's NATS/JetStream, CloudEvents normalization, and enrichment pipeline.