auto: docs sync 2026-07-16T12:00:15+00:00
Files changed: engine/.embcache.json engine/changelog.md engine/lint-report.md vault/docs/hardware/environment.md vault/docs/hardware/ip-allocation.md vault/docs/software/central.md vault/docs/software/conduit.md vault/runbooks/add-peertube-channel.md vault/runbooks/central-deploy-cutover.md vault/runbooks/conduit-operations.md vault/runbooks/peertube-remote-runner.md
This commit is contained in:
parent
07c7169238
commit
77b4715384
11 changed files with 54 additions and 48 deletions
|
|
@ -7,15 +7,14 @@ aliases: []
|
|||
related:
|
||||
- [[central-deploy-cutover]]
|
||||
- [[services]]
|
||||
- [[navi]]
|
||||
- [[fleet-platform-baseline]]
|
||||
- [[caddy]]
|
||||
- [[conduit]]
|
||||
updated: 2026-07-15
|
||||
- [[fleet-platform-baseline]]
|
||||
- [[fleet-patch-audit]]
|
||||
updated: 2026-07-16
|
||||
---
|
||||
# central — Data-Hub Spine
|
||||
|
||||
> **RETIRED + DROPPED (2026-07-15).** Central has been replaced by [[conduit]]. Its app services were stopped and disabled 2026-07-14 (zero live consumers remained); on 2026-07-15 its database was archived to pi-nas (sha256-verified) and dropped (`DROP DATABASE central`, ~41 GB reclaimed), and the shared Postgres instance was cleaned back to plain (TimescaleDB removed). Central is recoverable only from the pi-nas archive. **Everything below this point is historical** — it describes how central worked while it was live, and is kept for reference only.
|
||||
> **RETIRED + DROPPED (2026-07-15).** Central has been replaced by [[conduit]]. Its app [[services]] were stopped and disabled 2026-07-14 (zero live consumers remained); on 2026-07-15 its database was archived to pi-nas (sha256-verified) and dropped (`DROP DATABASE central`, ~41 GB reclaimed), and the shared Postgres instance was cleaned back to plain (TimescaleDB removed). Central is recoverable only from the pi-nas archive. **Everything below this point is historical** — it describes how central worked while it was live, and is kept for reference only.
|
||||
|
||||
## Overview
|
||||
|
||||
|
|
@ -76,7 +75,7 @@ All three units are **enabled and active**; [[deployment]] survives reboot. Deps
|
|||
|
||||
Authenticated app with login/sessions/CSRF, first-run setup wizard, operator management, adapter configuration (incl. **no-code creation of generic REST/GeoJSON sources**, v0.15.0), stream viewer, **JetStream consumer management** (`/consumers` — view + delete consumers; central's own `archive-*` durable consumers are protected/non-deletable; CSRF + audit-logged), enrichment pipeline, monitoring-area management, API key management, audit log, and manual resend.
|
||||
|
||||
**Auth-exempt tile endpoints** (used by navi, verified HTTP 200):
|
||||
**Auth-exempt tile endpoints** (used by [[navi]], verified HTTP 200):
|
||||
|
||||
| Endpoint | Format |
|
||||
|----------|--------|
|
||||
|
|
@ -89,7 +88,7 @@ Authenticated app with login/sessions/CSRF, first-run setup wizard, operator man
|
|||
|
||||
navi-traffic (navi's in-VM :8421 extraction service) was **retired on 2026-05-26** and cut over to central. recon-vm's nginx (`/etc/nginx/sites-available/navi.echo6.co`, port 8440) proxied `^~ /api/traffic/` → `central.echo6.mesh:8000` with a 120s tile cache. navi-traffic:8421 is confirmed dead and disabled.
|
||||
|
||||
**As of 2026-07-14, navi's `/api/traffic/` tiles were repointed to [[conduit]]:** recon-vm's nginx now rewrites `^~ /api/traffic/` → `/up/tomtom_flow_tiles/...` and proxies to `central.echo6.mesh:8010` (Conduit, co-resident on the same CT 104 host). Central's own tile endpoint (`:8000/api/traffic/...`) still exists and still works, but is no longer on navi's hot path — it remains the rollback target if the Conduit cutover needs to be reverted. See [[conduit-operations]] for the cutover and rollback procedure.
|
||||
**As of 2026-07-14, navi's `/api/traffic/` tiles were repointed to [[conduit]]:** recon-vm's nginx now rewrites `^~ /api/traffic/` → `/up/tomtom_flow_tiles/...` and proxies to `central.echo6.mesh:8010` ([[conduit]], co-resident on the same CT 104 host). Central's own tile endpoint (`:8000/api/traffic/...`) still exists and still works, but is no longer on navi's hot path — it remains the rollback target if the Conduit cutover needs to be reverted. See [[conduit-operations]] for the cutover and rollback procedure.
|
||||
|
||||
## Dependencies
|
||||
|
||||
|
|
|
|||
|
|
@ -6,11 +6,11 @@ tags:
|
|||
aliases: []
|
||||
related:
|
||||
- [[conduit-operations]]
|
||||
- [[recon-operations]]
|
||||
- [[recon]]
|
||||
- [[central]]
|
||||
- [[navi]]
|
||||
- [[caddy]]
|
||||
- [[meshai]]
|
||||
updated: 2026-07-15
|
||||
- [[fleet-patch-audit]]
|
||||
updated: 2026-07-16
|
||||
---
|
||||
# Conduit — Raw-API Broker
|
||||
|
||||
|
|
@ -63,11 +63,11 @@ Everything Conduit brokers is a **source** — an addressable API identity, whet
|
|||
- **Single-flight coalescing** — concurrent misses for the same `(source, request_key)` → one upstream call.
|
||||
- **Quota guard** (`quota.py`) — per-source day/minute/month caps enforced against a durable `upstream_calls` log, atomic per-source `asyncio.Lock` (single-process). Protects free-tier upstreams (e.g. TomTom's free plan).
|
||||
- **Serve-stale** — on quota-block, upstream 429, or 5xx/transport failure, Conduit serves the last-known-good cached copy instead of failing the caller (`X-Conduit-Stale: 1`).
|
||||
- **Faithful 4xx passthrough** (PR #16, 2026-07-15) — a genuine upstream 4xx *except* 429 (e.g. TomTom flow's `400 "Point too far from nearest existing segment"`) is returned to the caller with its real status + body, uncached, not wrapped as a 502. Only 429/5xx/transport failures fall back to serve-stale-or-502. This unblocked the last meshai adapter (traffic) to migrate.
|
||||
- **Faithful 4xx passthrough** (PR #16, 2026-07-15) — a genuine upstream 4xx *except* 429 (e.g. TomTom flow's `400 "Point too far from nearest existing segment"`) is returned to the caller with its real status + body, uncached, not wrapped as a 502. Only 429/5xx/transport failures fall back to serve-stale-or-502. This unblocked the last [[meshai]] adapter (traffic) to migrate.
|
||||
- **Hot-reload of sources** — a source add/edit/delete made through the GUI calls `Broker.set_sources()` and takes effect with no restart. A direct out-of-band SQL change to `sources` still needs `systemctl restart conduit` — the GUI path is the live one.
|
||||
- **Retention engine** (`poller.py` + `store/history.py`) — opt-in per source (`retain=true`). A background `Poller` fetches retained sources on their `poll_interval_seconds` and appends changed raw payloads to `payload_history` (append-on-change, sha256-dedup). Idle by construction when nothing is retained. Read back via `GET /history/{source}` and `/history/{source}/{id}/body` (unauthenticated, mesh-internal). Purpose: accumulate raw feeds for later forecast/trend models.
|
||||
- **Management GUI** (`gui/`) — built in meshai's visual language. argon2 operator auth + two-tier CSRF + schema-reflection forms harvested from central. Pages: sources CRUD (incl. headers/quota/retention fields), API keys, a Cmd-K command palette. Admin operator provisioned. `/up` and `/history` stay unauthenticated (mesh-internal, tiles-trust model); the GUI is the authenticated surface.
|
||||
- **Keystore** (`keystore.py`, `crypto.py`) — AES-256-GCM `api_keys` by alias, under Conduit's own master key. Holds `tomtom`, `roads511`, and `firms` keys (harvested from central / provisioned) — meshai no longer holds any of these itself.
|
||||
- **Keystore** (`keystore.py`, `crypto.py`) — AES-256-GCM `api_keys` by alias, under Conduit's own master key. Holds `tomtom`, `roads511`, and `firms` keys (harvested from [[central]] / provisioned) — meshai no longer holds any of these itself.
|
||||
|
||||
## Modules (`src/conduit/`)
|
||||
|
||||
|
|
@ -99,7 +99,7 @@ Pull-based deploy: authored/pushed from a cortex clone, CT 104 pulls via a **rea
|
|||
|
||||
## Relationship to [[central]]
|
||||
|
||||
**Central is retired and dropped (2026-07-15) — Conduit replaced it.** Central's app services were stopped and disabled 2026-07-14 (zero live consumers remained), then on 2026-07-15 its database was archived to pi-nas (sha256-verified) and dropped (`DROP DATABASE central`, ~41 GB reclaimed); the shared Postgres instance was cleaned back to plain (TimescaleDB removed from `shared_preload_libraries`). Central is recoverable only from the pi-nas archive. Conduit's own `conduit` DB shares that same Postgres instance, which was never stopped.
|
||||
**Central is retired and dropped (2026-07-15) — Conduit replaced it.** Central's app [[services]] were stopped and disabled 2026-07-14 (zero live consumers remained), then on 2026-07-15 its database was archived to pi-nas (sha256-verified) and dropped (`DROP DATABASE central`, ~41 GB reclaimed); the shared Postgres instance was cleaned back to plain (TimescaleDB removed from `shared_preload_libraries`). Central is recoverable only from the pi-nas archive. Conduit's own `conduit` DB shares that same Postgres instance, which was never stopped.
|
||||
|
||||
Conduit was born by harvesting central's proven, decoupled pieces — the AES-256-GCM encrypted key store, the GUI auth/CSRF + schema-reflection form patterns, and the aiohttp+tenacity fetch idiom — while deliberately shedding central's NATS/JetStream, CloudEvents normalization, and enrichment pipeline.
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue