echo6-docs/vault/docs/software/central.md

title	type	tags	aliases	related	updated
central — Data-Hub Spine	reference	mesh		central-deploy-cutover services navi fleet-platform-baseline caddy conduit	2026-07-14

central — Data-Hub Spine

Overview

central is a multi-domain real-time data-hub spine. Adapters normalize upstream sources, publish CloudEvents to NATS/JetStream, and archive to TimescaleDB/PostGIS for historical and geospatial query. It is the live data backbone for navi traffic tiles and related situational-awareness feeds.

• URL (internal): http://central.echo6.mesh:8000 (mesh-only, no public exposure)
• Host: utility CT 104 (unprivileged Ubuntu LXC)
• Repo: github.com/zvx-echo6/central (public, Python, deployed at detached HEAD, tag v0.15.0)
• Deploy path: /opt/central (venv: /opt/central/.venv, env: /etc/central/central.env, system user: central)

Host

Attribute	Value
Container	utility CT 104
Local IP	192.168.1.104
Tailscale / mesh	100.64.0.12 → central.echo6.mesh
Resources	4 cores / 12 GB RAM / 100 GB disk
OS	Ubuntu LXC (unprivileged)

Architecture

The data flow is: upstream APIs → adapters (central-supervisor) → NATS/JetStream :4222 → central-archive (TimescaleDB/PostGIS :5432). The central-gui (FastAPI + HTMX, :8000) exposes the API consumed by navi via recon-vm's nginx (port 8440, ^~ /api/traffic/ → central.echo6.mesh:8000).

Systemd Services

All three units are enabled and active; deployment survives reboot. Deps: nats-server, postgresql@16-main.

Unit	Role
central-supervisor	Adapter scheduler + CloudEvents publisher
central-archive	JetStream → TimescaleDB consumer
central-gui	FastAPI + HTMX web app + API (the :8000 listener)

Ports

Port	Protocol	Purpose
:8000	HTTP	GUI / API (bound 0.0.0.0)
:4222	TCP	NATS client connections
:8222	HTTP	NATS monitoring
:5432	TCP	PostgreSQL 16 + TimescaleDB/PostGIS

Adapters (23 configured, 22 enabled)

Domain	Sources
Traffic	ITD 511, WZDX, TomTom flow/incidents, 511 cameras
Wildfire	WFIGS incidents/perimeters, InciWeb, FIRMS
Weather / Space-weather	NWS, SWPC k-index/protons/alerts
Hydro	NWIS
Earthquakes	USGS
Avalanche	avalanche.org
Disasters	GDACS, EONET
Satellite	CelesTrak TLE, sat positions/orbits, satpass_predict (n2yo_visualpasses disabled 2026-06-27)

GUI / API Surface

Authenticated app with login/sessions/CSRF, first-run setup wizard, operator management, adapter configuration (incl. no-code creation of generic REST/GeoJSON sources, v0.15.0), stream viewer, JetStream consumer management (/consumers — view + delete consumers; central's own archive-* durable consumers are protected/non-deletable; CSRF + audit-logged), enrichment pipeline, monitoring-area management, API key management, audit log, and manual resend.

Auth-exempt tile endpoints (used by navi, verified HTTP 200):

Endpoint	Format
/api/traffic/flow/{z}/{x}/{y}.png	PNG tile (raster, image/png)
/api/traffic/flow/{z}/{x}/{y}.pbf	PBF tile (vector, application/x-protobuf)

/health is also auth-exempt (returns {"status":"ok"}). All other /api/ endpoints redirect to /login.

Consumer — navi Integration

navi-traffic (navi's in-VM :8421 extraction service) was retired on 2026-05-26 and cut over to central. recon-vm's nginx (/etc/nginx/sites-available/navi.echo6.co, port 8440) proxied ^~ /api/traffic/ → central.echo6.mesh:8000 with a 120s tile cache. navi-traffic:8421 is confirmed dead and disabled.

As of 2026-07-14, navi's /api/traffic/ tiles were repointed to conduit: recon-vm's nginx now rewrites ^~ /api/traffic/ → /up/tomtom_flow_tiles/... and proxies to central.echo6.mesh:8010 (Conduit, co-resident on the same CT 104 host). Central's own tile endpoint (:8000/api/traffic/...) still exists and still works, but is no longer on navi's hot path — it remains the rollback target if the Conduit cutover needs to be reverted. See conduit-operations for the cutover and rollback procedure.

Dependencies

Component	Location
NATS / JetStream	Local (central CT 104)
PostgreSQL 16 + TimescaleDB + PostGIS	Local (central CT 104)
Upstream APIs	~20 external sources (see Adapters table)

Data Plane

Single TimescaleDB hypertable public.events (~40 GB, ~3.42 M rows, 331 chunks, compression not enabled). JetStream ~5.9 GB / 20 GB across 12 streams.

Deploy / State Notes (as of 2026-07-13)

• Deployed HEAD: v0.15.0 (detached HEAD — the box tracks tags; sha 3c8da28, pyproject.toml 0.15.0). All 3 units active, /health 200, 23 adapters configured / 22 enabled (only n2yo_visualpasses disabled).
• v0.15.0 (deployed 2026-07-02, PRs #120–124): the GUI can now add public REST/GeoJSON data sources no-code — a generic adapter kind created and configured entirely through adapter management, no repo change. None enabled in this deployment yet; all 23 current adapters remain bespoke per-source kinds (each config.adapters.kind is unique to its source).
• Deploy mechanism: manual, tag-based (git checkout <tag> → uv sync → central-migrate → restart). One-command scripts/deploy.sh ships in-repo; full procedure in the central-deploy-cutover runbook. No CI/CD; no automated DB backup (pre-flight pg_dump is the only migration safety net; migrations are forward-only).
• central-migrate env gotcha: manual calls need cd /opt/central && set -a && . /etc/central/central.env && set +a && … (the units supply this via WorkingDirectory + EnvironmentFile).
• EONET: now global as of v0.14.6 (2026-06-28) — bypass_bbox_filter=True + the adapter region key removed. NOTE: its events-table row count is a misleading health signal (dedup + partition expiry); use the CENTRAL_DISASTER stream as the EONET flow signal. avalanche_org zero-events is expected (off-season gate).
• Supervisor CPU ~17%→~1.4% and cursors.db 1.3 GB→496 MB after the v0.14.6 dedup/WAL fix + a VACUUM.

---

Last updated: 2026-07-13 — v0.15.0 deployed & verified (GUI no-code generic REST/GeoJSON adapters, PRs #120–124); 3.42 M events / ~40 GB, all units active.