matt/central
1
0
Fork 0
mirror of https://github.com/zvx-echo6/central.git synced 2026-08-26 09:21:36 +00:00
Code Issues Projects Releases Packages Wiki Activity
Central - data hub spine. Adapters -> NATS/JetStream -> archive.
267 commits 18 branches 77 tags 5.4 MiB
  • Python 88.6%
  • HTML 9.4%
  • CSS 1.2%
  • Shell 0.6%
  • PLpgSQL 0.2%
Find a file
malice 1ae3f3e7fd
 supervisor: shrink sat dedup window + WAL cursors.db to cut CPU/IO (#114) 
sat_positions and sat_orbits produce dedup IDs that are unique per
second (<norad_id>:<iso_timestamp>), so the inherited 14-day sweep
window accumulates ~3.5M rows in cursors.db with no benefit. Shrink
to 1 day (positions) and 2 days (orbits, to tolerate stable TLE
re-emission).

Open cursors.db in WAL + NORMAL sync mode in both adapters' startup().
WAL mode is file-level persistent, so this covers all adapters sharing
the same cursors.db. Previously journal_mode=DELETE + synchronous=FULL
forced an fsync on every mark_published() call (~190 calls/60s for
sat_positions alone), sustaining ~17% CPU on the supervisor process.

A redeploy is needed; the box is currently detached at v0.14.5.

Co-authored-by: Ubuntu <zvx@cortex.echo6.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 14:13:50 -06:00
docs v0.13.0: sat_orbits adapter (forward-orbit-track per satellite) + antimeridian splitter 2026-06-09 18:50:47 -06:00
etc-templates scaffold: initial repository structure 2026-05-15 19:16:24 +00:00
scripts scaffold: initial repository structure 2026-05-15 19:16:24 +00:00
sql v0.14.6: commit orphaned migration 036 + fix stale README/version metadata (#112) 2026-06-28 14:13:40 -06:00
src/central supervisor: shrink sat dedup window + WAL cursors.db to cut CPU/IO (#114) 2026-06-28 14:13:50 -06:00
systemd feat(gui): add auth core, setup gate, and first-run operator creation 2026-05-17 05:30:49 +00:00
tests v0.14.5: migration 042 re-asserts central ownership of config.monitoring_areas (#111) 2026-06-15 22:02:02 -06:00
.gitattributes chore: normalize line endings to LF 2026-05-16 22:26:12 +00:00
.gitignore feat(gui): add auth core, setup gate, and first-run operator creation 2026-05-17 05:30:49 +00:00
.python-version foundation: models, adapter ABC, config, CE wire, schema 2026-05-15 21:08:56 +00:00
CHANGELOG.md docs: add v0.3.0 changelog entry and network bindings reference (#29) 2026-05-18 14:26:09 -06:00
LICENSE scaffold: initial repository structure 2026-05-15 19:16:24 +00:00
pyproject.toml v0.14.6: commit orphaned migration 036 + fix stale README/version metadata (#112) 2026-06-28 14:13:40 -06:00
README.md v0.14.6: commit orphaned migration 036 + fix stale README/version metadata (#112) 2026-06-28 14:13:40 -06:00
uv.lock v0.11.1: satpass_predict adapter (server-side pass alerts for fixed observers) (#101) 2026-06-09 01:16:43 -06:00

README.md

Central

Central is the data hub spine for the Echo6 infrastructure. Adapters normalize upstream sources into a canonical event shape, publish CloudEvents to NATS/JetStream, and archive to TimescaleDB for historical query. Single-LXC deployment.

Status

Operational (v0.14.6) — deployed on utility CT 104 (central.echo6.mesh).

~22 adapters live across traffic, wildfire, weather, space-weather, hydrology, earthquakes, avalanche, disasters, and satellite. CloudEvents published to NATS/JetStream, archived to TimescaleDB/PostGIS. FastAPI + HTMX GUI/API on :8000.

Three systemd units on a single LXC:

  • central-supervisor — adapter lifecycle manager
  • central-archive — NATS consumer persisting events to TimescaleDB
  • central-gui — FastAPI + HTMX web interface / API (:8000)

Architecture

  • Python 3.12 (uv-managed)
  • NATS + JetStream for live event bus
  • TimescaleDB + PostGIS for archive and geospatial query
  • One supervisor process managing adapter lifecycle
  • One archive consumer process persisting events to TimescaleDB
  • Both processes systemd-managed

Testing

See docs/test-database.md for test database setup.

License

MIT. See LICENSE.