matt/central
1
0
Fork 0
mirror of https://github.com/zvx-echo6/central.git synced 2026-06-10 11:54:37 +02:00
Code Issues Projects Releases Packages Wiki Activity
Central - data hub spine. Adapters -> NATS/JetStream -> archive.
217 commits 8 branches 67 tags 4.7 MiB
  • Python 89.1%
  • HTML 9.3%
  • CSS 1.3%
  • PLpgSQL 0.3%
Find a file
Matt Johnson 9f67b4c1f2 v0.9.13: per-stream archived-events retention sweep 
Hourly supervisor sweep deletes archived events older than their stream's
config.streams.max_age_s (the /streams 1/7/14/30/365d buttons -- the per-stream
source-of-truth). Explicit STREAM_CATEGORY_DOMAINS map (category domains do not
equal stream subject domains for the traffic family); fail-safe skip on
missing/<=0 max_age_s; unmapped-domain warning so a new adapter cannot silently
dodge retention.

TimescaleDB hypertable gotcha: events.ctid is chunk-local, NOT globally unique --
DELETE batching keys on the composite PK (id, time). See the inline NOTE in
config_store.delete_events_older_than and the PR body for the incident write-up.

- config_store: delete_events_older_than (batched on (id,time)) + unmapped_event_domains
- supervisor: STREAM_CATEGORY_DOMAINS, _sweep_events_retention, _events_retention_loop
- streams_list.html: max-age also governs archived-events retention
- 9 tests

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 02:31:11 +00:00
docs feat(state_511_atis_cameras): Castle Rock 511 traffic cameras telemetry (v0.9.6) 2026-05-26 01:33:21 +00:00
etc-templates scaffold: initial repository structure 2026-05-15 19:16:24 +00:00
scripts scaffold: initial repository structure 2026-05-15 19:16:24 +00:00
sql v0.9.12: archive-level monitoring-area bbox filter 2026-05-26 23:40:17 +00:00
src/central v0.9.13: per-stream archived-events retention sweep 2026-05-27 02:31:11 +00:00
systemd feat(gui): add auth core, setup gate, and first-run operator creation 2026-05-17 05:30:49 +00:00
tests v0.9.13: per-stream archived-events retention sweep 2026-05-27 02:31:11 +00:00
.gitattributes chore: normalize line endings to LF 2026-05-16 22:26:12 +00:00
.gitignore feat(gui): add auth core, setup gate, and first-run operator creation 2026-05-17 05:30:49 +00:00
.python-version foundation: models, adapter ABC, config, CE wire, schema 2026-05-15 21:08:56 +00:00
CHANGELOG.md docs: add v0.3.0 changelog entry and network bindings reference (#29) 2026-05-18 14:26:09 -06:00
LICENSE scaffold: initial repository structure 2026-05-15 19:16:24 +00:00
pyproject.toml feat(tomtom_flow): TomTom Orbis vector flow-tile telemetry adapter + CENTRAL_TRAFFIC_FLOW (v0.9.3) 2026-05-25 23:25:44 +00:00
README.md docs: add test database setup, restore geom to test fixture 2026-05-17 18:26:48 +00:00
uv.lock feat(tomtom_flow): TomTom Orbis vector flow-tile telemetry adapter + CENTRAL_TRAFFIC_FLOW (v0.9.3) 2026-05-25 23:25:44 +00:00

README.md

Central

Central is the data hub spine for the infrastructure. Adapters normalize upstream sources into a canonical event shape, publish CloudEvents to NATS/JetStream, and archive to TimescaleDB for historical query. Single-LXC deployment.

Status

Phase 0 — scaffold. Not yet operational.

Architecture

  • Python 3.12 (uv-managed)
  • NATS + JetStream for live event bus
  • TimescaleDB + PostGIS for archive and geospatial query
  • One supervisor process managing adapter lifecycle
  • One archive consumer process persisting events to TimescaleDB
  • Both processes systemd-managed

Testing

See docs/test-database.md for test database setup.

License

MIT. See LICENSE.