mirror of
https://github.com/zvx-echo6/central.git
synced 2026-08-26 17:31:39 +00:00
Central - data hub spine. Adapters -> NATS/JetStream -> archive.
- Python 88.6%
- HTML 9.4%
- CSS 1.2%
- Shell 0.6%
- PLpgSQL 0.2%
f3cb603a22
Adds a new dashboard card (4th article on the index) where an operator
picks a time window (5 min / 30 min / 1h / 3h / 6h / 12h / 24h), clicks
Re-send, sees a confirmation modal showing the count of events per stream
that would be replayed, then confirms to re-publish them.
Each replayed message keeps its original subject + raw byte payload but
receives a new Nats-Msg-Id of the form {original}:resend:{ts_epoch_ms}
so JetStream's per-stream deduplication window does NOT silently drop
the replay. Consumers with deliver_policy=new see the messages as fresh;
archive UPSERTs on (id, time) so the events table does not grow.
Stream set is derived from central.streams.STREAMS -- only event_bearing=True
entries are touched (CENTRAL_META is excluded so audit/status messages
are not re-broadcast).
The v0.10.2 publish-time monitoring-area bbox filter is NOT applied here --
the operator is intentionally replaying messages that already passed
through it on their original publish.
Audit log: each completed wave emits a meta-event on
central.meta.action.resend with {operator, window_minutes, count, errors,
started_at, finished_at, elapsed_s, by_stream}. No new stream required
(CENTRAL_META filter already captures it).
Modal flow: inline-swap pattern (style A from step-0 design call), no JS
beyond HTMX. CSRF validated on the POST via the standard form-field
pattern mirroring adapters_edit_submit.
Tests: 12 new in tests/test_resend.py covering preview counts, replay
shape (subject + data preserved, suffix-style msg id), per-stream and
per-message failure tolerance, audit-log meta-event emission, audit-log
failure does not sink the operator-visible result, CENTRAL_META exclusion.
Full suite: 1010 passed / 1 skipped under both zvx and central. Ruff
clean on all touched files.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| docs | ||
| etc-templates | ||
| scripts | ||
| sql | ||
| src/central | ||
| systemd | ||
| tests | ||
| .gitattributes | ||
| .gitignore | ||
| .python-version | ||
| CHANGELOG.md | ||
| LICENSE | ||
| pyproject.toml | ||
| README.md | ||
| uv.lock | ||
Central
Central is the data hub spine for the infrastructure. Adapters normalize upstream sources into a canonical event shape, publish CloudEvents to NATS/JetStream, and archive to TimescaleDB for historical query. Single-LXC deployment.
Status
Phase 0 — scaffold. Not yet operational.
Architecture
- Python 3.12 (uv-managed)
- NATS + JetStream for live event bus
- TimescaleDB + PostGIS for archive and geospatial query
- One supervisor process managing adapter lifecycle
- One archive consumer process persisting events to TimescaleDB
- Both processes systemd-managed
Testing
See docs/test-database.md for test database setup.
License
MIT. See LICENSE.