mirror of
https://github.com/zvx-echo6/central.git
synced 2026-06-10 11:54:37 +02:00
Central - data hub spine. Adapters -> NATS/JetStream -> archive.
- Python 89.1%
- HTML 9.3%
- CSS 1.3%
- PLpgSQL 0.3%
malice
a3aea62d2fmeshai followup spec on top of v0.10.10. Three additive changes:
1. Tombstone emission. When a zone that previously passed the severity
gate stops passing (drops below danger_level 3, flips off_season=true,
or disappears from the upstream feed entirely), the adapter yields a
retraction Event so subscribers can clear it from their displays.
Mirrors the wfigs_incidents fall-off pattern:
- per-adapter sqlite table 'avalanche_org_observed' keyed by
(center_id, zone_name); tracks last_published_at + state.
- poll() diff: previously_published - currently_published = removed set.
- tombstone Event per removed zone: category
'avy.advisory.removed.<center_id_lower>', severity=0 (matches wfigs
tombstone convention), empty Geo, id '<cid>_<slug>:removed:<iso>'
(unique per emission so repeat retraction cycles aren't deduped).
- subject_for routes 'avy.advisory.removed.*' to
'central.avy.advisory.removed.us.<state>'.
- reason field on the tombstone is one of:
'off_season' -- upstream still has the zone, off_season=true
'below_threshold' -- upstream still has the zone, danger_level<3
'fallen_off_feed' -- zone absent from upstream entirely (center
reorganised etc.; meshai's renderer is
expected to treat this the same as
below_threshold for retraction rendering)
- State updates (upsert + delete) happen AFTER yields, matching the
wfigs at-least-once convention: a supervisor crash mid-publish
re-emits on the next poll rather than silently swallowing.
2. geo.bbox. Added shapely_shape(geometry).bounds as (W, S, E, N) to the
Geo construction in _build_event_record. Defensive try/except so a
malformed geometry doesn't crash; falls back to bbox=None.
3. Slug format change: underscores -> hyphens
('banner_summit' -> 'banner-summit'). One-line regex change. Safe to
ship because off-season published_ids count for avalanche_org was 0
at the time of this PR -- no live event.id values to invalidate. event.id
shape stays '<center_id>_<slug>' (underscore between center and slug
remains; only the slug itself changes).
Tests (13 new in tests/test_avalanche_org.py, 51 total):
- Slug parametrize: all 8 cases flipped to hyphens.
- Live-publish test: asserts bbox present and ev.id uses hyphenated slug.
- _removal_reason classification: 5 parametrized cases covering all three
reasons + None input.
- State-transition tests covering every cell of the matrix:
- Considerable -> Low: tombstone(below_threshold)
- Considerable -> off_season: tombstone(off_season)
- Considerable -> absent: tombstone(fallen_off_feed)
- Considerable -> Considerable: no tombstone (live publish only)
- Low -> off_season: no tombstone (never published, nothing to retract)
- Load-bearing test_no_duplicate_tombstone_across_consecutive_polls:
Considerable -> Low (emit tombstone + explicit DB query confirming
observed row deleted) -> still Low (no second tombstone) -> recovers
to Considerable (normal live publish). Guards against the most likely
bug class meshai is exposed to if the diff logic mishandles state.
- subject_for routing: tombstone -> 'central.avy.advisory.removed.us.id';
live publish still -> 'central.avy.advisory.us.id'.
- Tombstone-id uniqueness: each emission gets a fresh :removed:<iso>
suffix so JetStream's per-stream dedup doesn't swallow repeats.
Wiring NOT changed:
- streams.py / supervisor.py STREAM_CATEGORY_DOMAINS: tombstones still
route to CENTRAL_AVY (category prefix 'avy.advisory.removed.*' starts
with 'avy', covered by the existing ('avy',) family domain).
- gui partials, doc updates: no new adapter, no new domain -- no
consistency-test updates needed.
Diff size: +365 / -13 = +352 net. Slightly over the 300-line target;
220 of those lines are the 13 new tombstone tests (state-transition
correctness coverage). Adapter logic itself is +145 -- which is what
the new feature requires (sqlite table + 3 helpers + reason classifier
+ diff phase in poll). No defensive scaffolding beyond what wfigs
already establishes.
Full sweep: 1085 passed (+13 from this PR), ruff clean on both files.
Deploy plan: code-only change, no migration, no new stream/adapter
config rows. Squash-merge -> tag v0.10.11 -> pull on central -> restart
central-supervisor. NO archive restart (extending an existing stream,
not adding a new one). NO published_ids flush. Post-deploy verify the
poll still completes (events_yielded=0, events_omitted=6,
tombstones_emitted=0 during off-season since no zones were previously
published).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| docs | ||
| etc-templates | ||
| scripts | ||
| sql | ||
| src/central | ||
| systemd | ||
| tests | ||
| .gitattributes | ||
| .gitignore | ||
| .python-version | ||
| CHANGELOG.md | ||
| LICENSE | ||
| pyproject.toml | ||
| README.md | ||
| uv.lock | ||
Central
Central is the data hub spine for the infrastructure. Adapters normalize upstream sources into a canonical event shape, publish CloudEvents to NATS/JetStream, and archive to TimescaleDB for historical query. Single-LXC deployment.
Status
Phase 0 — scaffold. Not yet operational.
Architecture
- Python 3.12 (uv-managed)
- NATS + JetStream for live event bus
- TimescaleDB + PostGIS for archive and geospatial query
- One supervisor process managing adapter lifecycle
- One archive consumer process persisting events to TimescaleDB
- Both processes systemd-managed
Testing
See docs/test-database.md for test database setup.
License
MIT. See LICENSE.