matt/central
1
0
Fork 0
mirror of https://github.com/zvx-echo6/central.git synced 2026-06-10 11:54:37 +02:00
Code Issues Projects Releases Packages Wiki Activity
251 commits 8 branches 67 tags 4.7 MiB
Commit graph

2 commits

Author SHA1 Message Date
malice
a3aea62d2f
 		v0.10.11: extend avalanche_org adapter — tombstones, geo.bbox, hyphen slugs (#99) 
meshai followup spec on top of v0.10.10. Three additive changes:

1. Tombstone emission. When a zone that previously passed the severity
   gate stops passing (drops below danger_level 3, flips off_season=true,
   or disappears from the upstream feed entirely), the adapter yields a
   retraction Event so subscribers can clear it from their displays.

   Mirrors the wfigs_incidents fall-off pattern:
   - per-adapter sqlite table 'avalanche_org_observed' keyed by
     (center_id, zone_name); tracks last_published_at + state.
   - poll() diff: previously_published - currently_published = removed set.
   - tombstone Event per removed zone: category
     'avy.advisory.removed.<center_id_lower>', severity=0 (matches wfigs
     tombstone convention), empty Geo, id '<cid>_<slug>:removed:<iso>'
     (unique per emission so repeat retraction cycles aren't deduped).
   - subject_for routes 'avy.advisory.removed.*' to
     'central.avy.advisory.removed.us.<state>'.
   - reason field on the tombstone is one of:
       'off_season'      -- upstream still has the zone, off_season=true
       'below_threshold' -- upstream still has the zone, danger_level<3
       'fallen_off_feed' -- zone absent from upstream entirely (center
                            reorganised etc.; meshai's renderer is
                            expected to treat this the same as
                            below_threshold for retraction rendering)
   - State updates (upsert + delete) happen AFTER yields, matching the
     wfigs at-least-once convention: a supervisor crash mid-publish
     re-emits on the next poll rather than silently swallowing.

2. geo.bbox. Added shapely_shape(geometry).bounds as (W, S, E, N) to the
   Geo construction in _build_event_record. Defensive try/except so a
   malformed geometry doesn't crash; falls back to bbox=None.

3. Slug format change: underscores -> hyphens
   ('banner_summit' -> 'banner-summit'). One-line regex change. Safe to
   ship because off-season published_ids count for avalanche_org was 0
   at the time of this PR -- no live event.id values to invalidate. event.id
   shape stays '<center_id>_<slug>' (underscore between center and slug
   remains; only the slug itself changes).

Tests (13 new in tests/test_avalanche_org.py, 51 total):

- Slug parametrize: all 8 cases flipped to hyphens.
- Live-publish test: asserts bbox present and ev.id uses hyphenated slug.
- _removal_reason classification: 5 parametrized cases covering all three
  reasons + None input.
- State-transition tests covering every cell of the matrix:
  - Considerable -> Low: tombstone(below_threshold)
  - Considerable -> off_season: tombstone(off_season)
  - Considerable -> absent: tombstone(fallen_off_feed)
  - Considerable -> Considerable: no tombstone (live publish only)
  - Low -> off_season: no tombstone (never published, nothing to retract)
- Load-bearing test_no_duplicate_tombstone_across_consecutive_polls:
  Considerable -> Low (emit tombstone + explicit DB query confirming
  observed row deleted) -> still Low (no second tombstone) -> recovers
  to Considerable (normal live publish). Guards against the most likely
  bug class meshai is exposed to if the diff logic mishandles state.
- subject_for routing: tombstone -> 'central.avy.advisory.removed.us.id';
  live publish still -> 'central.avy.advisory.us.id'.
- Tombstone-id uniqueness: each emission gets a fresh :removed:<iso>
  suffix so JetStream's per-stream dedup doesn't swallow repeats.

Wiring NOT changed:
- streams.py / supervisor.py STREAM_CATEGORY_DOMAINS: tombstones still
  route to CENTRAL_AVY (category prefix 'avy.advisory.removed.*' starts
  with 'avy', covered by the existing ('avy',) family domain).
- gui partials, doc updates: no new adapter, no new domain -- no
  consistency-test updates needed.

Diff size: +365 / -13 = +352 net. Slightly over the 300-line target;
220 of those lines are the 13 new tombstone tests (state-transition
correctness coverage). Adapter logic itself is +145 -- which is what
the new feature requires (sqlite table + 3 helpers + reason classifier
+ diff phase in poll). No defensive scaffolding beyond what wfigs
already establishes.

Full sweep: 1085 passed (+13 from this PR), ruff clean on both files.

Deploy plan: code-only change, no migration, no new stream/adapter
config rows. Squash-merge -> tag v0.10.11 -> pull on central -> restart
central-supervisor. NO archive restart (extending an existing stream,
not adding a new one). NO published_ids flush. Post-deploy verify the
poll still completes (events_yielded=0, events_omitted=6,
tombstones_emitted=0 during off-season since no zones were previously
published).

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-06-08 23:08:22 -06:00
malice
e92b51c518
 		v0.10.10: new avalanche_org adapter — backcountry avalanche advisories (#98) 
meshai-requested adapter for avalanche.org's per-center map layers (SNFAC
Sawtooth + PAC Payette by default; operator-extensible to any avalanche.org
forecast center). Pure passthrough + severity gate, no cross-source fusion,
fits Central's adapter pattern cleanly.

Adapter surface:
- Polls https://api.avalanche.org/v2/public/products/map-layer/{center_id}
  per configured center; default cadence 1800s (30 min).
- Severity gate: only danger_level >= 3 publishes. danger_level 0/1/2
  (None/Low/Moderate), -1 ('no rating'), and off_season=true all omitted at
  adapter level. Idaho summer = all 4 SNFAC + 2 PAC zones yield 0 events;
  that's correct behavior, verified by the negative-case test against the
  frozen 2026-06-08 SNFAC fixture.
- Severity mapping (corrected from meshai's inverted spec): danger_level
  3 (Considerable) → severity 2, 4 (High) → 3, 5 (Extreme) → 4. Matches
  Central's 4-most-severe convention (nws.SEVERITY_MAP).
- Subject: central.avy.advisory.us.{state_lower} — one per state; v0.10.8's
  category-discriminated Nats-Msg-Id keeps multiple zones in the same state
  from colliding in JetStream dedup.
- Stream: CENTRAL_AVY (central.avy.>); 7-day / 1 GiB retention defaults.
- Event.data fields per meshai spec: center_id, zone_name, danger_level,
  danger_name, travel_advice (truncated to 200 chars), state, valid_date,
  end_date, off_season=false, latitude/longitude (polygon centroid via
  shapely), plus geo.geometry passes through as the upstream Polygon.

Tests (38 in test_avalanche_org.py):
- Pure helpers: _slug (8 cases), _parse_iso (6 cases), _centroid (2 cases).
- Severity gate: 3 publish cases (danger 3/4/5 → severity 2/3/4),
  4 omit cases (danger -1/0/1/2), off_season=true omit, missing state omit,
  unparseable geom omit, travel_advice truncation, subject derivation.
- Real-fixture negative case: 4-zone SNFAC fixture all omitted off-season.
- Real-fixture positive case: same fixture with synthetic winter overrides
  publishes all 4 with valid centroids on actual Idaho polygons.
- End-to-end poll() with mixed severities and the new wiring (streams
  registry + supervisor family map).
- Defensive: empty center_ids list yields nothing without crashing.

Wiring + plumbing:
- src/central/streams.py: StreamEntry('CENTRAL_AVY', 'central.avy.>')
- src/central/supervisor.py: STREAM_CATEGORY_DOMAINS['CENTRAL_AVY']=('avy',)
- sql/migrations/035: seed config.streams row (mirror of 019/CENTRAL_SPACE,
  idempotent ON CONFLICT DO NOTHING). Note: migrations don't auto-run on
  supervisor restart -- see deferred ops list (schema_migrations cleanup
  blocks central-migrate from running anything cleanly).
- src/central/gui/templates/_event_rows/avalanche_org.html (8 lines)
- src/central/gui/templates/_event_summaries/avalanche_org.html (2 lines)
  Both required by the existing per-adapter template consistency tests.

Doc updates (required by existing doc-vs-registry tests):
- docs/PRODUCER-INTEGRATION.md §6.1: added 'avy' to top-level-domain list.
- docs/PRODUCER-INTEGRATION.md §8: added StreamEntry('CENTRAL_AVY',...) line
  to the verbatim snippet.
- docs/CONSUMER-INTEGRATION.md §3 stream layout table: added CENTRAL_AVY row.
- docs/CONSUMER-INTEGRATION.md §6: new '### avalanche_org' subsection with
  source, subject convention, dedup key, severity gate, Event.data field
  table, and off-season behavior note.
- tests/test_events_feed_frontend.py: added avalanche_org to _SAMPLE_INNER
  and _EXPECTED_SUBJECT (the events-JSON subject-derivation coverage tests).

Budget note: this PR is well over the ~400-line target -- the new-adapter
surface picked up downstream consistency tests (doc validators + frontend
sample coverage + template partials) I didn't anticipate at probe time.
Most of the overrun is the SNFAC fixture (1,135 lines pretty-printed JSON,
non-code) and the adapter + tests pair. Stripping the fixture and the
required doc/template edits would leave ~620 lines of code; the fixture
itself is a frozen snapshot, not a maintenance burden.

Full sweep: 1072 passed, 0 failures (+41 from this PR), ruff clean on
all new files. One PRE-EXISTING ruff violation in supervisor.py (unused
poll_start variable at line 388) surfaces when we touch supervisor.py;
confirmed not introduced by this PR via git stash check.

Deploy plan (NEW STREAM — archive restart required per
[[feedback_new_stream_needs_archive_restart]]):
1. Squash-merge -> tag v0.10.10 -> push.
2. On central: pull main -> systemctl restart central-supervisor -> ALSO
   systemctl restart central-archive (new event-bearing stream; archive
   enumerates consumers at startup and doesn't hot-reload).
3. Migration 035 deferred to morning per the schema_migrations cleanup
   task -- the stream creation itself doesn't depend on it (supervisor
   creates JetStream streams from the STREAMS registry at startup; the
   config.streams row is for operator-tunable retention only).
4. Verify: nats stream info CENTRAL_AVY (created), poll log shows
   yielded=0 / omitted=N (off-season), no positive publishes during
   summer (correct).

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-06-08 21:57:56 -06:00