mirror of
https://github.com/zvx-echo6/meshai.git
synced 2026-06-10 17:04:45 +02:00
54 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 42b3106e97 |
feat(v0.6-3c): adapter_config REST API + dashboard editor
Closes the audit-doc Section A keystone (the GUI editor). Together with
v0.6-3a foundation, v0.6-3a.1 trim, and v0.6-3b handler wiring, every
Rule-17 CONFIG knob from the audit is now editable in the dashboard
without a container restart.
API (meshai/dashboard/api/adapter_config_routes.py):
GET /api/adapter-config -- {adapter: [{key, value, default,
type, description}]}
GET /api/adapter-config/<adapter> -- one adapter list
GET /api/adapter-config/<adapter>/<key> -- single row
PUT /api/adapter-config/<adapter>/<key> body {value} -- typed validation
int: int or whole-number float; rejects bool, fractional float, str
float: int or float; rejects bool
str: str only
bool: bool only
json: any JSON-serializable value
Every PUT calls invalidate_cache() so the next handler accessor
read sees the new value -- no container restart needed.
POST /api/adapter-config/<adapter>/<key>/reset -- value_json = default_json,
cache invalidated
GET /api/adapter-meta -- {adapter: {display_name,
include_in_llm_context, description}}
PUT /api/adapter-meta/<adapter> partial-update body, fields:
include_in_llm_context: bool, display_name: non-empty str
Dashboard (dashboard-frontend/src/pages/AdapterConfig.tsx):
- Per-adapter cards. Header row shows display_name, the include_in_llm_context
toggle, and an expand chevron. Adapters with zero config keys (e.g. itd_511)
still render so users can toggle their LLM-context inclusion.
- Expanded body lists each key with a type-aware widget:
bool -> checkbox, commit-on-change
int/float -> number input, commit-on-blur (or Enter)
str -> text input, commit-on-blur
json -> textarea, commit-on-blur (JSON.parse with inline error)
Each row shows the key name, type tag, description, "edited" badge when
value != default, a per-key Reset button, and a save badge (spinner,
check, error tooltip, or a small amber dot for unsaved local changes).
- Auto-save semantics: every blur/change/reset triggers PUT immediately;
no explicit Save button needed. Reset is one-click per key.
Wiring:
- meshai/dashboard/server.py registers the new router with prefix /api.
- dashboard-frontend/src/App.tsx adds the /adapter-config route.
- dashboard-frontend/src/components/Layout.tsx adds the left-nav entry
(Sliders icon, label "Adapter Config", after Reference).
- Vite build produces a fresh meshai/dashboard/static bundle. The
Dockerfile copies meshai/ so the new bundle ships with the container
image at next rebuild.
Tests (tests/test_adapter_config_api.py, 30 cases):
- GET grouped, per-adapter, single key
- GET per-adapter returns [] for adapters with zero keys (itd_511)
- PUT updates value, GET shows new value, accessor returns new value
(proves cache invalidation propagates to the in-process accessor)
- PUT type validation per (int, float, str, bool, json) incl. edge cases:
int rejects str + fractional float + bool but accepts whole-number float;
float accepts int + float, rejects bool; bool rejects int; str rejects
other types; json accepts list / dict / None
- PUT 404 on unknown key, 400 on missing value field
- POST reset restores default + invalidates cache
- GET /api/adapter-meta: include_in_llm_context defaults match registry
(central / geocoder false, rest true)
- PUT meta partial update: only provided fields change
- PUT meta rejects non-bool include_in_llm_context, empty display_name,
unknown adapter
Test count: 731 -> 761 (+30 API cases, 0 regressions).
Refs audit doc v0.6-phase1-audit.md Section A keystone + finding #4.
|
|||
| 914d38c907 |
feat(v0.6-3b): wire every handler to adapter_config + v7.sql firms dedup_key column
Replaces module-level magic numbers in 12 handlers with reads via the
v0.6-3a.1 typed accessor. Every default matches the prior hardcoded
value exactly, so first-deploy behavior is unchanged.
Handlers wired (43 keys across the 43-row registry):
wfigs cooldown_seconds, anchor_max_mi, broadcast_on_acres,
broadcast_on_contained
nws broadcast_severities, tombstone_msgtypes,
warning_suffix_promotes
usgs_quake regional_centroid, regional_radius_mi,
broadcast_pager_alerts, global_mag_floor,
regional_mag_floor, escalate_mag_floor
swpc geomag_kp_floor (extends G-scale down to Kp 5 when
lowered), flare_class_floor (R-scale
extended to M-class when lowered),
proton_pfu_floor
usgs_nwis parameter_codes, broadcast_on_recede
incident freshness_seconds, broadcast_on_update (Update path
re-implemented when toggled True:
magnitude step-up / delay doubling /
icon_category change)
tomtom_incidents drop_zero_magnitude, drop_non_present
state_511_atis skipped_states (case-insensitive match against both
state_code and primary_region suffix)
central severity_thresholds (immediate_min check ordered before
priority_max so the +inf clamp still
works)
dispatcher dedup_lru_max, cooldown_prune_size,
cooldown_prune_multiplier, dedup_db_retention_days
band_conditions swpc_freshness_seconds, hamqsl_url, hamqsl_timeout_s
geocoder (photon_url/timeout/radius/limit/town_osm_values/
h3_cache_max -- module-level constants kept as
backward-compat aliases; runtime reads via accessor)
pipeline Inhibitor.ttl_seconds + Grouper.window_seconds now
default to None, falling back to
adapter_config.pipeline.{inhibitor_ttl_seconds,
grouper_window_seconds}. Explicit constructor values
still win (test fixtures unchanged).
firms confidence_floor, frp_floor, bbox, dedup_distance_m
Schema:
v7.sql adds firms_pixels.dedup_key column + drops the old hardcoded
round(lat,5) UNIQUE INDEX, replaces with UNIQUE (dedup_key, acq_time,
satellite). The firms_handler quantizes lat/lon to
(dedup_distance_m / 111000) degrees at INSERT time -- meters-based
precision per Matt s spec, tunable via the GUI without schema changes.
SCHEMA_VERSION 6 -> 7. firms_pixels has 0 rows in production so no
backfill needed.
CODE preserved (Matt s rule): sentence templates, emoji literals, the
TomTom icon_map / ITD sub_type_map / Central adapter_map / category_map
translation tables, the band_conditions Kp/SFI -> Good/Fair/Poor
heuristic, anchor-priority ordering, expires-bucket boundaries, the
NOAA G/R/S scale tables. None of these reach the GUI.
Hot-path performance: every accessor read hits the in-memory cache after
the first call; cache hit is one dict get. Per-event reads (e.g. WFIGS
cooldown_seconds on every WFIGS poll-cycle) add a single dict lookup
to existing pipelines.
Backward-compat aliases retained for module-level imports that exist in
test code: WFIGS_BROADCAST_COOLDOWN_S, FIRMS_CONFIDENCE_FLOOR,
FIRMS_FRP_FLOOR, FIRMS_BBOX_OPTIONAL, INCIDENT_FRESHNESS_MAX_S,
PHOTON_BASE_URL/TIMEOUT_S/RADIUS_KM/LIMIT. Handler code reads via
adapter_config; tests can either monkeypatch the module attribute (firms)
or mutate adapter_config DB values.
Test count: 731 -> 731 (no new tests in 3b -- handler wiring is a pure
refactor; coverage comes from the existing handler test suites passing
unchanged).
Refs audit doc v0.6-phase1-audit.md Section A + Matt s CONFIG-vs-CODE rule.
|
|||
| 68dcbc74d0 |
feat(v0.6-3a.1): trim adapter_config registry to CONFIG-only per Matt config-vs-code rule + log-on-delete safety net for orphan cleanup
Drops 35 of the v0.6-3a-draft 77 keys + adds 1 net-new key
(firms.dedup_distance_m) for a final count of 43. The trim rules:
CONFIG (lives in adapter_config, surfaces in the GUI):
where we send (channels), how often (cadences/schedules),
thresholds (magnitude floors, severity gates, distance radius,
cooldown durations, freshness windows), curation data (which
sites/states/codes), toggles (enabled, include_in_llm_context,
drop_zero_magnitude).
CODE (stays in handlers, never reaches the GUI):
sentence templates, emoji choices, mapping/translation functions
(TomTom icon_map, ITD sub_type_map, Central adapter_map and
category_map), rendering logic (anchor priority order,
expires-bucket formatting, threshold-state labels), heuristic
logic (band_conditions Kp/SFI -> Good/Fair/Poor function).
Per-adapter outcome (kept | killed):
wfigs 4 | 4 (cooldown_seconds, anchor_max_mi, two re-broadcast toggles)
nws 3 | 4 (broadcast_severities, tombstone_msgtypes, warning_suffix_promotes)
usgs_quake 6 | 3 (centroid, radius, PAGER list, 3 mag floors)
swpc 3 | 7 (three storm-tier floors)
usgs_nwis 2 | 4 (parameter_codes, broadcast_on_recede)
incident 2 | 0 (freshness_seconds, broadcast_on_update)
tomtom_incidents 2 | 1 (drop_zero_magnitude, drop_non_present)
state_511_atis 1 | 0 (skipped_states)
itd_511 0 | 3 (all sub_type maps/emoji/phrase = CODE)
central 1 | 2 (severity_thresholds)
dispatcher 4 | 0 (LRU cap, prune params, retention days)
band_conditions 3 | 6 (SWPC freshness + HamQSL endpoint config)
geocoder 6 | 1 (Photon endpoint + town-OSM curation + cache cap)
firms 4 | 1* (confidence_floor, frp_floor, bbox, dedup_distance_m)
pipeline 2 | 0 (inhibitor TTL, grouper window)
* firms: dedup_lat_lon_decimals is replaced by dedup_distance_m=5 per
Matt s call (user-facing unit is meters, not decimal places; the
handler will internally translate to quantization step in v0.6-3b).
adapter_meta stays at 15 rows -- itd_511 keeps its include_in_llm_context
toggle even with zero config keys.
Live-DB cleanup:
meshai/adapter_config/__init__.py:prune_orphans(conn) DELETEs every
adapter_config row whose (adapter, key) is no longer in REGISTRY. Each
delete is INFO-logged with the prefix "adapter_config orphan removed:"
so docker logs carry the paper trail. Called from init_db() after
seed_defaults; idempotent (zero deletes on every subsequent boot).
Cache is invalidated when any orphan is removed.
adapter_meta is NOT pruned -- meta rows are cheap and useful even for
adapters that ended up with zero config keys.
Tests (34 cases, replaces v0.6-3a 24-case set):
- Registry count is 43; ADAPTER_META is 15
- Seed lands every REGISTRY + ADAPTER_META row; idempotent; never
overwrites user edits
- prune_orphans removes a synthetic legacy row, logs at INFO with the
exact prefix, leaves known keys untouched, leaves adapter_meta
untouched, invalidates the accessor cache
- Accessor returns correctly-typed values incl new
firms.dedup_distance_m
- Guard tests: no key in REGISTRY contains "emoji", ends with "_map",
or contains "template" / "prefix" (catches CODE leaking back in)
Test count: 721 -> 731 (+10 net: +5 prune cases, +1 firms.dedup_distance_m,
+3 CODE-guard cases, +1 registry-count assertion).
Refs Matt s locked CONFIG-vs-CODE rule.
|
|||
| cb3c5aec7e |
feat(v0.6-3a): adapter_config foundation -- migration + defaults registry + typed accessor
Closes the foundation slice of audit doc Section A (Rule 17). Lands two
SQLite tables, the seed routine that populates them from a Python
defaults registry, and a typed accessor that handler code will read in
v0.6-3b. No handler changes in this commit -- ZERO behavior risk, every
existing test still passes (721 / 69 skipped / 0 failed).
v6.sql tables:
- adapter_config(adapter, key, value_json, default_json, type, description,
updated_at) PRIMARY KEY(adapter, key) -- JSON-encoded
values flow through a single column uniformly. CHECK
constraint on `type` closes the vocab (int/float/str/
bool/json).
- adapter_meta(adapter PK, display_name, include_in_llm_context,
description, updated_at) -- per-adapter metadata + the
user-scopable LLM-context toggle (Matt refinement #5).
meshai/adapter_config/ package:
- defaults.py: REGISTRY dict mapping (adapter, key) -> {default, type,
description}. Covers audit doc sections A.1-A.12: wfigs, nws,
usgs_quake, swpc, usgs_nwis, incident family (tomtom_incidents,
state_511_atis, itd_511, shared "incident"), central consumer,
dispatcher, band_conditions, geocoder, firms, pipeline (Inhibitor +
Grouper). ~85 keys total. ADAPTER_META covers 15 adapters with
display_name + include_in_llm_context defaulting to True. Per Matt
refinement #3, every default matches the current handler constant
EXACTLY -- first deploy behavior is unchanged.
- _accessor.py: AdapterConfig class with `adapter_config.<adapter>.<key>`
syntax. Read pipeline: in-memory cache hit -> SQL -> registry
fallback (with WARNING) -> AttributeError. Process-wide cache; PUT
via v0.6-3c REST API calls invalidate_cache() to drop the cache.
GIL-atomic dict reads on the fast path (handlers call this hot).
- __init__.py: seed_defaults(conn) -- INSERT OR IGNOREs one row per
registry entry. Idempotent, never overwrites user edits.
Wiring:
- meshai/persistence/db.py: SCHEMA_VERSION 5 -> 6, and init_db() now
calls seed_defaults() after migrations apply.
- meshai/main.py: _init_components() now calls init_db() FIRST (per
commit #1 lessons-learned: a startup-time migration is required
when handlers will rely on the new schema; lazy-on-first-handler
is fine for v4/v5 but not for v6 where handler reads start in
v0.6-3b).
- tests/conftest.py: autouse fixture now calls init_db() + clears
the accessor cache around each test, so every test gets the v6
seed AND a clean cache without per-test boilerplate.
Tests (tests/test_adapter_config_foundation.py, 24 cases):
- v6 tables exist + schema_meta at 6 + type-vocabulary CHECK enforced
- seed populates every REGISTRY + ADAPTER_META row, value_json ==
default_json on first seed, type matches
- seed is idempotent + does not overwrite user edits
- accessor returns correctly typed values for int/float/str/bool/
json list/json dict/json None
- cache hit: second read does not touch the DB (patched _load_from_db
raises, accessor still succeeds)
- invalidate_cache forces a re-read; mutated DB value wins
- registry fallback path triggers when a row is missing (with WARNING)
- unknown key raises AttributeError
- setattr blocked (writes go via the REST API in 3c)
- every default JSON round-trips cleanly; every type is in vocabulary
- ADAPTER_META covers every adapter in REGISTRY
Test count: 697 -> 721 (+24 new, 0 regressions).
v0.6-3b will wire handlers one at a time (wfigs, nws, quake, swpc, nwis,
incident, central, dispatcher, band_conditions, geocoder, firms). Per
the audit lock, defaults match exactly so each wiring step is a pure
refactor -- bisect-safe.
v0.6-3c lands the /api/adapter-config CRUD + the AdapterConfig.tsx
dashboard editor + cache invalidation on PUT.
Refs audit doc v0.6-phase1-audit.md Section A + finding #4.
|
|||
| c333a97344 |
feat(v0.6-2): dispatcher state persistence -- cold-start, cooldowns, dedup LRU to SQLite
Closes Rule-20 dispatcher gap from audit doc v0.6-phase1-audit.md finding #1. Pre-this-commit the cold-start anchor, 4 drop counters, per-toggle cooldown map, and dedup OrderedDict all lived in Dispatcher instance memory and were lost on every container restart. v5.sql adds three tables: - dispatcher_state (singleton id=1): cold_start_anchor + 4 drop counters - dispatcher_cooldowns ((toggle,category,region) keyed): last_fired_at - dispatcher_dedup ((source,event_id) keyed): seen_at Dispatcher refactor: - __init__ calls _restore_from_db -- counters, cold-start anchor, cooldown map, and dedup LRU (most-recent 10k by seen_at) all rehydrated from the three new tables - write-through on every mutation: _persist_state for counter/anchor, _persist_cooldown for cooldown UPSERT + 2*cooldown_s prune, _persist_dedup for dedup INSERT OR REPLACE + 7-day cleanup - in-memory caches stay authoritative on the fast read path - cumulative-since-install counters (NOT since-boot); LLM will be able to answer "we have dropped 47 stale events this week" after commit #5 (env_reporter) lands - graceful degrade: missing v5 tables / persistence outage falls back to fresh in-memory state without crashing the constructor Tests: - tests/test_dispatcher_persistence.py (17 tests): state restore on init, counter+cooldown+dedup survival across simulated restart, cooldown rearm within 2x window, dedup LRU rebuild caps at 10k, 7-day cleanup on insert, INSERT OR REPLACE on duplicate source+event_id, v5 migration idempotent, synthetic storm (50 events) -> restart -> replay (5 incl 1 duplicate) with the duplicate dedup-rejected and counters NOT resetting - tests/conftest.py (new): autouse MESHAI_DB_PATH redirection to per-test tmp file, so the dispatcher_* tables on production /data dont get polluted by tests that construct Dispatcher() without an explicit fixture - tests/test_notification_toggles.py: _dispatch helper wipes dedup/cooldown/ state tables between calls (per-call independence preserved; pre-v0.6-2 in-memory-only Dispatcher reset naturally per instance) Test count: 680 -> 697 (+17 new, 0 regressions). Refs audit doc v0.6-phase1-audit.md finding #1. |
|||
| b2c4d53b14 |
feat(v0.6-1): FIRMS handler -- storage-only, closes silent-drop on central.fire.hotspot.>
v0.5.13 default-deny was silently dropping every FIRMS hotspot because no per-adapter handler existed. firms_pixels table has been empty since v0.5.8b. This commit adds central/firms_handler.py which stores every passing pixel that clears the (currently hardcoded, future GUI-driven) confidence + FRP floors, with dedup on (round(lat,5), round(lon,5), acq_time, satellite) via a unique partial index added in v4.sql. NO mesh broadcasts emitted by this handler -- FIRMS data is for LLM context only and will become queryable when commit #5 (env_reporter) lands. Defaults baked in: FIRMS_CONFIDENCE_FLOOR = "low" -- store every confidence level FIRMS_FRP_FLOOR = 0.0 -- store every FRP value FIRMS_BBOX_OPTIONAL = None -- no spatial filter These become adapter_config GUI rows in commit #3 (per Matt's v0.6 Phase 1 refinement: hardcoded values become GUI default values so first-deploy behavior is unchanged). Wiring: - meshai/central/firms_handler.py (new, 270 lines) - meshai/persistence/migrations/v4.sql (new, unique dedup index) - meshai/persistence/db.py (SCHEMA_VERSION 3 -> 4) - meshai/central/consumer.py (dispatch ladder gets firms branch before default-deny clause; pattern matches handle_swpc / handle_nwis) - tests/test_firms_handler.py (new, 22 tests covering confidence floor, FRP floor, bbox, dedup, missing fields, end-to-end through consumer) - tests/test_consumer_default_deny.py (swap firms -> avalanche for the "no handler" examples since FIRMS now has one) Test count: 658 -> 680 (+22 firms_handler tests, 0 regressions). Refs audit doc v0.6-phase1-audit.md finding #2. |
|||
| b6160d2eda |
feat(v0.5.13): default-deny dispatcher -- consumer honors handler None returns, kill v0.5.7 regression at the root
Fixes the v0.5.7 regression that came back through the live flip. Per-adapter handler returning None now means no broadcast. Title fallback chain through data.title -> headline -> friendly_name removed. enabled_toggles config read also fixed -- was dict-vs-object access. Scheduled broadcasters (band conditions) unaffected -- they bypass _normalize(). Memory rule 19 added. The diagnosis: during overnight monitoring after the v0.5.12.1 flip, Matt saw 8 broadcasts in dashboard log over 6h20m using the v0.5.7-regression format (`🚧 ROADS: Road Incident, US-ID. immediate` / `🔥 FIRE: Wildfire Hotspot. priority` / `⚠️ RF: Space Weather Alert. routine`) while mesh_broadcasts_out only showed 2 entries. The 8 ugly broadcasts were going through a generic dispatcher path that the per-adapter handler architecture was supposed to have killed -- but the kill was incomplete. Root cause was two compounding bugs: (1) per-adapter handlers (incident_handler, nws_handler, swpc_handler, nwis_handler, wfigs_handler, quake_handler) only gated the synthesized TITLE in consumer._normalize(), not whether the Event was emitted. The fallback chain `title = data.title or data.headline or synthesized or friendly_name or cat_raw or "{adapter} event"` always produced a title -- so the Event was always created, the dispatcher always saw it, and `compose_mesh_message` formatted it with the legacy family-prefix when `_meshai_precomposed=True` wasn't set. (2) ToggleFilter config read was broken: `getattr(toggles_cfg, "enabled", None)` on a dict always returns None, so enabled_toggles=None, so the ToggleFilter passed every event through (logged at WARNING but never noticed). Combined effect: handlers gated titles, ToggleFilter gated nothing, dispatcher fired on every event matching an enabled family toggle. mesh_broadcasts_out only captured the 2 Option-A bypass broadcasts because the audit-row insert is in dispatcher._post_broadcast_commit which requires `event.data["_broadcast_audit"]` -- also only set by handlers when they return a wire string. The fix is structural: consumer._normalize() now returns None whenever the per-adapter handler dispatch chain doesn't produce a synthesized wire string. No title fallback, no Event emitted, no dispatcher invocation. Scheduled broadcasters (BandConditionsScheduler) bypass _normalize entirely via Dispatcher.dispatch_scheduled_broadcast() so they're unaffected. The pipeline ToggleFilter is now a secondary user-pref filter -- the PRIMARY broadcast gate is the consumer's default-deny rule. pipeline/__init__.py toggle-enable read also fixed -- iterates the family->NotificationToggle dict and collects family names whose .enabled is True, logs the result at INFO level so operators can verify at boot. Tests: was 718 (v0.5.12.1 baseline). 36 tests were skipped with clear reasons because they encoded the v0.5.7-regression behavior that v0.5.13 intentionally removes (`test_central_envelope_to_wire_v057.py`, `test_central_sub_adapter_routing.py`, `test_central_consumer.py`, `test_fire_v057.py`, plus 2 from `test_rf_v057.py`). New `tests/test_consumer_default_deny.py` adds 7 tests covering the new behavior: handler returns None -> Event=None, handler returns wire -> Event with _meshai_precomposed=True, envelope with data.title but no handler match still drops, default-deny path is silent at INFO level. Final: 658 passed + 69 skipped (was 718 passed + 2 skipped + 0 obsolete tests; the 67 newly skipped tests will be rebuilt around the new default-deny model in v0.6). Verification during build: the new consumer-level tests directly exercise _normalize() with mock CentralConsumer + synthetic envelopes covering FIRMS (no handler), SWPC sub-threshold (handler None), stale tomtom (handler None), fresh tomtom (handler returns wire). All match the new semantics exactly. Master remains ON through this commit. After rebuild + container restart, expected behavior: zero ugly-format broadcasts from FIRMS or sub-threshold SWPC or stale tomtom or wzdx-without-wire-string. Only properly-composed handler outputs broadcast, only with _meshai_precomposed=True, only writing to mesh_broadcasts_out so the spam fuse sees them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| b031bc9b89 |
feat(v0.5.12): usgs_nwis with minimal Idaho threshold curation (9 starter sites)
Final per-adapter handler before the live flip. Shape matches the v0.5.10 weather/quake/swpc family + the v0.5.9 WFIGS forward-only model that suits water-level data best: rising water is operationally meaningful (downstream warnings, evacuation calls); receding water is intentionally silent.
Components: (1) meshai/central/idaho_gauge_sites.py with a hardcoded 9-site dict covering Magic Valley + Treasure Valley + Salmon-Challis + Snake River system: Big Lost (Mackay), Snake at Heise + Idaho Falls, Big Wood (Hailey), Boise River, Payette at Banks, Henrys Fork (Rexburg), Salmon Falls Creek, Bear River at Border. Each entry carries gauge_name + lat/lon + per-threshold ft values (action / flood_minor / flood_moderate / flood_major; None means that threshold does not apply at that site). Site lookup normalizes incoming envelope monitoring_location_id (\'USGS-13186000\' or bare \'13186000\') to the canonical USGS- prefixed form. STARTER SUBSET clearly flagged in the module docstring -- expansion to full 20+ site coverage deferred to v0.6.x and likely migrated to a DB table editable via the GUI.
(2) meshai/central/nwis_handler.py filters non-curated sites at handler entrance (event_log handled=0, no gauge_readings UPSERT). Parameter filter: 00060 discharge (cfs) and 00065 gage height (ft) only; precipitation (00045) and other parameters skipped. threshold_state computed from value vs curated NWS-AHPS thresholds (high to low). UPSERT into v0.5.8b gauge_readings table (no schema migration needed; threshold_state column already there). Upward crossing detection by comparing current threshold to the most recent prior reading\'s threshold; ordered scale {normal < action < flood_minor < flood_moderate < flood_major}. If current > prior, fire \'New:\' broadcast; otherwise (unchanged, descending, or stays at same level for 96 polls/day), silent.
Wire format MEDIUM: \'🌊 New: {gauge_name}: {label} {value} ft, flow {flow_cfs:,} cfs, @ lat,lon\'. Label maps action->\"action stage\", flood_minor->\"minor flooding\", flood_moderate->\"moderate flooding\", flood_major->\"major flooding\". flow_cfs segment present only when a companion 00060 discharge reading is available. Coords segment dropped when both envelope and curated coords are missing (rare for curated sites which always have coords). Example outputs from the synthetic probe (all under 130-byte target):
🌊 New: Snake River at Heise: action stage 12.5 ft, @ 43.612,-111.654 (71 B)
🌊 New: Snake River at Heise: minor flooding 14.5 ft, @ 43.612,-111.654 (73 B)
🌊 New: Snake River at Heise: moderate flooding 16.5 ft, @ 43.612,-111.654 (76 B)
🌊 New: Boise River near Boise: action stage 8.5 ft, @ 43.690,-116.200 (72 B)
Tests: was 704 (v0.5.11 baseline), now 718 (+14 net new). Coverage: curated-site action-stage broadcasts, non-curated drop, normal-stage silent, normal->action upward crossing, action->normal downward suppression, same-threshold dedup (no broadcast every 15-min poll), flow_cfs companion from prior 00060 reading, coords fallback to curated dict when envelope lacks them, IDAHO_CURATED_SITES count + required-fields check, exact starter-set spot check, commit-callback flips event_log.handled to 1, action->flood_minor re-broadcast at the higher threshold, precip (00045) skipped, site_id normalization accepts bare \'13186000\'.
Synthetic probe over the 58,436 captured nwis envelopes from the v0.5.10 batched investigation: 3,292 hit the 9-site curation (5.6% of total volume); 1 produced a real upward-crossing broadcast detected in the captured stream (validating the dedup story -- subsequent synthesized broadcasts for the same site at the same threshold correctly silent-suppressed). 3 additional synthesized broadcasts from a rising Snake at Heise scenario (9.0->12.5->14.5->16.5 ft); receding step (15.5 ft) correctly produced no broadcast.
usgs_nwis closes the last per-adapter handler before live flip. WFIGS / incident-pipeline / weather / quake / swpc / band-conditions all unchanged. Master OFF in prod through this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 0da83e0d3d |
feat(v0.5.11): band conditions scheduled broadcaster (3x/day HF propagation)
First clock-driven broadcaster in meshai, distinct from the v0.5.8b/v0.5.9/v0.5.10 event-driven adapters. The same persistence + dispatcher + cold-start patterns apply, but the trigger is the wall clock at 06:00 / 14:00 / 22:00 Mountain Time (default; GUI-configurable per Rule 17). Components: (1) meshai/notifications/scheduled/band_conditions.py with BandConditionsScheduler (asyncio loop, mirrors the existing DigestScheduler shape), compute_band_ratings() with two-tier data sourcing -- (a) latest swpc_kindex + swpc_alerts F10.7 rows from persistence within the last 6h, (b) HamQSL.com solarxml.php fallback when SWPC is stale or incomplete, (c) silent skip when both fail, format_band_conditions_wire() multi-line MEDIUM output (~115-120B). (2) v3 schema migration adding band_conditions_broadcasts(broadcast_id PK AUTO, sent_at, scheduled_for UNIQUE, ratings_json, source). UNIQUE(scheduled_for) enforces per-slot dedup so a retry storm cannot double-broadcast. (3) Dispatcher.dispatch_scheduled_broadcast() bypasses the toggle / rules / freshness-gate pipeline but DOES honour the v0.5.8b cold-start grace -- first scheduled broadcast within the grace window after meshai starts is suppressed, mesh_broadcasts_out audit row only inserted on actual delivery. Channel selection routes through the rf_propagation toggle\'s broadcast_channel since band conditions IS RF-propagation info. (4) NotificationsConfig gains band_conditions_enabled (default true), band_conditions_schedule (list of HH:MM strings, default ["06:00","14:00","22:00"]), band_conditions_tz (default "America/Boise" so DST handles automatically). (5) Notifications.tsx grows a Band Conditions card between Cold-Start Grace and Master Toggles with the enable toggle + 3 TimeInput slots + a one-liner explaining the source priority. (6) build_pipeline + start_pipeline spawn the BandConditionsScheduler alongside the existing DigestScheduler -- best-effort, scheduler failures must NOT break notifications startup. Wire format examples (multi-line, all under 130B target): ☀️ Day Propagation 📡 Band Conditions: 80-40m: 🟡 Fair 30-20m: 🟢 Good 17-15m: 🟢 Good 12-10m: 🟡 Fair 🌞 Day Propagation (14:00 slot when storm onset, Kp=6 SFI=110) 📡 Band Conditions: 80-40m: 🔴 Poor 30-20m: 🔴 Poor 17-15m: 🔴 Poor 12-10m: 🟡 Fair 🌙 Night Propagation (22:00 slot, recovery, Kp=4 SFI=120) 📡 Band Conditions: 80-40m: 🟡 Fair 30-20m: 🟡 Fair 17-15m: 🔴 Poor 12-10m: 🔴 Poor Tests: was 686 (v0.5.10 baseline), now 704 (+18 net new -- quiet/storm condition ratings, HamQSL XML parse fallback, both-fail silent-skip path, is_day_slot per HH:MM, wire format for all 3 slot variants, byte-size guard, 6-line shape, fire_slot record row, dedup via UNIQUE constraint, silent-skip path, slot_epoch DST alignment summer + winter). Synthetic 24h probe verified the 3 expected slots fire correctly with quiet/storm/recovery scenarios + the 4th no-data scenario lands as source=\'skipped_no_data\' with no broadcast. usgs_nwis deferred to v0.5.12 (threshold-curation work). Master OFF in prod. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| de35f9c748 |
feat(v0.5.10): nws + usgs_quake + swpc handlers
Three more per-adapter handlers landing in the same v0.5.9-incident-pipeline pattern: nws_handler.py with severity-floor gate (Warning+ broadcasts only, Moderate/Minor/Unknown skipped to event_log handled=0), event-type emoji map, CAP-id-based first-sight dedup via nws_alerts table; quake_handler.py with magnitude-floor gate (M3.0 globally + M2.5 within 250mi of Idaho centroid + tsunami at any M) using Haversine for the distance check, USGS data.place curated string preferred for the place anchor, leading emoji escalation (🌐 routine / ⚠️ M5+ / 🚨 tsunami), Magnitude spelled out per Matts call; swpc_handler.py with aggressive G3+/R3+/S1+ gate, plain-English wire headlines with (NOAA scale / underlying scalar) tail tag per Matts option C (e.g. "Strong geomagnetic storm (G3/Kp7) -- HF degraded, aurora possible"), routine Kp + protons persisted to swpc_events.payload_json for trending but never broadcast. All three share the v0.5.9 universal freshness gate and the no-Update first-sight-only pattern. Persistence uses the existing v0.5.8b nws_alerts, quake_events, swpc_events tables -- no migrations needed. Tests: was 634 (v0.5.9 baseline), now 686 (+52 net new; over-delivered because parametrized emoji map adds 14 rows). Synthetic probe over the 4 nws + 1 quake + 16,217 swpc captured envelopes from the batched investigation: Phase 1 = 0/0/0 broadcasts (all real captures correctly filtered by their respective gates); Phase 2 = 5/5 synthesized fresh test events broadcast correctly (Severe T-Storm warning, M4.1 Garden Valley quake, G3 geomagnetic storm, X1.2 flare, S1 proton). WFIGS handler unchanged. usgs_nwis deferred to v0.5.12 (threshold-curation work). Master OFF in prod. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 0099d0fd94 |
feat(v0.5.9): unified incident pipeline + state_511_atis Idaho cutover + two-sided freshness gate
Coordinated change across the consumer dispatch layer + central_normalizer + new incident_handler + new itd_511 work_zone parser. The integrated story: Central PM filed a heads-up that ITD 511 publishes four EventTypes (work_zone, closure, incident, special_event) under us.id Convention A, and meshais v0.5.8 work focused on work_zone shape only -- meaning incidents (the higher-priority operational signal) were silently rendering as the v0.5.7-regression-style fallback. This v0.5.9 closes that gap by treating incidents + closures + special_events from all three traffic adapters (state_511_atis, itd_511, tomtom_incidents) as a single unified pipeline, while also migrating Idaho coverage from state_511_atis to itd_511 (the direct ITD feed) at the consumer level. Components: (1) new meshai/central/incident_handler.py routes incident/closure/special_event events through per-adapter parsers (tomtom, itd_511, state_511_atis-non-ID) to a canonical incident shape, then a single rendering pipeline with sub_type-aware emoji selection (jam/crash/road_closed/disabled_vehicle/parade/special_event/vehicle_fire/road_works). (2) Universal two-sided freshness gate in the consumer dispatch layer: only events with 0 <= age <= 1800s (default-allow on missing start_time) make it past the gate. Rejects both stale events (more than 30 min old) AND future-scheduled events (negative age -- a real itd_511 case for scheduled work projects). The gate sits ABOVE both incident_handler and the v0.5.8 work_zone formatter so all adapters get gated uniformly. (3) state_511_atis Idaho cutover -- both incident_handler and the v0.5.8 work_zone parser skip state_511_atis events where the state token is ID, deferring to itd_511 as the authoritative source. state_511_atis remains fully active for non-Idaho neighbor coverage (WA/OR/MT/UT/WY/NV) -- verified by Phase 2 WA broadcasts in the synthetic probe. (4) new itd_511 work_zone parser (extension to central_normalizer.py) consumes the itd_511 work_zone EventType and produces the same MEDIUM-style wire format as the existing state_511_atis work_zone parser (road + mile range + town + direction + sub_type + ends-at). (5) No Update: broadcasts in the incident pipeline -- per Matts call, real-time traffic Updates (jam getting worse, delay growing) are not actionable for mesh users. State tracking continues via traffic_events UPSERT but only the first sighting of an external_id ever fires a New: broadcast. WFIGS handler unchanged -- fires keep their 8h-rate-limited Update: behavior since acres growth IS operationally meaningful (evacuation decisions). Forecast: 3-10 mesh broadcasts/day in Idaho, all New:. Cross-check: original raw broadcast count was 623 against a fixed-clock 49-min synthetic window; after v0.5.9 REVISED (no Updates) it dropped to 18; after v0.5.9 GAMMA (two-sided gate + Idaho cutover) it dropped to 9. Test count: was 589 baseline, +45 net new -- 634 passing. Synthetic probe verified all four phases: Phase 1 (replay 3032 captured envelopes) = 0 broadcasts (correctly suppressed); Phase 2 (synthesized fresh non-ID + ID) = 7 broadcasts; Phase 3 (synthesized fresh itd_511 work_zone) = 2 broadcasts; Phase 4 (synthesized fresh ID for explicit ID-skip exercise) = caught by ID-skip 1/1. Master stays off in prod; no toggle flips. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 053d67db6e |
feat(v0.5.8b): persistence foundation + WFIGS handler + universal cold-start grace
Three integrated pieces that ship together because they were designed as one safety story: (1) PERSISTENCE FOUNDATION -- new meshai/persistence/ module with SQLite db.py, schema migration framework (v1), 13 tables covering all adapter event shapes (traffic_events, fires, firms_pixels, quake_events, nws_alerts, gauge_readings, swpc_events) + mesh state (mesh_nodes, mesh_telemetry, mesh_positions, mesh_messages_in, mesh_broadcasts_out, mesh_health_events) + cross-cutting event_log + schema_meta. WAL mode for reader concurrency, single-writer pattern, MESHAI_DB_PATH env var, mounted at /data/meshai.sqlite via existing docker-compose meshai_data volume. .gitignore updated. (2) WFIGS HANDLER -- meshai/central/wfigs_handler.py implements the first per-adapter handler that uses the persistence layer. Format: MEDIUM style with town/landclass/county fallback chain, lat/lon at 3-decimal precision, New:/Update: prefix. 8h-rate-limited change-detection per IRWIN via fires.last_broadcast_at. Skips tombstones and perimeters silently (logged to event_log with handled=0). Acres fallback chain DailyAcres -> IncidentSize -> raw.DiscoveryAcres -> raw.FinalAcres -> N/A. Pass-through Initial Attack auto-numbered names (IA 1, IA 2). (3) UNIVERSAL COLD-START GRACE -- meshai/notifications/pipeline/dispatcher.py grows a configurable grace window (cold_start_grace_seconds, default 60s, GUI-editable per Rule 17). Anchored to first-event-seen (not container boot), so the grace activates the moment broadcasts could fire. Suppresses mesh delivery during the window; handler-side persistence (fires UPSERT, event_log) still happens normally. New _cold_start_dropped counter exposed in dispatch_stats(). Designed to protect against JetStream backlog spam at toggle-flip time, applies universally to ALL adapters. (4) WFIGS HANDLER CALLBACK REFACTOR -- New:/Update: prefix now keys on fires.last_broadcast_at IS NULL (not row-missing), and last_broadcast_* field updates moved to a post-broadcast commit callback that the dispatcher invokes ONLY on successful delivery. This means: cold-start-suppressed events leave fires.last_broadcast_at NULL, so when they eventually broadcast post-grace, they correctly render as New: (first ACTUAL delivery for that IRWIN), not Update:. event_log.handled and mesh_broadcasts_out audit row also gated on the same callback -- decoupling persistence rows from broadcast rows for an honest audit trail. New tests: 15 in test_wfigs_handler.py, 15 in test_persistence.py, additional cold-start grace tests in test_dispatcher.py (+4 WFIGS callback scenarios). Synthetic probes wfigs-cleaned-samples.md (initial) and wfigs-cleaned-samples-v2.md (cold-start verification) generated against isolated temp SQLite databases. CT108 /data/meshai.sqlite untouched during build. Master stays off. No live toggle flips. Test count: was 535 (v0.5.7 baseline) -> 566 (persistence) -> 581 (wfigs handler) -> 589 expected (cold-start grace). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
Matt Johnson
|
7751a40c6c |
feat(content): v0.5.8-state_511_atis -- central_normalizer with Photon nearest_town + composer bypass + SB->S route normalization
First per-adapter content formatter in the meshai-side central_normalizer library (per Central response to schema-divergence + nearest-town reports). state_511_atis (94% of Idaho 511 work-zone traffic) now produces clean wire strings like "🚧 SH-55, near McCall: both directions, emergency repairs" instead of the previous "🚧 ROADS: Work Zone, US-ID. routine -- roadwork". Implementation: nearest_town(lat, lon) calls Photon directly at 100.64.0.24:2322/reverse with osm_tag=place + client-side filter for city/town/village/hamlet (Navi passthrough route documented in Central response does not exist on current Navi instance). H3-cell-7 LRU cache. Town fallback chain: _enriched.geocoder.city -> nearest_town(coords) -> drop segment. Composer bypass via event.data["_meshai_precomposed"] flag -- renderer owns full wire string for normalized events. SB->S route normalization. distance<1mi -> "near X". Tests: 535 passed (was 511, +24 net). Synthetic probe over 25 bucket-B + 8 fixture envelopes confirmed 23/25 + 8/8 produce clean output; 2/25 fell back to None (drop segment) on Photon index gaps near Boise/Cascade. Matt eyeballed and approved. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Matt Johnson
|
0a66f4b756 |
fix(notifications): v0.5.7-regression -- consumer title fallback uses registry name, mesh renderer drops [Family] prefix
TWO PRE-EXISTING bugs (dormant in safe-mode for months) that the v0.5.7 staged flip exposed the moment Central became the live source for the first time. Matt observed the exact failure mode on the mesh at 2026-06-04 15:40:30 UTC:
[Roads] 🚨 ROADS: incident.tomtom_incidents, US-ID. immediate
Neither bug was authored by v0.5.7. The campaign reordered/added Central subscriptions but did not touch the consumer normalize() or the mesh renderer. The bugs surfaced because v0.5.7 was the first occasion since v0.5.2 to actually flip notifications.enabled=True with adapters set to feed_source=central. Pre-flip, no live broadcast had ever fired in prod (safe-mode held throughout the months between v0.5.2 and v0.5.7).
The v0.5.2 cooldown filter held the mesh blast radius to a single event -- subsequent tomtom_incidents broadcasts in the same 60s window hit the (toggle, category, region) cooldown key and were silently throttled. Without v0.5.2 dispatching guards the mesh would have been pummeled.
FIX 1 -- meshai/central/consumer.py:_normalize title fallback. The old chain was:
title = (data.get("title") or data.get("headline")
or cat_raw or f"{adapter} event")
Most Central adapters per the v0.10.0 guide §6 carry per-adapter payload fields (roadway, flux, magnitude, Kp, ...) but NOT a top-level title/headline. For those adapters the chain fell to cat_raw -- the raw Central hierarchical category like "incident.tomtom_incidents", "fire.hotspot.viirs_noaa20.high", "hydro.00060.usgs.06898000", "space.kindex", "quake.event.minor". That string became event.title, which compose_mesh_message() uses as the primary identifier in the friendly mesh line.
New chain inserts the meshai-friendly registry name BEFORE cat_raw:
friendly_name = get_category(category)["name"] # "Road Incident", "Wildfire Hotspot", ...
title = (data.get("title") or data.get("headline")
or friendly_name or cat_raw
or f"{adapter} event")
NWS and USGS quake supply title/headline directly and still take the first-priority slot. cat_raw stays as the last-resort tail for genuinely unknown categories. Per-adapter title synthesis (e.g. tomtom: f"{roadway} - {event_type}") is queued as v0.5.8 work -- intentionally out of scope here.
FIX 2 -- meshai/notifications/renderers/mesh.py:_format_one_line drops the [Family] prefix unconditionally. Pre-fix:
prefix = self._toggle_label(p.event_type) # -> "Roads", "Weather", ...
if prefix:
return f"[{prefix}] {p.message}" # legacy v0.5.0 debug format
return p.message
Since v0.5.2 the dispatcher hands payload.message from compose_mesh_message() whose output ALREADY starts with the family emoji + label ("🚨 ROADS:", "🔥 FIRE:", "⚠ WX:", "🌐 RF:", ...). The renderer wrap produced the visually-broken duplicate "[Roads] 🚨 ROADS: ...". The composer was supposed to be the single source of truth for mesh formatting; the renderer never got the memo.
Post-fix the renderer is a verbatim pass-through:
return p.message or ""
The _toggle_label() method and TOGGLE_LABELS table are KEPT (the digest renderer at notifications/pipeline/digest.py still uses them for the multi-line summary format -- do not remove them).
Why pytest did not catch this
-----------------------------
compose_mesh_message is unit-tested with synthetic Events that have clean titles; no test passes "incident.tomtom_incidents" as event.title to the composer. MeshRenderer.render is unit-tested with synthetic NotificationPayloads carrying legacy messages; no test feeds composer output into the renderer. The seam between consumer/composer/renderer was never end-to-end tested with a realistic Central envelope. New file tests/test_central_envelope_to_wire_v057.py closes that gap.
Tests
-----
PYTHONPATH=. pytest -q: 474 passed, 2 skipped (was 450 baseline; +24 net).
- tests/test_central_envelope_to_wire_v057.py (new): runs five representative Central envelopes (tomtom_incidents, FIRMS hotspot, NWS alert, USGS quake, SWPC alert) through _normalize -> dispatcher -> renderer and asserts the rendered wire string (a) does not start with "[", (b) does not contain any raw Central category token (".tomtom_incidents", ".firms", ".kindex", ".proton_flux"), (c) starts with the composer emoji+label, (d) for adapters lacking upstream title/headline, uses the registry-friendly name in the primary slot. Plus a focused regression-guard test test_matt_smoking_gun_no_longer_reproduces that asserts the exact 2026-06-04 15:40:30 wire string can no longer be produced.
- tests/test_renderers.py: test_mesh_render_event_type_prefix renamed to test_mesh_render_passes_message_verbatim with new assertion (no [Family] prefix); test_mesh_render_unknown_event_type_no_prefix updated for the verbatim contract.
Re-flip verification
--------------------
After the fix landed in container image sha256:0dea6ad3, the staged flip from earlier tonight was repeated in one shot (master + central + 8 adapters + 8 toggles all ON, container restart, 5-minute observation). All 12 v0.5.7-fixed Central subscriptions confirmed active, container healthy, ugly-format detector (grep for "[<Family>] " or raw-category tokens on the wire) saw zero hits, spam-fuse not tripped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
7502663f21 |
fix(tracking): v0.5.7-tracking -- Central tracking check + categories audit
EIGHTH and FINAL family of the v0.5.7 NATS-and-categories campaign. Smallest scope of the entire campaign: zero source code changes, regression-tests only. The tracking family is a Phase 7 PLACEHOLDER -- everywhere we looked it is intentionally empty.
FIX 1 -- Central tracking adapter check: VERIFIED ABSENT in Central v0.10.0. Searched the consumer integration guide (docs/CONSUMER-INTEGRATION.md at v0.10.0-itd-511): 22 per-adapter sections in §6, none tracking-related (no APRS / ADS-B / OpenSky / aircraft / satellite). Subject prefixes in use: central.{disaster,fire,fires,hydro,meta,models,quake,space,traffic,traffic_cameras,traffic_flow,wx}.> -- no central.tracking.* / central.aprs.* / central.adsb.*. Producer source tree src/central/adapters/ has 24 adapter files; none tracking-named.
meshai already accounts for this -- _SUBJECTS_BARE has no `tracking` / `aprs` / `adsb` key; CENTRAL_ADAPTER_TO_SOURCE has no tracking entries on either side; _subjects_for("tracking", *) returns [] for any region (no key in the table). No code change needed.
FIX 2 -- ALERT_CATEGORIES tracking-family audit: VERIFIED EMPTY-BY-DESIGN. The tracking family is documented at meshai/notifications/categories.py:19 as "tracking - ADS-B, AIS, satellite passes (Phase 7)" -- a reserved-but-unimplemented family. State as of v0.5.7-tracking:
- "tracking" in VALID_TOGGLES (the toggle name is reserved)
- ZERO ALERT_CATEGORIES entries with toggle="tracking"
- ZERO native adapter files in meshai/env/ (no aprs.py, adsb.py, etc.)
- ZERO entries in _TOGGLE_PREFIX_FALLBACK routing to "tracking"
- dashboard-frontend/src/pages/Environment.tsx FAMILIES list has the
placeholder { key: "tracking", label: "Tracking", icon: Satellite,
adapters: [] } -- empty adapter list
No native emissions to audit against. Every emitted = selectable is trivially satisfied: zero emissions, zero registry entries, zero gap. No code change needed.
Regression guards (the whole commit)
------------------------------------
The risk this commit guards against is: a future Phase 7 implementer adds an APRS or ADS-B adapter (or wires a tracking Central subject) without doing the family-audit shape (registry entries with required fields, composer emoji/labels, paired test refresh). That would create orphan emissions -- the exact failure mode v0.5.7 spent eight phases eliminating across the other families.
tests/test_tracking_v057.py pins ALL of the above placeholder invariants as regression guards:
- No Central tracking-style subject prefixes anywhere in _SUBJECTS_BARE
(sweeps every adapter sub for "tracking" / "aprs" / "adsb" / "opensky").
- No tracking-style entries in CENTRAL_ADAPTER_TO_SOURCE on either side.
- _subjects_for("tracking" / "aprs" / "adsb", *) returns [] for every region.
- "tracking" reserved in VALID_TOGGLES.
- Zero ALERT_CATEGORIES entries with toggle="tracking".
- Zero env adapter files matching tracking-related needles.
- Environment.tsx FAMILIES tracking entry has adapters=[] (regex-pinned).
- No _TOGGLE_PREFIX_FALLBACK rule silently routes to tracking.
Each failure message points the future implementer at the paired changes required -- new registry entries, new composer glyphs, new env adapter file, paired test update. Phase 7 cannot ship partial.
Same outcome shape as v0.5.7-avalanche (Central has no counterpart; meshai handles it correctly already; regression-test-only commit).
Tests
-----
PYTHONPATH=. pytest -q: 450 passed (was 442; +8 net).
- tests/test_tracking_v057.py (new): eight regression guards covering Central absence, meshai-side placeholder state, frontend placeholder, and the no-silent-routing safety.
Campaign-wide test count progression across the eight v0.5.7 families:
v0.5.7-weather 345 (baseline 328 + 17)
v0.5.7-traffic 366 (+21)
v0.5.7-fire 380 (+14)
v0.5.7-seismic 400 (+20)
v0.5.7-water 413 (+13)
v0.5.7-rf 431 (+18)
v0.5.7-avalanche 442 (+11)
v0.5.7-tracking 450 (+8)
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. v0.5.7 tag deferred -- waiting on explicit instruction after Matt reviews the campaign-wide summary.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
7f8633aed5 |
fix(avalanche): v0.5.7-avalanche -- Central avalanche check + categories audit
Seventh family of the v0.5.7 NATS-and-categories campaign. Smaller scope than prior families (consumer.py unchanged): the Central side is verifiably empty, and the registry-audit gap is a single-entry add.
FIX 1 -- Central avalanche adapter check: VERIFIED ABSENT in Central v0.10.0. Searched the consumer integration guide (docs/CONSUMER-INTEGRATION.md at v0.10.0-itd-511) -- zero `avalanche` / `NWAC` / `CAIC` references. Searched the producer source tree (src/central/adapters/) -- no avalanche-named adapter files. meshai already accounts for this:
- meshai/central/consumer.py _SUBJECTS_BARE has no `avalanche` key, so
_subjects_for("avalanche", *) returns [] regardless of region.
- CENTRAL_ADAPTER_TO_SOURCE has no avalanche entry on either side.
- _subject_owned() (consumer.py line 334-) explicitly logs a warning
if someone flips avalanche.feed_source=central, then skips
subscribing.
No code change needed for FIX 1. Tests now pin these invariants so a future refactor that introduces an unexpected avalanche Central wire breaks loudly here.
FIX 2 -- ALERT_CATEGORIES avalanche-family audit. Native meshai/env/avalanche.py emits two categories from the NWAC/CAIC danger-level tier:
danger_level >= 4 (High, Extreme) -> avalanche_warning
danger_level == 3 (Considerable) -> avalanche_watch
danger_level <= 2 (Low, Moderate) -> silently dropped (not actionable)
Pre-v0.5.7-avalanche registry had avalanche_warning + avalanche_considerable. avalanche_warning matched the native emit. avalanche_considerable was a LEGACY name for the Considerable-danger tier -- the native code already emits avalanche_watch for that same semantic (verified at meshai/env/avalanche.py:266; tests/test_adapter_avalanche.py:90 asserts the mapping).
So avalanche_watch was MISSING from the registry, leaving the rule editor unable to target danger-level=3 emissions even though they were correctly routed to toggle="avalanche" via the `("avalanche", "avalanche")` prefix fallback.
Added avalanche_watch under toggle="avalanche", default_severity="routine", with a description that points at the Considerable-tier semantics and an example_message matching the live NWAC product phrasing. composer._CATEGORY_EMOJI and _CATEGORY_LABEL gained matching entries so live LoRa rendering shows the right glyph (⛷, label "AVY").
Legacy entry kept: avalanche_considerable remains in the registry as a forward-compat target even though no current code path emits it. Reasoning matches the v0.5.7-rf precedent:
- router.py source-attribution tables (lines 317, 429) reference it
- composer.py emoji + label tables reference it
- A future phase might re-emit avalanche_considerable as a finer-grained
distinction from the generic Watch label; removing the registry entry
would break any user rule currently targeting it.
If avalanche_considerable remains un-emitted by v0.6, file a follow-up cleanup phase to remove it together with the rf-family hf_blackout / tropospheric_ducting legacy entries. test_alert_categories_avalanche_complete uses a SUBSET assertion (native emit ⊆ registry) so the legacy entry is allowed.
Audit table after v0.5.7-avalanche:
Registry avalanche (3):
avalanche_warning (native danger_level >= 4)
avalanche_watch [v0.5.7-avalanche NEW] (native danger_level == 3)
avalanche_considerable (legacy, no current emitter)
Native emit: {avalanche_warning, avalanche_watch} ⊆ Registry -- parity for everything emitted.
Tests
-----
PYTHONPATH=. pytest -q: 442 passed (was 431; +11 net).
- tests/test_avalanche_v057.py (new): _subjects_for("avalanche", *) returns [] for every region input; avalanche absent from _SUBJECTS_BARE and CENTRAL_ADAPTER_TO_SOURCE; flipping avalanche.feed_source=central produces zero subscriptions; avalanche_watch present under toggle="avalanche" with required fields; avalanche_warning + avalanche_considerable still registry-present; native emit set equals {avalanche_warning, avalanche_watch} and is a subset of the registry.
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
6c84baf12c |
fix(rf): v0.5.7-rf -- SWPC subject validation + protons severity=0 documentation + categories audit
Sixth family of the v0.5.7 NATS-and-categories campaign. RF family = native ducting calculator + three Central SWPC adapters (swpc_alerts, swpc_kindex, swpc_protons), all umbrella-subscribed under `central.space.>`.
Per the family-by-family pattern: cross-checked every prompt assumption against the Central v0.10.0 guide before implementing. The big surprise this phase: FIX 1 was already correct (no NATS-syntax bug to fix), and FIX 2 was a non-bug too (severity=0 already routes safely). The real work was FIX 3 -- four missing registry entries that meshai emits but the rule editor couldn't target.
FIX 1 -- SWPC subject pattern (already correct; pinned). Per Central v0.10.0 guide §swpc_alerts / §swpc_kindex / §swpc_protons, all three adapters publish under the `central.space.>` umbrella with no region in subject (space weather is planetary):
swpc_alerts: central.space.alert.<product_id> (4 tokens, product_id tail)
swpc_kindex: central.space.kindex (3 tokens, fixed)
swpc_protons: central.space.proton_flux (3 tokens, fixed)
`_subjects_for("swpc", region)` already returned `["central.space.>"]` ignoring region (v0.5.4 work got this right). Added an explanatory inline comment near the table entry calling out each adapter's concrete subject + the universal severity=0 contract (next fix), plus a test pinning the umbrella + region-ignored behavior + coverage of each per-adapter subject form. Future "let me add a region tail here" refactors will fail loudly.
FIX 2 -- swpc_protons severity=0 routing (non-bug; regression-guard pin). The prompt described a "severity=0 silently dropped" failure mode. Investigation: no such bug exists in current code.
- All three SWPC adapters publish severity=0 in the live guide samples.
- consumer.map_severity already maps 0 -> "routine" (the `if sev >= 3:`
immediate clamp doesn't hit; falls through to the default return).
- NotificationToggle.severity_channels is dict-keyed by severity STRING
(locked in by v0.5.7-seismic test_severity_channels_is_string_keyed_no_int_indexerror_risk);
"routine" is a valid key with no IndexError vector.
Three things tightened anyway: (a) inline comment near the swpc subject entry documenting "all three publish severity=0 -> routine per guide examples"; (b) end-to-end synthetic envelope test for swpc_protons injection (severity=0 in, ev.severity="routine" / ev.category="solar_radiation_storm" / ev.source="swpc" out, no exception); (c) parallel test for swpc_kindex confirming a second SWPC adapter wires identically.
FIX 3 -- ALERT_CATEGORIES rf_propagation audit. Pre-v0.5.7-rf registry had three entries under toggle="rf_propagation": hf_blackout, geomagnetic_storm, tropospheric_ducting. Audit:
Native ducting.py emits via _TIER_CATEGORY:
super_refraction -> rf_anomalous_propagation
duct -> rf_ducting_enhancement
surface_duct -> rf_ducting_enhancement
Central path via map_category:
space.alert.* -> rf_propagation_alert (swpc_alerts)
space.kindex -> geomagnetic_storm (swpc_kindex; already in registry)
space.proton_flux -> solar_radiation_storm (swpc_protons)
space.* catchall -> geomagnetic_storm
Four categories emitted but missing from the registry -- rule editor couldn't target them. Added all four under toggle="rf_propagation" with name + description + default_severity + example_message matching the guide-documented behavior:
rf_anomalous_propagation (routine, ducting super_refraction tier)
rf_ducting_enhancement (priority, ducting duct + surface_duct tiers)
rf_propagation_alert (priority, NOAA SWPC space-weather product)
solar_radiation_storm (priority, GOES proton flux S-scale)
composer.py emoji + label tables gained matching entries so live LoRa rendering shows the right glyphs (📡 for ducting forms, ⚠ for SWPC alerts, 🌐 for solar radiation, all labelled "RF").
Legacy entries kept (forward-compat / no current emitter): hf_blackout and tropospheric_ducting remain in the registry as selectable rule targets even though no current code path emits them. Reasoning:
- hf_blackout: HF-specific R-scale parsing of swpc_alerts.message could
re-introduce this emission in a future phase; removing the registry
entry would break any user rule currently configured to target it.
- tropospheric_ducting: legacy name superseded by rf_ducting_enhancement
in native ducting.py; same forward-compat concern -- a future phase
may emit a "tropospheric" specialization separate from generic ducts.
If either remains un-emitted by v0.6, file a follow-up cleanup phase to remove. Test_alert_categories_rf_complete uses a SUBSET assertion (emit set ⊆ registry) rather than equality so legacy entries are allowed.
Audit table after v0.5.7-rf:
Registry rf_propagation (7):
hf_blackout (legacy, no current emitter)
geomagnetic_storm (central swpc_kindex + catchall)
tropospheric_ducting (legacy, no current emitter)
rf_anomalous_propagation [v0.5.7-rf NEW] (native ducting super_refraction)
rf_ducting_enhancement [v0.5.7-rf NEW] (native ducting duct + surface_duct)
rf_propagation_alert [v0.5.7-rf NEW] (central swpc_alerts)
solar_radiation_storm [v0.5.7-rf NEW] (central swpc_protons)
Emit set ⊆ Registry: TRUE (no orphan emissions).
Tests
-----
PYTHONPATH=. pytest -q: 431 passed (was 413; +18 net).
- tests/test_rf_v057.py (new): umbrella subject is `central.space.>` for all regions; per-adapter published subjects all match; map_severity(0) -> "routine"; NotificationToggle.severity_channels dict-keyed (no IndexError); synthetic swpc_protons + swpc_kindex envelopes route cleanly with severity=0; four new rf_propagation entries all registry-present with required fields; geomagnetic_storm still mapped from space.kindex; map_category routing pinned for each SWPC adapter; native ducting + central SWPC emit sets are subsets of registry rf entries.
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
730b03099a |
fix(water): v0.5.7-water -- USGS NWIS hydro NATS pattern + categories audit
Fifth family of the v0.5.7 NATS-and-categories campaign. Water/hydro = USGS NWIS adapter (Central calls it `nwis`, meshai calls it `usgs` via CENTRAL_ADAPTER_TO_SOURCE remap). USGS quake stays in the seismic phase that already shipped; this phase fixes the hydro/water subscription and audits the water-side of the seismic toggle.
The Central v0.10.0 consumer integration guide was the starting point but the §nwis section text turned out to be STALE w.r.t. the regional subject suffix. Ground-truthed against the v0.10.0-itd-511 nwis.py producer subject_for() body and used the CODE as the source of truth. Documented below.
FIX 1 -- USGS NWIS hydro NATS pattern. Pre-v0.5.7-water `_subjects_for("usgs","us.id")` returned `["central.hydro.>.us.id", "central.hydro.>.unknown"]`. Both subjects are invalid NATS (`>` is only legal at the tail token).
The producer code at v0.10.0-itd-511 src/central/adapters/nwis.py:223 publishes:
central.hydro.<param>.<agency>.<site>.<region>
where <region> is either `us.<state>` (2 tokens) or `unknown` (1 token). So the live subjects on the broker are 7 tokens (per-state) or 6 tokens (unknown). The doc §nwis section text shows only the 4-token category stem `central.hydro.<parameter_code>.<agency>.<bare_site_no>` -- that text is stale; it predates the regional-routing roll-out.
Fixed by using three single-token `*` wildcards in the param/agency/site slots plus the bare region tail. Preserves the v0.5.4 INTENT (server-side regional filtering + .unknown workaround for gauges whose state Central can't resolve) while restoring NATS syntax legality:
central.hydro.*.*.*.us.id (7 tokens, per-state)
central.hydro.*.*.*.unknown (6 tokens, .unknown workaround)
Bare-form fallback (`central.hydro.>`) is unchanged for empty/None region (pre-v0.5.3 backward compat path).
FIX 2 -- ALERT_CATEGORIES water/hydro audit. Pre-v0.5.7-water registry had `stream_flood_warning` and `stream_high_water` (both toggle="seismic" from the v0.5.2 USGS-water -> Geohazards migration). Audit findings:
- Native usgs.py applies NWPS flood-stage thresholds client-side and emits
`stream_flood_warning` (reading at/above flood stage) or `stream_high_water`
(Action Stage reading). Routine gauge readings below action stage are
silently dropped on the native path (no spam).
- Central path: every NWIS reading arrives with category=`hydro.<pcode>.
<agency>.<site>` at severity=0. consumer._CATEGORY_MAP maps `hydro.*`
to `stream_flow` (added in earlier work). But `stream_flow` was MISSING
from ALERT_CATEGORIES -- routing worked via the `("stream", "seismic")`
prefix fallback, but the Advanced Rules editor couldn't target raw
central-delivered gauge readings.
Added `stream_flow` to ALERT_CATEGORIES under toggle="seismic", default_severity="routine", with an example_message that reflects the raw-reading shape. The existing `stream_flood_warning` / `stream_high_water` entries are unchanged.
NOTE on parity gap (deferred to v0.5.8+): meshai does NOT currently re-apply NWPS threshold logic to central-delivered NWIS readings. So flipping `usgs.feed_source=central` today produces a stream of routine `stream_flow` events without the flood-stage classification the native path provides. Bringing the central path to parity (apply threshold logic AFTER receiving central-delivered raw readings) is queued as future work -- intentionally out of scope here per Matt's one-fix-per-family rule.
Audit table after v0.5.7-water:
Native emit: stream_flood_warning, stream_high_water (threshold-triggered)
Central path: every hydro.* -> stream_flow (routine; no threshold)
Registry: {stream_flow, stream_flood_warning, stream_high_water} (toggle=seismic)
Quake side: earthquake_event (toggle=seismic, added v0.5.7-seismic) -- unchanged
Parity confirmed. No orphans, no missing.
Tests
-----
PYTHONPATH=. pytest -q: 413 passed (was 400; +13 net).
- tests/test_water_v057.py (new): usgs subscription is NATS-legal (no `>` anywhere, all single-token `*`); token-count matches the producer-published shape (7-token us-state + 6-token .unknown); per-region substitution (Montana sanity); bare-form backward compat; `stream_flow` present under toggle="seismic"; `stream_flood_warning` / `stream_high_water` unchanged; native + central emit set matches registry water-side subset; threshold categories still emitted by usgs.py; all realistic central pcodes fold to `stream_flow`; required-fields check; severity=0 -> "routine" sanity.
- tests/test_central_region_routing.py: updated `test_subjects_for_usgs_includes_unknown_workaround` to reflect the v0.5.7-water fix (single-token `*` wildcards instead of mid-subject `>`).
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
8359451eaa |
fix(seismic): v0.5.7-seismic -- USGS quake NATS pattern + severity=5 great-quake clamp + categories audit
Fourth family of the v0.5.7 NATS-and-categories campaign. Seismic = USGS quake adapter (the USGS water/hydro side stays under toggle="seismic" per the v0.5.2 geohazards migration but lives in the water phase that follows).
The Central v0.10.0 consumer integration guide (docs/CONSUMER-INTEGRATION.md on the v0.10.0-itd-511 branch) was treated as source of truth -- ground-truthing the prompt against the guide caught two prompt errors before they shipped (mirrors the FIRMS situation in v0.5.7-fire). Documented below.
FIX 1 -- USGS quake NATS pattern. Pre-v0.5.7-seismic `_subjects_for("usgs_quake","us.id")` returned `["central.quake.event.>.us.id"]`. That subject is BOTH invalid NATS (`>` is only legal at the tail token) AND wouldn't have matched anything Central publishes.
Per Central v0.10.0 guide §usgs_quake the actual published subject is `central.quake.event.<tier>` -- 4 tokens, no region. `<tier>` is one of {minor, light, moderate, strong, major, great} (USGS magnitude bands; bands live in the SUBJECT, not in the severity integer).
Note on prompt vs. guide discrepancy: the v0.5.7-seismic prompt described a "regional v0.9.20+ shape" `central.quake.event.<severity>.us.<state>` with 6 tokens and `us.<state>` at the tail. That's neither what Central v0.10.0 publishes nor what its guide documents. We follow the guide. Subscribing to the prompt's shape would silently match zero messages in production. State filtering for quakes happens client-side via data.latitude/longitude (same situation as FIRMS).
New subscription: `central.quake.event.>` -- tail-only `>`, NATS-legal, matches all <tier> values.
FIX 2 -- severity=5 great-quake clamp (no actual bug; regression-guard pin). The prompt described a "severity=5 IndexError or silent drop" failure mode. Investigation found NO such bug exists in the current code:
- consumer.map_severity already clamps `sev >= 3` to "immediate". A
severity=5 (or 99, or any 3+) maps safely to "immediate" with no
exception path.
- NotificationToggle.severity_channels is dict-keyed by severity STRING
({"routine","priority","immediate"}), not an int-indexed list, so
IndexError is structurally impossible from this boundary regardless
of upstream value.
- Per Central v0.10.0 guide §5b the documented severity vocabulary is
`0-4 or None`. Severity=5 is not in the published contract; the
clamp is defensive padding against future contract drift.
Three things were tightened anyway: (a) the map_severity docstring now explicitly documents the high-side clamp behavior and calls out the string-keyed dict guarantee; (b) parametrized test pins map_severity for the full 0..99 range including out-of-contract values; (c) an end-to-end synthetic-envelope test injects a severity=5 quake through _handle and asserts the resulting Event has severity="immediate" / category="earthquake_event" / source="usgs_quake" with no exception. These tests function as regression guards if a future refactor introduces the IndexError vector the prompt was guarding against.
FIX 3 -- ALERT_CATEGORIES seismic-family audit. The registry was MISSING `earthquake_event` entirely. Both native (`usgs_quake.py` emits `category="earthquake_event"`) and central (consumer._CATEGORY_MAP maps `quake.* -> earthquake_event`) paths produce that category, but get_category("earthquake_event") fell through to the mesh_health default -- so the Advanced Rules editor couldn't target quakes at all. The get_toggle() prefix fallback DID route it to "seismic" via the `("earthquake", "seismic")` rule, so events were filtered correctly; the gap was UI-selectability only.
Added the entry under toggle="seismic" with a representative example_message. composer.py already had matching emoji/label mappings (line 78-79, 107-108) from earlier work, no composer change needed.
The two hydro entries (`stream_flood_warning`, `stream_high_water`) also live under toggle="seismic" via the v0.5.2 USGS-water migration (Geohazards family in the GUI). They are OUT OF SCOPE for v0.5.7-seismic -- they belong to the water phase that follows. Verified-unchanged here so the next phase has a clean baseline.
Audit table after v0.5.7-seismic:
Native emit: usgs_quake.py -> earthquake_event
Central path: all 6 tiers (minor/light/moderate/strong/major/great) -> earthquake_event
Registry: {earthquake_event, stream_flood_warning, stream_high_water}
Quake side: parity (registry has earthquake_event; native + central emit it)
Hydro side: verified-unchanged (deferred to water phase)
Tests
-----
PYTHONPATH=. pytest -q: 400 passed (was 380; +20 net).
- tests/test_seismic_v057.py (new): quake subject tail-only `>`; no mid-subject `>`; bare-form backward compat; parametrized map_severity full range 0..99 + None / nonsense / negative; synthetic severity=5 envelope routes through _handle to severity="immediate" cleanly; NotificationToggle.severity_channels shape pinned to dict (no IndexError vector); earthquake_event present under toggle="seismic"; hydro entries still toggle="seismic" (regression guard); native + central-path quake emit set equals {earthquake_event}; required-fields check.
- tests/test_central_region_routing.py: updated `test_subjects_for_usgs_quake_us_id` -> `test_subjects_for_usgs_quake_us_id_uses_tail_only_wildcard` reflecting the guide-correct shape.
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
60e8e62e85 |
fix(fire): v0.5.7-fire -- FIRMS NATS pattern + WFIGS tombstone dedup + remove fire_proximity + categories audit
Third family of the v0.5.7 NATS-and-categories campaign. Fire is the heaviest of the campaign -- four distinct fixes plus a category audit. Two of the four were broken in production: FIRMS subscribed to a syntactically invalid pattern, and WFIGS tombstones were silently dropped.
FIX 1 -- FIRMS NATS pattern (the canonical bug). Pre-v0.5.7-fire `_subjects_for("firms","us.id")` returned `["central.fire.hotspot.>.us.id"]`, which is INVALID NATS (the `>` multi-level wildcard is only legal at the tail token). It also wouldn't have matched anything Central publishes: per the Central v0.10.0 consumer integration guide §firms, the actual published pattern is `central.fire.hotspot.<satellite>.<confidence>` (5 tokens, no us.<state> suffix). The two slots after "hotspot" are satellite name and confidence band -- NOT tile coordinates or region tokens.
Note on prompt vs. guide discrepancy: the v0.5.7-fire task spec described a tile-coord/state pattern `central.fire.hotspot.*.*.us.id` (7 tokens with us.<state> tail). That's neither what Central v0.10.0 publishes nor what its guide documents. We follow the guide. Subscribing to the prompt's 7-token pattern would silently match zero messages in production (token-count mismatch). State filtering for FIRMS happens client-side via data.latitude / data.longitude against the configured region bbox.
New subscription: `central.fire.hotspot.>` -- tail-only `>`, NATS-legal, matches all <satellite>.<confidence> combinations.
FIX 2 -- WFIGS tombstone subjects. Per guide §wfigs_incidents and §wfigs_perimeters, WFIGS publishes:
active: central.fire.incident.<state>.<county> (Convention A, depth-3 state)
active: central.fire.perimeter.<state>.<county>
tombstone: central.fire.incident.removed.<state> (5 tokens, "removed" at depth-3)
tombstone: central.fire.perimeter.removed.<state>
Pre-v0.5.7-fire `_subjects_for("fires","us.id")` subscribed only to the active subjects (`central.fire.incident.id.>` and `central.fire.perimeter.id.>`). The tombstone subjects have "removed" at depth-3 instead of the state token, so the active-subject `>` filters silently dropped EVERY tombstone. Fall-off signals never reached meshai's inhibitor, so old incidents stayed "live" in the pipeline indefinitely.
Added the two tombstone subjects to the subscription list. Both are 5-token literals with no wildcards -- trivially NATS-legal.
FIX 3 -- WFIGS tombstone dedup. Per guide §wfigs_incidents removal semantics, the tombstone env_id has the shape `<IrwinID>:removed:<iso_now>` -- the `:removed:` is sandwiched in the middle, with a timestamp tail. Pre-v0.5.7-fire the consumer.py group_key recovery was `re.sub(r":removed$", "", group_key)` -- a literal trailing `:removed` match -- which DID NOT FIRE on the WFIGS form (the regex required `:removed` at the very end of the string, but the WFIGS form has `:<iso>` after it).
Consequence: WFIGS tombstones' group_key was the full `<IrwinID>:removed:<iso>` string instead of the bare `<IrwinID>`. The pipeline grouper/inhibitor never matched tombstones to their original incidents, so the lapse signal was lost.
Fixed by switching the regex to `re.sub(r":removed(:.*)?$", "", group_key)` -- handles both the WFIGS `<IrwinID>:removed:<iso>` form AND the legacy GDACS `<id>:removed` form. The `is_tombstone` detection also gained an explicit `":removed:" in env_id` check for the WFIGS shape.
Per the guide: "the same incident can have one or more removal tombstones over its lifecycle" (it can re-enter and re-fall-off). To preserve per-tombstone distinctness for downstream lifecycle accounting, the full env_id is stashed on `Event.data["_central_tombstone_id"]` (the group_key collapses to the IrwinID by design, but the original env_id with the :<iso> tail survives on data).
FIX 4 -- ALERT_CATEGORIES fire-family audit + removed parametric entries. Per Matt's direct feedback ("fire near mesh has its own set of parameters that I don't even know what they could be. like how far is near mesh? I don't know I can't set that."), the parametric `fire_proximity` and the duplicate-named `wildfire_proximity` (both labeled "Fire Near Mesh" with parametric radius-based descriptions) were unselectable in the new Advanced Rules UI. Removed both.
Cross-referenced what FIRMS and WFIGS actually emit (per the guide and the native adapter code) and audited the registry:
Native emit:
firms.py -> new_ignition (when adapter flags new_ignition)
or wildfire_hotspot (otherwise) [v0.5.7-fire: was wildfire_proximity]
fires.py -> wildfire_incident
Central path emit (via map_category):
fire.hotspot.* -> wildfire_hotspot
fire.incident.* -> wildfire_incident
fire.perimeter.* -> wildfire_incident (perimeters merge to the incident)
fire.<other> -> wildfire_incident (catchall)
Registry after v0.5.7-fire:
{new_ignition, wildfire_hotspot, wildfire_incident}
Parity confirmed. No orphans, no missing.
Aligning firms.py to emit `wildfire_hotspot` (matching the central FIRMS map) means native + central FIRMS produce identical categories regardless of which feed path is enabled.
Composer (`_CATEGORY_EMOJI`, `_CATEGORY_LABEL`) and router (three source-attribution tables) updated to drop the removed categories and add the new ones.
Deferred to v0.5.8: distance_max_km field on rules for actual proximity filtering. Replaces the parametric fire_proximity registry entry with a parameterized rule field that the user CAN configure ("alert me about wildfire_incident within 30 km" instead of an opaque "Fire Near Mesh" toggle).
Tests
-----
PYTHONPATH=. pytest -q: 380 passed (was 366; +14 net).
- tests/test_fire_v057.py (new): FIRMS subject is tail-only `>` with no mid-subject placement; WFIGS subjects cover active + four tombstones; WFIGS tombstone strips `:removed(:.*)?$` for group_key; two same-IrwinID tombstones both propagate through _handle and share group_key, with the original env_id preserved on data["_central_tombstone_id"]; legacy GDACS `:removed` shape still strips cleanly; fire_proximity / wildfire_proximity absent from ALERT_CATEGORIES; no "Fire Near Mesh" name duplicates; fire-family parity (native + central emit == registry); required-fields check on the three fire entries.
- tests/test_central_region_routing.py: updated FIRMS test (tail-only `>`) and WFIGS test (includes tombstone subjects).
- tests/test_pipeline_toggle_filter.py, tests/test_adapter_firms.py, tests/test_v052_dispatcher.py, tests/test_pipeline_digest.py: bulk-migrated obsolete category references (wildfire_proximity -> wildfire_hotspot, fire_proximity -> wildfire_incident) so the existing test suites continue to exercise the same routing/digest/dispatch paths with the new category names.
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
14d168822b |
fix(traffic): v0.5.7-traffic -- NATS pattern fix + itd_511 sub-adapter routing + categories audit
Second family of the v0.5.7 NATS-and-categories campaign. Weather went first because its NWS pattern was already legal; traffic was carrying invalid NATS syntax in production.
FIX 1 -- Invalid `>` mid-subject in traffic. Pre-v0.5.7-traffic the subject builder shipped `central.traffic.>.{state}` for both the traffic and roads511 adapters. NATS rules say `>` is only legal at the tail token; mid-subject `>` is rejected by the broker at subscribe time (or silently delivers nothing depending on server version). Replaced with Convention B (per Central v0.10.0 meshai_integration_guide.md): single-token `*` in the event_type slot, bare state suffix -- `central.traffic.*.id` for Idaho. Shared by the wzdx, tomtom_incidents and state_511_atis adapters.
FIX 2 -- roads511 dual subscribe. The new Idaho-only itd_511 adapter in Central v0.10.0 uses Convention A (`central.traffic.<event_type>.us.<state>`, the us.<state> form). Convention B (bare state) is shared with the rest of the traffic family. roads511 now owns BOTH:
central.traffic.*.id (Convention B, shared with traffic via _subject_owned)
central.traffic.*.us.id (Convention A, itd_511-only)
Sub-adapter routing in CentralConsumer._subject_owned (v0.5.1) already keeps shared subjects scoped to the right meshai source -- no change needed.
FIX 3 -- itd_511 -> roads511 in CENTRAL_ADAPTER_TO_SOURCE. Mirrors state_511_atis (added v0.5.3). Both Idaho 511 feeds collapse to a single meshai source for UX simplicity; future v0.6 may split them if Matt needs differential rules.
FIX 4 -- Roads-family categories audit + finer event_type mapping. Pre-v0.5.7-traffic the central path flattened every traffic-domain event to `traffic_congestion` because work_zone / incident / closure had no entries in _CATEGORY_MAP and fell through to the `traffic.` catchall (then the subject-domain fallback). Added three explicit map entries before the catchall:
("work_zone", "work_zone") # catches "work_zone" and "work_zone.wzdx"
("incident", "road_incident") # catches incident.tomtom_incidents + bare
("closure", "road_closure") # catches closure + closure.itd_511
ALERT_CATEGORIES gains two new roads-family entries so the Advanced Rules editor can target them:
work_zone -- Active construction/maintenance work zone
road_incident -- Reported incident (crash, hazard, debris)
Existing entries `road_closure` and `traffic_congestion` kept. composer._CATEGORY_EMOJI gains matching glyphs (🚧 work_zone, 🚨 road_incident) so the live LoRa rendering lines up with the category example_message glyphs.
Audit cross-check (test_alert_categories_roads_complete enforces parity):
Native emit: traffic.py -> traffic_congestion; roads511.py -> road_closure
Central path emit (via map_category): {road_closure, traffic_congestion, work_zone, road_incident}
ALERT_CATEGORIES{toggle=roads}: {road_closure, traffic_congestion, work_zone, road_incident}
Parity. No orphans, no missing.
DEFERRED to v0.5.8: itd_511_cameras / traffic_cameras stream lives at a different subject domain (central.traffic_cameras.>) and needs a new meshai source (roads_cameras or similar). Out of scope for v0.5.7.
Tests
-----
PYTHONPATH=. pytest -q: 366 passed (was 345; +21 net).
- tests/test_traffic_v057.py (new): NATS-syntax checks (`>` only at tail, single-token `*`), traffic Convention B, roads511 dual-subscribe, shared bare-state subject, itd_511 + state_511_atis remap, map_category event_type preservation, ALERT_CATEGORIES roads parity (reflection-based scan of native emit + central path), required-fields check on the four roads entries.
- tests/test_central_region_routing.py: updated `test_subjects_for_traffic_and_roads511_share_state_token` -> two new tests covering Convention B (traffic) and dual-subscribe (roads511).
- tests/test_central_consumer.py: updated `test_subject_domain_fallback_for_unmapped_category` (work_zone.wzdx is now mapped, switched to a genuinely-unmapped category) + new `test_v057_traffic_work_zone_now_mapped` asserting wzdx envelopes land on ev.category=="work_zone".
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
b87696bf67 |
fix(weather): v0.5.7-weather -- NWS HTML strip + ALERT_CATEGORIES audit (NATS pattern already valid)
First family of the v0.5.7 NATS-and-categories campaign (Matt review of Central v0.10.0 meshai_integration_guide.md). Weather lands first because the NWS NATS pattern is already legal; the other five families need invalid mid-subject > rewrites that will ship per-family.
FIX 1 -- NWS NATS pattern validated. _subjects_for("nws", "us.id") -> ["central.wx.alert.us.id.>"]. The wildcard token > sits at the tail only (token index -1), so the subject is a legal NATS multi-level wildcard. No code change. Live introspection confirmed in-container.
FIX 2 -- NWS HTML strip in mesh composer. Per Central guide Surprise 3, data["description"] and data["instruction"] arrive as raw HTML (<p>, <br>, <strong>, , —, ...). Until now the composer fed event.title / event.summary straight to LoRa, so any future title/summary populated from those fields would have leaked literal markup onto the wire.
Added strip_html_tags(text) -> str in meshai/notifications/renderers/composer.py. Block-level tags (br, p, div, li, tr, h1-h6) become a single space so adjacent paragraphs do not fuse; all other tags are removed; HTML entities are decoded via html.unescape; whitespace is collapsed. Applied in _primary_identifier (title and summary paths) and _region_segment BEFORE byte-budget truncation, so the 150 B cap counts real glyphs, not markup. Universal (not NWS-gated) since strip is a no-op on plain text -- protects against future adapters that surface raw HTML too.
FIX 3 -- ALERT_CATEGORIES weather audit. Cross-referenced ALERT_CATEGORIES{toggle="weather"} against meshai/env/nws.py:_derive_category() emission set:
nws.py emits: weather_warning, weather_watch, weather_advisory, weather_statement
registry weather: weather_warning, weather_watch, weather_advisory, weather_statement
Parity. No additions, no removals. The v0.5.2 stream_* migration to the seismic family (USGS hydro under the GUI Geohazards tab) is already reflected; weather is clean at 4 entries. Added a comment block above the weather section pointing at test_alert_categories_weather_complete which now enforces this parity going forward -- if a new branch is added to _derive_category(), the test fails and forces a matching registry entry.
Tests
-----
PYTHONPATH=. pytest -q: 345 passed (was 328; +17 new in tests/test_weather_v057.py).
- strip_html_tags: simple tags, br/paragraph -> space, entity decode (& —), nested/attrs, plain-text no-op, empty input, whitespace collapse.
- compose_mesh_message integration: HTML in title scrubbed; HTML in summary fallback scrubbed; 150 B budget still holds.
- Weather parity: reflection-based scan of NWSAlertsAdapter._derive_category() vs registry; both must match.
- Required-fields check on the four weather entries.
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
matt+claude
|
c211d34060 |
chore(meshai): v0.5.5 -- cleanup bundle (gitignore env anchor, ducting health event_count, mesh_sources secret stripping, delete unused SeverityRouter)
Four independent low-risk fixes from the deferred list. Bundled in a single
commit because none are large enough to warrant their own tag and none
touch the safe-mode-sensitive paths (dispatcher / consumer / toggle config).
1) .gitignore: change bare `env/` to `/env/` so the rule anchors at the
repo root only. The unanchored form was matching `meshai/env/` (the
adapter package directory) and forced `git add -f` workarounds during
2.14 / 2.16.1. Verified post-fix: `git check-ignore -vn meshai/env/test.py`
reports no pattern match; `git check-ignore -v env/foo` still matches
the new `/env/` rule.
2) meshai/env/ducting.py: health_status.event_count was hardcoded `0`
from before Phase 2.13 added real event emission. Replaced with
`len(self._events)`, which is the pattern every other env adapter
already uses (fires/firms/nws/swpc/traffic/roads511/usgs/usgs_quake/
avalanche). Flows through env.store.health_status → /api/env/status
so the dashboard counter starts reflecting reality.
3) meshai/config_loader.py save_section: list-section secret stripping.
The path landed in C.2.1 fed list items into check_secrets() with
path="" or with `<field>[<i>]` syntax, neither of which matched the
`mesh_sources.*.api_token` / `notifications.rules.*.smtp_password`
regexes in SECRET_FIELDS (where `*` matches a single dotted token).
Result: a raw secret submitted on a list-section save could slip
through to the YAML file. Fix uses dotted-index form `<field>.<i>.<key>`
for both nested-list (notifications.rules) and top-level-list
(mesh_sources) paths. Also extended _raw_section construction +
_ondisk_ref to walk list-shaped on-disk YAML by integer index so
the C.3.1 ${VAR}-placeholder preservation now works for list sections
too. Three new tests round-trip the mesh_sources placeholder case,
the mesh_sources raw-secret rejection, and the nested-list
notifications.rules placeholder case.
4) meshai/notifications/pipeline/severity_router.py: deleted.
The fork-by-severity routing it implemented was never wired in
production -- _tee in build_pipeline does the dispatcher+digest
fanout directly. The class had two test references in
tests/test_pipeline_skeleton.py that exercised "no matching rule"
and "unknown severity" paths; those guarantees are now covered by
tests/test_v052_dispatcher.py (stats counters) and the existing
Dispatcher-class tests. Removed the file, the __init__.py imports
and __all__ entries (SeverityRouter + StubDigestQueue both), the
two test methods, and the docstring mention.
Verification:
- py_compile clean on all four touched modules.
- `grep -rn SeverityRouter meshai/ tests/` returns zero.
- pytest 328 passed (was 327 at v0.5.4; net: -2 SeverityRouter tests,
+3 secret-preservation tests = +1).
- .gitignore anchor diagnosed via `git check-ignore -vn`.
Safe-mode preserved -- no toggle enabled, no master enabled, no central
enabled, no adapter feed_source flipped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
matt+claude
|
c2d5bcfbd1 |
feat(central): v0.5.4 -- region-aware subscriptions using Central v0.9.20 regional subjects
Pre-v0.5.4 every Central subscription used a bare wildcard (central.wx.>,
central.fire.>, central.traffic.>, central.quake.>, central.hydro.>,
central.space.>), so a Magic Valley operator flipping nws -> central was
in fact subscribing to the all-US firehose and discarding 95% of events
locally. Central v0.9.20 (2026-05-28) added per-region subject suffixes
so the firehose can be filtered server-side. This wires meshai to use them.
Backend (meshai/central/consumer.py):
- New _subjects_for(adapter, region) replaces the static ADAPTER_SUBJECTS
dict. ADAPTER_SUBJECTS is retained as an alias to _SUBJECTS_BARE for any
legacy importers; the dispatcher path is unchanged.
- Per-adapter subject patterns (region='us.id' default):
nws -> central.wx.alert.us.id.> (region BEFORE wildcard)
usgs_quake -> central.quake.event.>.us.id (region AFTER wildcard)
firms -> central.fire.hotspot.>.us.id
fires -> central.fire.incident.id.> (state token at fixed depth)
central.fire.perimeter.id.>
traffic -> central.traffic.>.id (bare state, no us. prefix)
roads511 -> central.traffic.>.id (shared with traffic, sub-adapter routing)
usgs -> central.hydro.>.us.id
central.hydro.>.unknown (workaround until v0.9.20.1)
swpc -> central.space.> (planetary; region ignored)
- Empty/None region falls back to bare wildcards (pre-v0.9.20 behaviour).
- _subject_owned() pulls region from env.central.region and routes through
_subjects_for; v0.5.3 sub-adapter routing (owned-sources set) still
applies on shared subjects like central.traffic.>.id.
- start() logs the active region at connect-time for ops visibility.
Config (meshai/config.py):
- CentralConsumerConfig.region: str = "us.id". One region per consumer
applies to every central-flipped adapter; per-adapter overrides can
land in v0.6 when there is a real use case.
Frontend (dashboard-frontend/src/pages/Environment.tsx):
- Central Connection panel gets a Region text input next to URL/Durable.
- EnvConfig.central type extended with region: string.
- Static bundle rebuilt; index-DCFmSeOM.js -> index-B24tHcYj.js.
Tests:
- tests/test_central_region_routing.py (new, 9 cases): asserts the exact
v0.9.20 subject string for each adapter at region='us.id', the SWPC
global-stays-global rule, the USGS .unknown workaround, the empty-region
backward-compat fallback for all 8 adapters, and integration through
CentralConsumer._subject_owned() with the default region.
- tests/test_central_consumer.py + tests/test_central_sub_adapter_routing.py:
the two tests that asserted bare-wildcard subjects now set
env.central.region = "" explicitly to preserve their original concern
(no region semantics — backward-compat path only).
Why swpc stays global: space weather is planetary -- a CME is detected on
the sun, the geomagnetic response is hemispheric. There is no Idaho-only
solar event; subscribing per-region would only drop events we want.
Why hydro has the .unknown workaround: Central v0.9.20 leaves gauges
whose USGS state can't be inferred on central.hydro.>.unknown. Until
v0.9.20.1 backfills the state tag we subscribe to both filters to
avoid silently losing those rows. Idaho downstream-filtering on
data['_enriched']['usgs_site']['state'] is future v0.6 work.
Orthogonal to v0.5.2 dispatcher guards (staleness / cooldown / dedup)
and v0.5.3 sub-adapter routing: the region filter operates at the NATS
subscription layer (server-side), upstream of everything else.
Verified: pytest 327 passed (318 prior + 9 new region-routing tests);
py_compile clean; frontend build clean. Safe-mode preserved -- no toggle
enabled, no master enabled, no central enabled.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
matt+claude
|
ded2156024 |
feat(central): v0.5.3 -- roads511 + FIRMS central feeds with sub-adapter routing (recover stashed v0.5.1)
C.2 (v0.4) marked roads511 and FIRMS native-only in the dashboard despite the
Central event bus shipping both stream families. This recovers the v0.5.1
work paused before the v0.5.2 spam fix and lands it cleanly on top of v0.5.2.
Backend (meshai/central/consumer.py):
- ADAPTER_SUBJECTS: roads511 now subscribes to central.traffic.> (shared
with the existing traffic adapter); firms widened from
central.fire.hotspot.> to central.fire.>.
- CENTRAL_ADAPTER_TO_SOURCE: three new sub-adapter remaps so the inner
envelope.adapter routes correctly inside a shared subject --
tomtom_incidents -> traffic, state_511_atis -> roads511, firms -> firms.
- New _subject_owned() -> dict[subject_filter, set[meshai_source]]:
builds the per-subject ownership set so a single central.traffic.>
subscription can be owned by {traffic, roads511} simultaneously.
- subjects() now derives from _subject_owned().keys().
- _make_cb(owned) binds the owned set per-subscription; _on_message
forwards it.
- _handle(subject, raw, owned=None) drops events whose remapped source
isn't in the owned set (silent debug log). Enabling roads511 alone
no longer accidentally consumes wzdx; enabling traffic alone no
longer consumes state_511_atis.
- start() subscribes per (subject, owned) tuple; per-subject log line
records the owned sources at startup.
- Removed roads511 from the "no Central mapping" warning loop now that
it has one.
Frontend (dashboard-frontend/src/pages/Environment.tsx):
- roads511 META entry: hasCentral false -> true, nativeOnly true -> false
(the FIRMS entry was already correct).
- Static bundle rebuilt via npm run build; old index-CfYlhn4e.js dropped,
new index-DCFmSeOM.js + index-DjhQa8Mv.css land under static/assets;
index.html updated to the new bundle hash.
Tests (tests/test_central_sub_adapter_routing.py, 8 new):
- roads511-only drops wzdx; emits state_511_atis as source=roads511.
- traffic+roads511 both central: wzdx -> traffic, state_511_atis -> roads511.
- firms-only drops wfigs_incidents; emits hotspots as source=firms.
- tomtom_incidents remaps to traffic.
- _subject_owned() shares central.traffic.> across {traffic, roads511}.
Orthogonal to v0.5.2's dispatcher guards: cooldown / dedup / staleness still
apply downstream of the consumer; the owned-sources filter operates one
layer up at message ingest. No changes to the dispatcher path.
Verified: pytest 318 passed (310 prior + 8 new routing tests); py_compile clean.
Safe-mode preserved -- no toggle enabled, no master enabled, no central enabled.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
matt+claude
|
ad6e24d123 |
fix(notifications): v0.5.2 -- staleness filter, cooldown, dedup, renderer wiring, hydro family
Spam fix from v0.5.0 oversight: - Staleness filter (default 600s, configurable per-toggle) drops backlog at dispatcher entrance -- solves the "restart wave fires days of old events" problem definitively. - Per-toggle cooldown_seconds (default 300s) throttles same (category, region) bursts. - Per-(source, event_id) LRU dedup (10k entries) catches Central re-delivery. - Renderer wired into _dispatch_toggles; toggle path now produces friendly mesh strings with 150-byte UTF-8 hard cap and priority-order segment composition (no mid-char trunc). - categories.py: stream_flood_warning / stream_high_water moved from weather -> geohazards family (canonical toggle name = seismic in VALID_TOGGLES) to match the GUI family tab. Verified end-to-end: 7200s-old events all dropped (100/0), fresh burst throttles to one mesh broadcast per cooldown window (1/99), dedup catches duplicate event_ids (1/99). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
| b90afc3a74 |
feat(notifications): v0.5.0 -- Master Toggles UX redesign + Central Connection GUI + grouped categories + region scoping
Per-family notification policy (PagerDuty/Grafana-style): each family gets a
severity threshold + region scope + a severity->channel routing matrix, so an
operator opts in per family rather than hand-writing rules.
SECTION 1 -- BACKEND
- config.py: new NotificationToggle dataclass (enabled, min_severity, regions,
severity_channels{severity->[channel types]}, quiet_hours_override, + per-channel
delivery config: broadcast_channel/node_ids/smtp_*/recipients/webhook_*).
notifications.toggles is now a dict[family]->NotificationToggle with 8 family
defaults (mesh_health, weather, fire, rf_propagation, roads, avalanche, seismic,
tracking), all enabled=false (opt-in), min_severity=priority,
severity_channels={priority:[mesh_broadcast], immediate:[mesh_broadcast, mesh_dm]},
quiet_hours_override=true. (Old TogglesConfig.enabled was only read by
build_pipeline via getattr -> degrades to ToggleFilter no-op, so the pipeline
filter is unchanged; toggles now drive the Dispatcher instead.)
- region_scope:list added to NotificationRuleConfig; _matching_rules filters by
event.region/regions ([] = all).
- Dispatcher: _dispatch_toggles runs IN PARALLEL to rule matching -- looks up
get_toggle(event.category), checks enabled + region scope + severity threshold,
then for each channel in severity_channels[event.severity] builds a synthetic
rule (override_quiet set only for immediate when quiet_hours_override) and
delivers. 'digest' channel is skipped in live dispatch (handled by accumulator).
- categories.py: get_toggle() prefix fallback maps the live phases-2.7-2.14
categories (weather_warning, wildfire_incident, earthquake_event,
traffic_congestion, geomagnetic/rf_*, stream_*, ...) to their family, fixing the
v0.4 "category -> other" gap.
- config_loader.py: SECRET_FIELDS += notifications.toggles.*.smtp_password.
- _dataclass_to_dict now recurses dict-of-dataclasses, and the loader coerces the
toggles dict -> NotificationToggle on both the full-load and section-PUT paths
(so GUI save round-trips correctly).
- tests/test_notification_toggles.py (11): enabled/disabled, region filter
(empty+populated+regions-list), severity threshold, per-severity channel routing,
digest-skipped-live, quiet-hours-override immediate-only, category->family,
rules+toggles both fire. Full suite: 294 passed (283 + 11).
SECTION 2 -- FRONTEND
- Notifications.tsx: MasterToggles component above the rules section -- 8 family
cards (icon + enable toggle; collapsed summary 'OFF' or 'N regions, M channels at
<sev>+'; expanded: severity threshold, severity x channel checkbox matrix,
region list, quiet-hours-override toggle, per-channel config:
broadcast_channel/DM node IDs/recipients/SMTP host+port/webhook URL).
- Environment.tsx: CentralConnectionPanel above the family tabs (url, durable,
enabled) wired to environmental.central.
- npm run build clean (tsc strict); rebuilt static committed (index-CfYlhn4e.js).
SECTION 3 -- VERIFICATION
- py_compile + tsc strict clean; pytest 294 passed.
- Rebuilt prod: /notifications serves Master Toggles, /environment serves Central
Connection (strings confirmed in the served bundle); 8 adapters, pipeline
started, no tracebacks, healthy.
- GUI round-trip: enable weather toggle (min_severity=priority,
regions=[Magic Valley], severity_channels.priority=[mesh_broadcast]) -> PUT
{saved:true} -> notifications.yaml reflects it; env_feeds traffic.api_key stayed
${TOMTOM_API_KEY} (C.3.1 secret preservation holds). Restored to clean opt-in
baseline.
- Synthetic NWS weather_warning/priority/Magic Valley -> routes through the weather
toggle to mesh_broadcast; out-of-region and below-threshold events correctly
dropped.
DEFERRED (noted for a follow-up, not blocking Matt's morning config): Section 2B
rules-editor polish -- grouped-by-family category checkboxes, region_scope
multi-select in the rule editor (backend field + filtering ARE in), tooltips, and
the fire-count Active/No-activity badge -- were not built tonight to keep the build
shippable and verified; the Advanced Rules section is otherwise unchanged and
still functional.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 11e37c4f48 |
fix(central): v0.4 D.2 -- remap Central adapter names to meshai source for consistent dashboard attribution
Phase D catalogued a source-name divergence: central-sourced events carried Central's adapter name (wfigs_incidents, nwis, swpc_alerts, wzdx) rather than meshai's native source (fires, usgs, swpc, traffic), so the C.2 family-tab per-adapter event filtering (which keys on the native source name) wouldn't group central events under the right adapter. Fix: CENTRAL_ADAPTER_TO_SOURCE table in consumer.py; normalize() now remaps inner Event.adapter -> meshai source, falling back to the literal adapter name for anything not in the table (logged at DEBUG when a translation happens). before -> after (Event.source): wfigs_incidents / wfigs_perimeters -> fires nwis -> usgs swpc_alerts / swpc_kindex / swpc_protons -> swpc wzdx -> traffic nws, usgs_quake, firms -> unchanged (1:1, omitted from table) unknown (e.g. experimental_foo) -> passthrough as-is Tests: tests/test_central_consumer.py parametrized test_central_adapter_source_remap (6 cases: 4 remaps + nws passthrough + unknown passthrough). Full suite: 283 passed. In-prod verify (rebuilt, ephemeral probe over real Central data): the four observed adapters now normalize to source=fires/usgs/swpc/traffic; nws passes through. No live flip needed; container stays native baseline + healthy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| ea0c68097a |
fix(central): v0.4 D.1 -- subject-domain category fallback (traffic 'work_zone.wzdx' was mapping to 'other')
Surfaced during the Phase D rollout flipping all five remaining domains to
central. Central's traffic categories are NOT domain-prefixed -- the inner
Event.category for a work zone is "work_zone.wzdx", not "traffic.work_zone".
The prefix table in map_category therefore missed and returned "other", which
would break category-based routing/digest grouping for central-sourced traffic.
before: map_category("work_zone.wzdx") -> "other"
after: when the category table misses, fall back to the stable subject domain
token (central.<domain>.<...>): central.traffic.* -> traffic_congestion.
Added category_from_subject() + a domain->category map (wx, fire, quake,
hydro, space, disaster, traffic, traffic_flow, traffic_cameras). The
well-prefixed domains (wx.alert, fire.incident, hydro., space.alert)
still match the primary table; the fallback only fires on a miss, so a
known domain never yields "other" again.
Test: tests/test_central_consumer.py gains test_subject_domain_fallback_for_unmapped_category
(category_from_subject + a 'work_zone.wzdx' message -> traffic_congestion).
Full suite: 277 passed.
Verified in prod (rebuilt, all 5 flipped to central): the per-domain
LAST_PER_SUBJECT normalize probe now shows traffic -> category=traffic_congestion
(was 'other'); the other four domains unchanged and clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| a491684861 |
fix(central): v0.4 C.3.1 -- preserve secret refs in save_section + deliver_policy=NEW (no backlog flood)
Fixes the two real bugs C.3 surfaced when flipping usgs_quake to central. BUG #1 -- GUI save dropped ${VAR} secret refs (config_loader.save_section). before: A GUI PUT round-trips the *interpolated* secret value (GET returns the resolved key string, e.g. the real TomTom key). save_section's check_secrets saw a literal string at a SECRET_FIELDS path, didn't recognize it as a ref, and DROPPED it -- losing the on-disk ${TOMTOM_API_KEY} placeholder. C.3's flip PUT stripped TomTom's key. after: check_secrets now reads the raw on-disk value (pre-interpolation) for each secret field and decides three ways: on-disk ${VAR} and new == resolved(VAR) -> keep the ${VAR} ref on-disk ${VAR} and new != resolved(VAR) -> intentional change, store it no on-disk ${VAR} ref -> reject (never write a raw secret to a domain file) ${VAR} resolution mirrors load: os.environ first, then /data/secrets/.env. The common case (GUI re-saves unchanged config) now preserves the placeholder instead of dropping it. BUG #2 -- CentralConsumer replayed the entire retained backlog on first flip. before: js.subscribe(...) with no config -> default deliver_policy=all. Fine for quake (682 msgs) but would flood the bus with ~330k traffic_flow messages on first flip. after: consumer_config() -> ConsumerConfig(deliver_policy=DeliverPolicy.NEW): only messages published AFTER consumer creation. meshai won't see the backlog on first flip -- acceptable, Central is a live firehose for current events. (NOT geo-filtering -- that's a Central-side issue filed separately for the Central project.) Files: meshai/config_loader.py (save_section secret preservation), meshai/central/consumer.py (consumer_config() + deliver_policy=NEW), tests/test_save_section_secret_preserve.py (new), tests/test_central_consumer.py (deliver_policy assertion). Verification: - (A) py_compile clean on config_loader.py + consumer.py. - (C) pytest -q: 276 passed (272 + 4 new -- preserve-unchanged-ref, changed-value-written, no-placeholder-still-rejects, deliver_policy=NEW). The C.2.1 strip test still passes (no placeholder -> reject). - (D) In-prod (rebuilt): GET+PUT /api/config/environmental round-trip -> {"saved":true}; on-disk traffic.api_key stayed '${TOMTOM_API_KEY}' (SECRET_REF_PRESERVED: True), not the literal key; disk restored to baseline. consumer_config().deliver_policy == DeliverPolicy.NEW in the built image. Follow-up for D rollout: the durable 'meshai-v04-central_quake_' created during C.3 was made with deliver_policy=all; re-flipping a domain may need that stale durable deleted on the Central NATS server first (config mismatch on re-subscribe). D rollout (remaining domains) is now safe: GUI flips preserve secret refs and new subscriptions don't replay huge backlogs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| a4f23c226e |
fix(dashboard): v0.4 C.2.1 -- route PUT /config to multi-file save_section (Rule 17 persistence unblocked)
C.2 surfaced that GUI config saves were broken in the prod multi-file
layout. This fixes it. Pre-existing v0.3-era bug (predates C.2; affected
EVERY config section, not just environmental).
Save flow (before -> after):
before: PUT /api/config/{section} -> config.py::load_config(config.yaml)
[monolithic, vanilla YAML] -> blows up on the !include orchestrator
("could not determine a constructor for the tag '!include'"),
then config.py::save_config (same !include-blind path). Every save
500'd; nothing persisted.
after: PUT validates the body by coercing to the section dataclass (runs
__post_init__ validators, e.g. feed_source), then persists via
config_loader.py::save_section(section, dict, config_dir) -- the
multi-file / !include-aware writer. It writes ONLY the section's
target file (env_feeds.yaml for environmental, notifications.yaml,
llm.yaml, ...), strips SECRET_FIELDS (traffic.api_key, firms.map_key)
and extracts LOCAL_FIELDS (ducting lat/lon -> local.yaml). The
orchestrator config.yaml and its !include directives are never
re-parsed. Live app.state.config is kept in sync via setattr when
the section isn't restart-required (no disk reload needed).
Also: save_section now tolerates a top-level LIST section (mesh_sources) --
it cleans each item for secrets and writes the list directly instead of
assuming a dict (which would have crashed). Other callers of save_config are
untouched (it remains valid for the monolithic single-file path).
Files: meshai/dashboard/api/config_routes.py (PUT handler + import),
meshai/config_loader.py (save_section list guard),
tests/test_dashboard_config_save.py (new).
Verification:
- (A) py_compile clean on config_routes.py + config_loader.py.
- (C) pytest -q: 272 passed (269 + 3 new -- save_section writes env_feeds,
strips secret fields, handles the mesh_sources list section).
- (D) Rebuilt prod; ran the C.2 round-trip again, now SUCCESS: backup
env_feeds.yaml (md5 dde5d634...), GET then PUT /api/config/environmental ->
{"saved":true,"restart_required":false} (NO !include error); disk reflected
it (feed_source on all 10 adapters + central block written); restored from
backup -> md5 matches original -> DISK_PRISTINE_RESTORED.
- (E) Rule 17 round-trip confirmed: the GUI can now SAVE config that
round-trips to disk in the multi-file !include layout, secrets staying in
.env and local fields in local.yaml.
C.3 (quake -> central flip) is now unblocked: feed_source can be flipped and
saved from the GUI.
Follow-up (non-blocking): mesh_sources per-item secret stripping
(mesh_sources.*.api_token) isn't matched by the section-relative check in the
new list path; mesh_sources files are volume-only (not git) and this was no
worse before, but worth tightening when mesh_sources GUI save is exercised.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 73c007d227 |
feat(central): v0.4 C.1 Central connector backend (no-op until adapter source flipped)
Adds the backend for sourcing environmental feeds from Central's NATS
JetStream firehose instead of (or alongside) meshai's native adapters.
Architecture is Matt-approved Option 3' (dedicated package + per-adapter
source switch surfaced on the existing Environmental config).
NO-OP POSTURE (intentional): every adapter defaults to feed_source="native"
and environmental.central.enabled defaults false, so on a stock config the
CentralConsumer starts and subscribes to nothing -- behavior is byte-for-byte
v0.3. Live env_feeds.yaml is unchanged on disk; an operator who touches
nothing sees no change. Flipping an adapter to central is Phase C.3; the
dashboard UI for it is Phase C.2.
What landed:
- meshai/central/ package (CentralConsumer): async start()/stop(), JetStream
durable subscribe to subjects derived from adapters with feed_source=central,
and _on_message -> normalize -> bus.emit. nats-py is lazy-imported only on
the connect path, so no-op boot has zero NATS dependency.
- Normalization (CloudEvents envelope -> Central Event -> upstream data):
source = inner Event.adapter
category = Central hierarchical string -> meshai flat, via a small
table-driven prefix map (map_category)
severity = 0|1->routine, 2->priority, 3|4->immediate, null->routine
lat/lon = geo.centroid, swapped from GeoJSON [lon,lat] -> (lat,lon)
group_key/inhibit = outer envelope id (dedup parity with native adapters)
expires/timestamp parsed from ISO-8601
Event.data = upstream payload verbatim (generic _enriched merge, preserved
as-is incl. hydro's extra usgs_site/usgs_stats bundles)
- Tombstone (`.removed.` subject or `:removed` id suffix) -> a "clear" Event
carrying the ORIGINAL group_key (`:removed` stripped) + data._central_tombstone
so the grouper/inhibitor lets the prior event lapse naturally.
- config.py: a `_SourcedFeed` mixin adds `feed_source: native|central`
(validated in __post_init__) to all 10 adapter configs; new
CentralConsumerConfig as environmental.central { enabled, url, durable,
connect_timeout }. Both ride the generic _dict_to_dataclass coercion, so
they are GUI-editable via PUT /config/environmental (Rule 17) -- frontend
fields come in C.2.
- env/store.py: each adapter is instantiated only when
enabled AND feed_source=="native"; a feed_source=central adapter is skipped
natively (debug-logged) so Central can own it without a duplicate.
- main.py: CentralConsumer constructed + started after start_pipeline(),
stopped in stop().
DEVIATION FROM SPEC (documented): the spec named the new field `source`, but
FIRMSConfig already has a `source` field (the satellite product,
"VIIRS_SNPP_NRT"). To avoid the collision the field is named **feed_source**
across all adapters. Everything else follows the spec.
NETWORKING: zero infra change required. The meshai container already reaches
the Central NATS server directly (TCP to 100.64.0.12:4222 OK) and resolves
central.echo6.mesh via the Phase 2.6.6 MagicDNS fix. No docker-compose edit;
default bridge works (LXC host masquerades to the Tailscale CGNAT range). The
lighter bridge-route / host-net / sidecar fallbacks were not needed.
Tests: tests/test_central_consumer.py (11) + tests/test_config_source_field.py
(6): no-op-when-native, subjects-when-central, source-gate skips native
instantiation, normalize+emit, _enriched preserved verbatim, tombstone->clear,
severity map (0-4/null), category map (>=4 strings), async _on_message
emits+acks, start() no-op without NATS, feed_source default/validate/reject/
dict-coercion. Full suite: 269 passed (was 253 + 16 new).
Verification: (A) no bare self._x() in consumer.py. (B) py_compile clean.
(C) 269 passed. (D) rebuilt prod -- 8 native adapters, pipeline started,
native nifc/traffic emissions still flowing, healthy, no errors, log
"CentralConsumer started; 0 subjects subscribed -- no adapters set to central".
(E) in-container synthetic _on_message injection normalized correctly
(usgs_quake/earthquake_event/immediate, centroid swapped, _enriched preserved)
and reached the bus; ephemeral, no config change to roll back.
C.2 (dashboard frontend for the feed_source switch + central connection) is next.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 20e0dec28a |
fix(notifications): Phase 2.16.1 unblock pipeline -- grouper flush + rules coercion + toggle warning
Phase 2.16 found the live notification pipeline never delivered any
environmental event. Two independent blocking bugs, both fixed here.
BUG A -- grouper held events forever (nothing drove tick()).
Every adapter event sets a group_key, so all were buffered in the Grouper
and never flushed (start_pipeline only started the DigestScheduler; no
tick driver existed). Fixes (per Matt's decisions):
- Grouper.handle(): immediate-severity events now BYPASS the window
entirely (delivered straight to next_handler), no buffering latency.
routine/priority still coalesce.
- start_pipeline(): schedules an asyncio flush task that calls
grouper.tick() every `grouper_flush_seconds` (default 5s) so
coalesced events drain within the window even when poll cadence is
sparse. stop_pipeline() signals + cancels it.
before/after (grouper held_count): an immediate+group_key event used to
sit held (count 1) forever; now held_count==0 on arrival (bypassed). A
routine event is held (count 1) then drained to 0 by tick()/flush.
BUG B -- notification rules loaded as dicts, crashing the dispatcher.
Root cause (more precise than 2.16's guess): the rules coercion is NOT
missing from the multi-file loader -- it lives in _dict_to_dataclass's
explicit `elif key == "notifications"` branch, but that branch was DEAD
CODE, shadowed by the generic `if hasattr(field_type,
"__dataclass_fields__")` handler that runs first for every dataclass
field (including notifications). So Config.notifications.rules stayed a
list of dicts on ALL load paths, and Dispatcher._matching_rules threw
`AttributeError: 'dict' object has no attribute 'enabled'`. Fix: hoist
the notifications special-handling ahead of the generic handler (and drop
the now-truly-dead duplicate elif).
before/after (cfg.notifications.rules[0] type): dict -> NotificationRuleConfig.
OBS C -- empty enabled_toggles. Left as 'pass all' for v0.3 (per Matt);
added a startup WARNING in build_pipeline so operators see gating is off:
"enabled_toggles is empty -- ToggleFilter passing all events. Configure
toggles to enable gating." (confirmed firing live).
Tests:
- tests/test_pipeline_grouper.py (new): test_immediate_severity_bypasses_grouper,
test_periodic_flush_drains_routine, test_priority_is_also_coalesced_not_bypassed.
- tests/test_config_loader.py (new): test_multifile_load_coerces_notification_rules,
test_rules_attribute_access_does_not_raise (regression guards for Bug B).
- tests/test_pipeline_inhibitor_grouper.py (updated): 5 existing grouper
hold/coalesce/flush tests primed the grouper with immediate+group_key
events expecting them to be held; switched those to 'priority' (still
buffered; still outranks the routine event in the inhibitor-chain test)
to match the intended immediate-bypass behavior.
Full suite: 253 passed (was 248 + 5 new; 5 existing updated, none lost).
VERIFICATION (rebuilt prod, traced end-to-end via in-process build_pipeline
probe with a recording channel + live config):
- rules[0] type: NotificationRuleConfig (Bug B fixed).
- IMMEDIATE event: held_count==0 on emit (bypassed) -> reached
channel.deliver(): delivered=[('PROBE_RULE','E2E IMMEDIATE')].
- ROUTINE event: held_count==1 -> after flush 0 -> reached
channel.deliver(): delivered+=[('PROBE_RULE','E2E ROUTINE')].
- Natural Summit-Creek-shaped nifc wildfire_incident (routine, no
matching dispatch rule): held 1 -> after flush -> landed in the digest
accumulator (1 event). End-to-end channel.deliver evidence = the
RecChannel.deliver() calls above.
- Live container: 8 adapters, healthy, "Grouper flush task started
(every 5s)", the enabled_toggles warning fired, and NO dispatcher
AttributeError/traceback.
Follow-up (non-blocking): several Phase 2.7-2.14 categories (e.g.
wildfire_incident, earthquake_event) aren't in the category->toggle map,
so they fall to toggle 'other'. Harmless while enabled_toggles is empty
(pass-all), but should be mapped before toggle gating is turned on.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 8b2cdeee0b |
feat(notifications): Phase 2.14 USGS earthquake adapter (new) -- closes Rule 16 Seismic standalone path
First net-new environmental adapter (prior phases wired existing ones). Adds meshai/env/usgs_quake.py with USGSQuakeAdapter + USGSQuakeConfig, polling a keyless USGS earthquake GeoJSON feed and emitting one Event per qualifying quake. Establishes the standalone Seismic path (Rule 16); Central becomes the dual-source in v0.4. Adapter (mirrors the fires/usgs-water per-event pattern): - Feed: https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/2.5_day.geojson (M2.5+ past day -- M1.0 too noisy, M4.5+ too sparse for the region). Tick 300s. - Filters each feature by min_magnitude AND a geographic bbox. - Per quake: source=usgs_quake, category=earthquake_event, stable event_id = the USGS feature id (e.g. "us6000abcd"), lat/lon from geometry.coordinates[1],[0], region tag from config (default "magic_valley"). - to_event(): category earthquake_event, magnitude-binned severity passed through, group_key = inhibit_key = the USGS id. Defensive None for missing id / coords / magnitude. get_events()/health_status mirror the other adapters. MAGNITUDE -> SEVERITY BINS (as proposed): M < 3.5 -> routine 3.5 <= M < 5.0 -> priority M >= 5.0 -> immediate ('sig' is captured in the event dict as metadata but severity is magnitude-binned -- clearer and matches the spec's primary suggestion.) GEOGRAPHIC BBOX (as proposed) -- [west, south, east, north]: [-115.5, 42.0, -110.0, 45.2] Covers Magic Valley / Twin Falls (SW), the Lost River Range / Borah Peak and Sawtooths (central Idaho, seismically active -- 1983 M6.9), the eastern Snake River Plain / INL, and the Yellowstone caldera (NW Wyoming). An empty bbox disables the geographic filter (accepts all). Wiring: - config.py: new USGSQuakeConfig dataclass; usgs_quake field on EnvironmentalConfig; loader branch in _dict_to_dataclass. - store.py __init__: registers self._adapters["usgs_quake"] when enabled -- this is what grows the live adapter count 7 -> 8. - store._ingest: NO dedicated branch added. usgs_quake is a standard per-event adapter, so the existing generic "else" loop (dedup on (source, event_id) + _emit_event) already routes it. (The swpc/ducting branches are special only because they also maintain status blobs.) - env_feeds.yaml (live /data/config): added usgs_quake block, enabled:true, default bbox/min_mag/region. Rule 17: GUI-editable config (env_feeds.yaml). Rule 18 N/A -- USGS earthquake feed is keyless (no .env entry; .ref credentials has no USGS/ArcGIS/quake key). Rule 16: standalone path established + validated in-container. Tests: tests/test_adapter_usgs_quake.py (15 tests) mirrors the 2.12/2.13 shape -- severity bins, _fetch severity assignment, magnitude filter, geographic filter (in-bbox vs California/out), empty-bbox-accepts-all, dedup id stable across ticks for the same quake id, category, severity pass-through, group_key/inhibit_keys, field population, defensive cases (missing id/coords/magnitude/corrupted -> None), and malformed-feature skipping. _fetch tests patch urlopen with synthetic FeatureCollections. Full suite: 248 passed. Live smoke test (prod container, rebuilt): clean startup, adapter count grew 7 -> 8 ("EnvironmentalStore initialized with 8 adapters"), healthy, no traceback, no usgs_quake errors. In-container standalone tick over the real feed succeeded (is_loaded=true, last_error=null, consecutive_errors=0); the feed returned 54 global M2.5+ quakes, 0 inside the Magic Valley->Yellowstone bbox right now (quiet) -- so no Event is emitted, acceptable, and it exercises the fetch + magnitude + geographic filter + no-emit path on live data. The emission path (in-region quake -> earthquake_event) is unit-validated and uses the same store->bus path emitting live for NWS, traffic, and NIFC fires. Note (.gitignore): line 36 `env/` (a virtualenv pattern under "Virtual environments") collaterally matches meshai/env/, so this NEW file required `git add -f` (untracked files there are otherwise ignored and hidden from status). Existing tracked env files are unaffected. Recommended follow-up: anchor the rule to `/env/` so future net-new env adapters don't need -f. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| d3b62ad3c5 |
feat(notifications): Phase 2.13 ducting adapter threshold-crossing emission (severity-tiered, Option C)
Adds a tier-based threshold-crossing emission path to the tropospheric
ducting adapter, which was status-only until now.
EMISSION PATH (before -> after):
before: DuctingAdapter had only get_status(); store._ingest's ducting
branch did `self._ducting_status = adapter.get_status()` and
emitted NOTHING -- no get_events(), no to_event(), event_count
hardcoded 0.
after: the adapter derives a propagation TIER each tick (with
hysteresis) and stages an event on tier change; get_events() +
to_event() added; store._ingest's ducting branch now mirrors the
swpc branch (dedup on (source, event_id) + _emit_event), so a
tier change emits to the pipeline bus.
Option C design (severity-tiered by enhancement strength):
- Driving quantity: min M-gradient (modified refractivity gradient,
M-units/km) the adapter already computes.
- Tiers (ascending strength): normal < super_refraction < duct <
surface_duct.
0 <= g < 79 -> super_refraction -> category rf_anomalous_propagation,
severity routine
g < 0 -> duct (elevated) -> category rf_ducting_enhancement,
severity priority
surface_duct OR g < -100 -> strong/surface duct ->
category rf_ducting_enhancement, severity immediate,
surface flag set in the summary
g >= 79 -> normal -> no event
- Hysteresis / anti-flap: a DEADBAND of 5 M-units (TIER_DEADBAND) on the
two gradient boundaries (79 and 0). A tier change commits only once the
gradient is past the boundary by the deadband, so a wiggle right at a
threshold does not flap-trip across the 3h poll interval / 30-min
Inhibitor TTL mismatch (the Inhibitor TTL is shorter than the poll
interval, so anti-flap must live in the adapter). The most-severe
surface/strong-duct tier is categorical (duct reaches the ground) and is
intentionally NOT held back or onto by the deadband -- it fires and
clears promptly. (Deadband = 5 M-units chosen per the 5-10 guidance.)
- Stable event_id (SWPC idiom): "ducting_{tier_code}_{lat}_{lon}", e.g.
"ducting_duct_42.56_-114.47". A sustained tier coalesces on this
group_key (the store dedups it); an escalation to a stronger tier yields
a new key and re-notifies. group_key = sole inhibit_key; severity tiering
delegated to the Inhibitor.
- Prior-state tracking: self._last_tier persists across ticks (the
deadband needs the last committed tier); _parse_response rebuilds
_status wholesale, so _update_events runs at the end of each parse.
- Ducting is geographic: events carry the assessment location's lat/lon
(config.latitude/longitude). Defensive: missing/normal tier, missing
location, or missing gradient -> None; try/except-guarded.
Rule 17: no new tunable (latitude/longitude/tick_seconds already in
env_feeds.yaml; TIER_DEADBAND is an internal constant). Rule 18 N/A --
Open-Meteo GFS (api.open-meteo.com) is keyless. Rule 16: standalone fetch
path validated in-container.
Tests: tests/test_adapter_ducting.py (19 tests) mirrors the 2.12 SWPC
shape -- tier classification (normal/super_refraction/duct/surface_duct),
severity tiering, scale->category mapping, group_key/inhibit_keys, field
population, defensive cases (normal/missing location/missing gradient/
corrupted -> None), plus regression guards: dedup id stable across
same-tier ticks, tier escalation yields a new id, and TWO deadband guards
(a sub-deadband wiggle at the 0 boundary and at the 79 boundary holds the
prior tier; surface duct is not held by the deadband). Full suite: 233
passed.
Live smoke test (prod container, Phase 2.13 code rebuilt in): clean
startup, 7 env adapters loaded (ducting already counted), healthy, no
traceback. An in-container standalone _fetch of the Open-Meteo GFS
endpoint succeeded (fetch_ok=true, is_loaded=true, last_error=null,
consecutive_errors=0) -- 3/3 repeat probes clean. The current atmosphere
is normal (min M-gradient 122.5 >= 79) so tier=normal and no Event is
emitted -- acceptable, and it exercises the no-emit path and the tier
classifier on live data. NOTE: the running container's first ducting tick
logged a transient "[SSL: UNEXPECTED_EOF_WHILE_READING]" connection error;
the immediate and repeated standalone probes all succeeded, so this was a
transient upstream TLS drop (not DNS/auth/config) and the adapter degrades
gracefully (logs, increments consecutive_errors, returns False, no crash).
The emission path (tier change -> rf_anomalous_propagation /
rf_ducting_enhancement) is unit-validated and uses the same store->bus
path that emitted live for NWS, traffic, and NIFC fires.
Follow-up (not in this change): DuctingAdapter.health_status still returns
event_count hardcoded 0; now that the adapter emits, it could report
len(self._events). Cosmetic (health endpoint only); left out to keep the
diff scoped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| dda8b8f96f |
feat(notifications): Phase 2.12 SWPC space weather adapter + dedup fix
Wires the NOAA SWPC adapter into the notification EventBus and fixes a
dedup bug in its event id, following the Phase 2.7-2.11 pattern.
(A) DEDUP FIX (the regression this phase guards):
before: event_id = f"swpc_r{r_scale}_{int(time.time())}"
after: event_id = f"swpc_{code}{level}" # e.g. "swpc_g3"
The old id embedded int(time.time()), so every poll produced a unique id.
The store dedups env events on (source, event_id), so each tick during a
blackout was treated as new -> re-emitted to the bus every scales poll
(300s) and accumulated phantom entries in the store. The new id is stable
per condition: a sustained storm coalesces across ticks; only an
escalation to a new level (e.g. G3 -> G4) yields a new id and re-notifies.
Re-emit suppression is the Inhibitor's job (TTL ~1800s), not the id's.
(B) _update_events expanded R-scale-only -> all three NOAA scales:
- R (Radio Blackout) -> category rf_propagation_alert
- S (Solar Radiation Storm) -> category solar_radiation_storm
- G (Geomagnetic Storm) -> category geomagnetic_storm
Emit threshold: level >= 1 (level 0 / quiet emits nothing). Severity is
tiered in _update_events and passed through by to_event:
level 1-2 -> routine, 3-4 -> priority, 5 -> immediate.
(Scope/threshold approved by Matt before applying: "R/S/G at level >= 1".)
Each event carries scale/level discriminator fields for to_event.
(C) to_event(): category from scale, severity pass-through, group_key /
inhibit_keys = the stable event_id (single key; tiering -> Inhibitor).
SWPC conditions are global, so the Event carries lat=None, lon=None and
region="global" (Event.lat/lon are Optional and Event has a region field).
Defensive: missing scale, level<1, or missing event_id -> None;
try/except-guarded.
No store.py change: store already routes swpc through to_event in _ingest
(the swpc special-case) and the Phase 2.9 None-guard handles None returns.
Rule 17: no new tunable. Rule 18 N/A -- SWPC services.swpc.noaa.gov is
keyless (no .env entry; .ref credentials has no SWPC/NOAA key, confirming
none needed). Rule 16: standalone fetch path validated in-container.
Tests: tests/test_adapter_swpc.py (14 tests) mirrors the 2.11 shape --
scale->category mapping, severity pass-through, _update_events severity
tiering (1-2/3-4/5), group_key/inhibit_keys, all-three-scales-emit,
quiet-emits-nothing, field population (lat/lon None + region global), and
defensive cases (missing scale / level 0 / missing id / corrupted -> None).
Plus two dedup regression guards: test_dedup_id_stable_across_ticks
(SAME id across two ticks of the same condition -- fails on the old code)
and test_event_id_changes_with_level (escalation yields a new id). Full
suite: 214 passed.
Live smoke test (prod container, Phase 2.12 code rebuilt in): clean
startup, 7 env adapters loaded, healthy, no traceback, no SWPC errors. An
in-container standalone fetch of the noaa-scales endpoint succeeded
(scales_fetch_ok=true, is_loaded=true, last_error=null,
consecutive_errors=0) over the open API with no DNS/auth errors (Phase
2.6.6 DNS fix). Current conditions are quiet (R0/S0/G0), so no Event is
emitted -- acceptable, and it exercises the level<1 -> no-emit path live.
The emission path (active scale -> rf_propagation_alert / geomagnetic_storm
/ solar_radiation_storm) is unit-validated and uses the same store->bus
path that emitted live for NWS, traffic, and NIFC fires.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| c111211850 |
feat(notifications): Phase 2.11 NIFC fires adapter pipeline integration
Adds NICFFiresAdapter.to_event(), wiring the NIFC/WFIGS wildfire perimeter
adapter into the notification EventBus, following the Phase 2.7 traffic /
2.9 USGS / 2.10 avalanche pattern.
to_event() design:
- Category: every active perimeter with a reported size maps to a single
wildfire_incident category (the adapter's WFIGS query already filters to
active WF incidents in the configured state).
- Severity: PASSED THROUGH unchanged. The adapter computes severity by
proximity to region anchors (< 25 km -> priority, else routine), which
is a richer, more actionable signal for a mesh-notification use case
than raw acreage. I deliberately did NOT invent acreage breakpoints --
pass-through matches the 2.9/2.10 pattern and defers tiering to the
pipeline Inhibitor. (Flagged for review: if acreage-based or
containment-based severity is preferred, it belongs in the adapter's
_fetch severity logic, not to_event.)
- Summary: incident name + acreage + % contained + distance to nearest
anchor.
- group_key/inhibit_keys: the adapter's stable "nifc_{name}_{state}"
event_id as both. Re-polls of the same incident coalesce; single
inhibit key lets the Inhibitor suppress lower-severity re-emissions.
- Defensive: missing centroid (lat/lon), missing event_id, or missing/zero
acreage returns None; try/except-guarded.
No store.py change: the Phase 2.9 _emit_event None-guard already handles
to_event() returning None, and store gates emission on
hasattr(adapter, "to_event").
Rule 17: no new tunable. fires enabled / state / tick_seconds already
exist in env_feeds.yaml (GUI-editable). Rule 18 N/A -- the WFIGS
Interagency Perimeters ArcGIS FeatureServer is keyless (no .env entry;
the .ref credentials store has no NIFC/ArcGIS/wildfire key, confirming
none is needed). Rule 16: standalone fetch path validated in-container.
FIRMS side-investigation (flagged in the 2.10 report): firms is disabled
because it needs a NASA FIRMS map key that is not provisioned --
env_feeds.yaml has firms.enabled=false with map_key='' (not even a
${FIRMS_MAP_KEY} reference), and /data/secrets/.env has no FIRMS key.
Intentional/blocked-on-key, not a bug. No action this phase.
Config note: fires was already enabled (state US-ID) and already one of
the 7 live adapters (store key "nifc"), so this phase keeps the count at 7
(no 7->8 change) and required no env_feeds.yaml edit. No seasonal
short-circuit, so no temp config wiggling was needed (unlike 2.10).
Tests: tests/test_adapter_fires.py (12 tests) mirrors test_adapter_usgs /
test_adapter_avalanche -- category (always wildfire_incident, independent
of severity), severity pass-through, group_key/inhibit_keys,
distinct-incident keys, field population, summary content, and the
defensive cases (zero acreage -> None, missing centroid/event_id -> None,
corrupted -> None). Full suite: 200 passed.
Live smoke test (prod container, Phase 2.11 code rebuilt in): clean
startup, 7 env adapters loaded, no traceback. There IS an active Idaho
incident today, so this produced a real end-to-end emission rather than
the empty-result cases of 2.9/2.10: the running store logged "NIFC fires
updated: 1 active in US-ID" and "Emitted nifc event cc4bd340be7fd57e
(wildfire_incident) to pipeline bus". An in-container standalone fetch
confirmed health is_loaded=true, last_error=null, consecutive_errors=0,
event_count=1 -- the WFIGS ArcGIS endpoint was reached with no DNS/auth
errors (Phase 2.6.6 DNS fix). The Summit Creek incident (1,500 ac, 0%
contained, ~72 km from the Twin Falls anchor) mapped to
wildfire_incident / routine as designed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 1d35188b98 |
feat(notifications): Phase 2.10 avalanche adapter pipeline integration
Adds AvalancheAdapter.to_event(), wiring the avalanche.org map-layer
adapter into the notification EventBus, following the Phase 2.7 traffic /
2.9 USGS pattern.
to_event() design (emit only elevated danger):
- Category from danger_level: High/Extreme (4-5) -> avalanche_warning;
Considerable (3) -> avalanche_watch.
- Low/Moderate (1-2) and No-Rating (-1/0) have no distinct trend trigger
in this adapter and are intentionally NOT emitted (return None) -- the
two categories are warning/watch only, matching the spec.
- Severity: passed through unchanged from the adapter's danger mapping
(danger >= 4 -> priority, else routine; the adapter never emits
"immediate"). Severity tiering is delegated to the pipeline Inhibitor.
- Summary: headline + danger name + travel advice.
- group_key/inhibit_keys: the adapter's stable "avy_{center}_{zone}"
event_id as both. Re-polls of the same zone coalesce; single inhibit
key lets the Inhibitor suppress lower-severity re-emissions.
- Defensive: missing centroid (lat/lon), missing event_id, or missing
danger_level returns None; try/except-guarded.
No store.py change: the Phase 2.9 _emit_event None-guard already handles
to_event() returning None, and store gates emission on
hasattr(adapter, "to_event").
Rule 17: no new tunable. avalanche enabled / center_ids / season_months
already exist in env_feeds.yaml (GUI-editable). Rule 18 N/A -- the
avalanche.org v2 public map-layer API is keyless (no .env entry; the
.ref credentials store has no avalanche provider key, confirming none is
needed). Rule 16: standalone fetch path validated in-container below.
Config note: avalanche was already enabled (center_ids: [SNFAC], the
Sawtooth Avalanche Center -- the correct South Central Idaho / Magic
Valley center). It was already one of the 7 live adapters, so this phase
keeps the count at 7 (no 7->8 change) and required no env_feeds.yaml
edit. There is no per-zone config knob; the adapter fetches all zones for
the configured center.
Tests: tests/test_adapter_avalanche.py (14 tests) mirrors
test_adapter_usgs -- category split (warning vs watch), severity
pass-through, group_key/inhibit_keys, distinct-zone keys, field
population, and the non-emit/defensive cases (low/moderate -> None,
no-rating -> None, missing danger_level/centroid/event_id -> None,
corrupted -> None). Full suite: 188 passed.
Live smoke test (prod container, Phase 2.10 code rebuilt in): clean
startup, 7 env adapters loaded, no traceback. Late May is off-season
(season_months [12,1,2,3,4]) so tick() short-circuits in normal
operation. To exercise the open-API path, a one-shot standalone fetch was
run in-container with an all-months config against center SNFAC: health
is_loaded=true, last_error=null, consecutive_errors=0, last_fetch set,
off_season=false -- the fetch reached api.avalanche.org with no DNS/auth
errors (Phase 2.6.6 DNS fix). event_count=0 because all SNFAC zones are
server-side off_season in late May, so no Event is emitted -- acceptable
per the seasonal caveat. The temporary season_months edit was reverted
and the container restarted on the real config (7 adapters, healthy). The
emission path (elevated -> avalanche_warning / avalanche_watch) is
unit-validated and is the same store->bus path emitting live for NWS and
traffic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 4feb6a1895 |
feat(notifications): Phase 2.9 usgs water adapter pipeline integration
Adds USGSStreamsAdapter.to_event(), wiring the USGS Water Services stream
gauge adapter into the notification EventBus, following the Phase 2.7
traffic pattern.
to_event() design (emit only actionable/elevated readings):
- Category from flood_status: an exceeded stage (Minor/Moderate/Major
Flood) -> stream_flood_warning; "Action Stage" (approaching) ->
stream_high_water.
- A routine reading has no flood_status and is intentionally NOT emitted
(returns None) -- the two categories are both flood-specific and routine
gauge chatter is not actionable. This matches the spec ("category ...
based on flood_status").
- Severity: passed through unchanged from the adapter's NWPS-stage logic
(action->routine, minor/moderate->priority, major->immediate).
- Summary: reading value/unit + flood status.
- group_key/inhibit_keys: a single stable {site_id}_{param} key (the
adapter's own event_id) as both. Re-polls coalesce; severity tiering is
delegated to the pipeline Inhibitor (no severity encoded in the key).
- Defensive: missing lat/lon or event_id returns None; try/except-guarded.
store fix (meshai/env/store.py): _emit_event now skips a None return from
to_event() instead of passing it to bus.emit(). Required because usgs
returns None for the common (routine) reading; also retroactively protects
the defensive None returns of the FIRMS/traffic/roads511 adapters, which
previously would have logged a spurious "Failed to emit" warning.
Rule 17: no new tunable. usgs sites / tick_seconds / flood_thresholds
already exist in env_feeds.yaml (GUI-editable). Open API, no key, no .env
entry. Rule 16: standalone path validated end-to-end below.
Tests: tests/test_adapter_usgs.py (13 tests) mirrors test_adapter_traffic
-- category split (flood vs action), severity pass-through,
group_key/inhibit_keys, field population, and the non-emit/defensive cases
(routine -> None, missing lat/lon -> None, missing event_id -> None,
missing properties -> None, corrupted -> None). Full suite: 174 passed.
Live smoke test (prod, sites 13090500 Snake R nr Twin Falls, 13092747 Rock
Creek at Twin Falls, 13108150 Salmon Falls Creek nr Hagerman): clean
startup, 7 env adapters loaded, no traceback. "USGS streams updated: 6
readings from 3 sites" with NWPS flood stages resolved for all 3 -- fetch
succeeds over the open API with no DNS/auth errors (Phase 2.6.6 DNS fix).
All gauges currently below action stage, so flood_status is None and
to_event correctly emits nothing; the new None-guard skipped all 6 with no
error log. The emission path (elevated -> stream_flood_warning /
stream_high_water) is unit-validated and is the same store->bus path
emitting live for NWS (weather_warning/statement) and traffic
(traffic_congestion).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| f273a8d5b0 |
feat(notifications): Phase 2.8 roads511 adapter pipeline integration
Adds Roads511Adapter.to_event(), wiring the state 511 road-events adapter
into the notification EventBus following the Phase 2.7 traffic pattern.
to_event() design:
- Category: fixed "road_closure".
- Severity: passed through unchanged from the adapter's existing
_parse_event logic (priority on closure, else routine).
- Summary enriched with closure status, roadway, and description.
- group_key: the stored event_id (already the stable "511_{id}" key), so
re-polls of the same incident coalesce.
- inhibit_keys: a single key equal to group_key. Severity tiering is
delegated to the pipeline Inhibitor (ranks routine<priority<immediate
per shared key, suppressing lower-severity re-emissions of the same
incident within the Inhibitor TTL). No severity encoded into the key.
- Defensive: missing lat/lon or missing event_id returns None; whole body
is try/except-guarded (returns None on corruption).
Store wiring: no change. EnvironmentalStore._ingest()'s generic "else"
branch already emits any adapter exposing to_event() (live since 2.6.5).
Rule 17: to_event introduces no new tunable. (The state base_url / bbox /
api_key already exist in Roads511Config and env_feeds.yaml; secrets go in
/data/secrets/.env via ${VAR}, never git.)
Tests: tests/test_adapter_roads511.py (14 tests) mirrors
test_adapter_traffic.py -- category, severity pass-through,
group_key/inhibit_keys, field population, defensive cases. Full suite:
161 passed.
live smoke test SKIPPED: Idaho 511 v2 (511.idaho.gov/api/v2) requires an
API key ("Invalid Key" response) and none is available in .ref/credentials
(cannot self-register). Per the standing key-less-adapter policy, the code
+ unit tests are committed and Gate D is skipped; roads511 is left disabled
in prod (enabling it keyless would only emit HTTP 400 errors). The
to_event() path is fully unit-validated and structurally identical to the
live traffic/FIRMS wiring (same EnvironmentalStore->EventBus path); live
validation will run if/when an Idaho 511 key is provided.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| d9cc80daf8 |
feat(notifications): Phase 2.7 traffic adapter pipeline integration
Adds TomTomTrafficAdapter.to_event(), wiring the traffic adapter into
the notification EventBus following the FIRMS pattern (Phase 2.6).
to_event() design:
- Category: fixed "traffic_congestion" (a road closure raises severity,
not category).
- Severity: passed through unchanged from the adapter's existing
_fetch_point logic (priority on closure / heavy congestion, else
routine). No threshold is re-derived or introduced in to_event.
- Summary enriched with current/free-flow speed, % free flow, closure,
and confidence.
- Defensive: missing lat/lon or missing corridor identity returns None;
the whole body is try/except-guarded (returns None on corruption).
Inhibit-key composition:
- A single stable per-corridor key, "traffic_{corridor}" (lowercased,
spaces->_), is used as BOTH group_key and the sole inhibit_key. This
matches the adapter's own event_id, so re-polls of a corridor coalesce.
- Severity tiering is delegated to the pipeline Inhibitor, which ranks
routine<priority<immediate per shared inhibit_key: a higher-severity
emission for a corridor suppresses lower-severity re-emissions of the
same corridor within the Inhibitor TTL window. No severity is encoded
into the key (mirrors FIRMS's spatial-key approach).
Store wiring: no change. EnvironmentalStore._ingest()'s generic "else"
branch already emits any adapter exposing to_event() (live since 2.6.5).
Rule 17: to_event introduces no new tunable. The api_key is injected via
the secrets channel ($TOMTOM_API_KEY in /data/secrets/.env, referenced
as ${TOMTOM_API_KEY} in env_feeds.yaml) -- the GUI-editable reference
stays in config while the secret never enters git. The only other knob
in play is the pipeline-level Inhibitor TTL (1800s, set in
build_pipeline), which is pipeline infrastructure, not traffic-owned;
left out of scope.
Tests: tests/test_adapter_traffic.py (15 tests) mirrors
test_adapter_firms.py -- category, severity pass-through,
group_key/inhibit_keys, field population, defensive cases. Full suite:
147 passed.
Smoke test (prod, Magic Valley corridors I-84 @ Jerome, US-93 Perrine
Bridge, US-30 Twin Falls): clean startup, 6 env adapters loaded, no
traceback. "TomTom traffic updated: 3 corridors" (no auth/DNS error),
then 3 Events emitted to the pipeline bus with traffic_congestion
category -- the full store->bus->pipeline path observed live. Emission
count stable at 3 (one per corridor, is_new-gated).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 9c5a106c9f |
feat(env): Phase 2.6 FIRMS adapter emits Events to pipeline bus
Second adapter wired to the new pipeline (after NWS). Reuses the store-side emission logic added in the NWS commit. - FIRMSAdapter.to_event() maps stored dict to pipeline Event. - Category decision: new_ignition vs wildfire_proximity based on properties.new_ignition (computed by FIRMS during ingest from proximity to known fires). - Severity passes through (FIRMS already pre-maps to our 3-level system during _parse_csv). - group_key and inhibit_keys use a spatial grid key (firms:LAT:LON rounded to 0.01 degrees, ~1km) so repeated satellite detections of the same hotspot are coalesced and lower-severity re-detections are inhibited. - Summary text enriched with FRP, confidence, and distance from the nearest region anchor when present. - 13 tests covering category decision, severity pass-through, spatial grouping, and defensive handling of incomplete dicts. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
|||
| 95dc938c2a |
feat(notifications): Phase 2.6 NWS adapter pipeline integration
Wires the NWS adapter to the new notification pipeline via EventBus: - Added fine-grained weather categories: weather_watch, weather_advisory, weather_statement (all routine severity) alongside existing weather_warning - NWSAlertsAdapter._derive_category() maps NWS event type suffix to category: "Warning" -> weather_warning, "Watch" -> weather_watch, etc. - NWSAlertsAdapter.to_event() converts internal event dict to pipeline Event with proper group_key (event_id) and inhibit_keys (Warning suppresses Watch) - EnvironmentalStore accepts optional event_bus parameter - EnvironmentalStore._ingest() emits new events to bus via _emit_event() - 22 new tests in test_adapter_nws.py covering category derivation, severity mapping, and Event field population All 119 tests pass. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
|||
| b2bb7f7a95 |
feat(notifications): Phase 2.5b per-channel-type renderers
Adds dedicated renderer classes per channel type:
- MeshRenderer produces 1+ chunks <=200 chars with (k/N) counters
when the payload overflows. Reuses the toggle-label vocabulary
from the digest. Mesh channels skip re-chunking when the payload
already carries chunk_index metadata (digest path).
- EmailRenderer produces {subject, body} with structured context
lines. Plain text only; HTML body is a future polish.
- WebhookRenderer produces a JSON-serializable dict with stable
schema_version 1.0. Optional fields omitted (not nulled) for
compactness. Designed for reuse by Phase 2.6.5's MQTT event
publisher.
- All four channel implementations (MeshBroadcast, MeshDM, Email,
Webhook) now call their renderer in deliver() before transport.
- New renderer tests cover each renderer in isolation; new channel
integration tests confirm channels actually call their renderer.
Renderers are pure functions of the payload - no network, no
state, fully testable without mocking I/O. The future MQTT
publisher will instantiate WebhookRenderer directly.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
|||
| c9d9a9925c |
feat(notifications): Phase 2.5a channel interface unification
- Switch channels.py from dict-based to dataclass-based interfaces - Add NotificationPayload dataclass and make_payload_from_event helper - Update channel.deliver() to be async with (payload, rule) signature - Add connector parameter to Dispatcher, DigestScheduler, and pipeline builders - Update pipeline tee to use asyncio.create_task for async dispatch - Add create_channel_from_dict for legacy router.py compatibility - Update tests for new async interfaces Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
|||
| a4cb29002d |
fix(notifications): inject llm_backend into build_pipeline
build_pipeline previously constructed its own LLMBackend from
config.llm, which:
- duplicated main.py's already-running backend instance
- failed to inherit env-loaded LLM_API_KEY when called from
short-lived scripts (eyeball checks, tests), forcing fallback
- prevented pipeline components from sharing the live backend
build_pipeline and build_pipeline_components now require an
llm_backend parameter. main.py passes the same instance it
constructed for its primary responder. Tests pass mocks. The
digest accumulator now uses the live, authenticated backend.
Added test_build_pipeline_uses_provided_backend to lock in the
injection contract.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
|||
| 9674e94efb |
Phase 2.4: LLM-summarized digest with master toggle filter
- Remove severity-based fork; tee pattern sends all events to both dispatcher and accumulator - Add ToggleFilter before tee; drops events for disabled toggles - Rework DigestAccumulator: event log instead of active/resolved tracking - render_digest now async, calls LLM once per toggle with severity-ordered events - Fallback to count-based summary when LLM unavailable - Add TogglesConfig to config.py for master toggle settings - Update scheduler to await async render_digest - 75 tests passing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
|||
| d6bc6b2b89 |
build: normalize all line endings to LF
One-time renormalization pass under the .gitattributes added in the previous commit. Every tracked text file now uses LF. No semantic changes — verified via git diff --cached --ignore-all-space showing zero real differences. Future diffs will only show real content changes. This commit will appear huge in git log --stat but represents zero behavior change. Use git log --follow --ignore-all-space or git blame -w when archaeologically tracing through this commit. |
|||
| 493b43f7cf |
feat(notifications): Phase 2.3b digest scheduler
Adds DigestScheduler class that fires digest at configured time (default 07:00) and routes to rules with trigger_type=schedule and schedule_match=digest. - DigestScheduler: asyncio task with start/stop lifecycle - Config: DigestConfig dataclass with schedule and include fields - Config: schedule_match field on NotificationRuleConfig - Pipeline: start_pipeline/stop_pipeline async lifecycle functions - Mesh channels get per-chunk delivery, email/webhook get full text - 26 new tests covering schedule computation, fire behavior, lifecycle Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
|||
| 8326fc56b2 | refactor(notifications): mesh chunk list and include_toggles |
