mirror of
https://github.com/zvx-echo6/meshai.git
synced 2026-06-10 17:04:45 +02:00
Third family of the v0.5.7 NATS-and-categories campaign. Fire is the heaviest of the campaign -- four distinct fixes plus a category audit. Two of the four were broken in production: FIRMS subscribed to a syntactically invalid pattern, and WFIGS tombstones were silently dropped.
FIX 1 -- FIRMS NATS pattern (the canonical bug). Pre-v0.5.7-fire `_subjects_for("firms","us.id")` returned `["central.fire.hotspot.>.us.id"]`, which is INVALID NATS (the `>` multi-level wildcard is only legal at the tail token). It also wouldn't have matched anything Central publishes: per the Central v0.10.0 consumer integration guide §firms, the actual published pattern is `central.fire.hotspot.<satellite>.<confidence>` (5 tokens, no us.<state> suffix). The two slots after "hotspot" are satellite name and confidence band -- NOT tile coordinates or region tokens.
Note on prompt vs. guide discrepancy: the v0.5.7-fire task spec described a tile-coord/state pattern `central.fire.hotspot.*.*.us.id` (7 tokens with us.<state> tail). That's neither what Central v0.10.0 publishes nor what its guide documents. We follow the guide. Subscribing to the prompt's 7-token pattern would silently match zero messages in production (token-count mismatch). State filtering for FIRMS happens client-side via data.latitude / data.longitude against the configured region bbox.
New subscription: `central.fire.hotspot.>` -- tail-only `>`, NATS-legal, matches all <satellite>.<confidence> combinations.
FIX 2 -- WFIGS tombstone subjects. Per guide §wfigs_incidents and §wfigs_perimeters, WFIGS publishes:
active: central.fire.incident.<state>.<county> (Convention A, depth-3 state)
active: central.fire.perimeter.<state>.<county>
tombstone: central.fire.incident.removed.<state> (5 tokens, "removed" at depth-3)
tombstone: central.fire.perimeter.removed.<state>
Pre-v0.5.7-fire `_subjects_for("fires","us.id")` subscribed only to the active subjects (`central.fire.incident.id.>` and `central.fire.perimeter.id.>`). The tombstone subjects have "removed" at depth-3 instead of the state token, so the active-subject `>` filters silently dropped EVERY tombstone. Fall-off signals never reached meshai's inhibitor, so old incidents stayed "live" in the pipeline indefinitely.
Added the two tombstone subjects to the subscription list. Both are 5-token literals with no wildcards -- trivially NATS-legal.
FIX 3 -- WFIGS tombstone dedup. Per guide §wfigs_incidents removal semantics, the tombstone env_id has the shape `<IrwinID>:removed:<iso_now>` -- the `:removed:` is sandwiched in the middle, with a timestamp tail. Pre-v0.5.7-fire the consumer.py group_key recovery was `re.sub(r":removed$", "", group_key)` -- a literal trailing `:removed` match -- which DID NOT FIRE on the WFIGS form (the regex required `:removed` at the very end of the string, but the WFIGS form has `:<iso>` after it).
Consequence: WFIGS tombstones' group_key was the full `<IrwinID>:removed:<iso>` string instead of the bare `<IrwinID>`. The pipeline grouper/inhibitor never matched tombstones to their original incidents, so the lapse signal was lost.
Fixed by switching the regex to `re.sub(r":removed(:.*)?$", "", group_key)` -- handles both the WFIGS `<IrwinID>:removed:<iso>` form AND the legacy GDACS `<id>:removed` form. The `is_tombstone` detection also gained an explicit `":removed:" in env_id` check for the WFIGS shape.
Per the guide: "the same incident can have one or more removal tombstones over its lifecycle" (it can re-enter and re-fall-off). To preserve per-tombstone distinctness for downstream lifecycle accounting, the full env_id is stashed on `Event.data["_central_tombstone_id"]` (the group_key collapses to the IrwinID by design, but the original env_id with the :<iso> tail survives on data).
FIX 4 -- ALERT_CATEGORIES fire-family audit + removed parametric entries. Per Matt's direct feedback ("fire near mesh has its own set of parameters that I don't even know what they could be. like how far is near mesh? I don't know I can't set that."), the parametric `fire_proximity` and the duplicate-named `wildfire_proximity` (both labeled "Fire Near Mesh" with parametric radius-based descriptions) were unselectable in the new Advanced Rules UI. Removed both.
Cross-referenced what FIRMS and WFIGS actually emit (per the guide and the native adapter code) and audited the registry:
Native emit:
firms.py -> new_ignition (when adapter flags new_ignition)
or wildfire_hotspot (otherwise) [v0.5.7-fire: was wildfire_proximity]
fires.py -> wildfire_incident
Central path emit (via map_category):
fire.hotspot.* -> wildfire_hotspot
fire.incident.* -> wildfire_incident
fire.perimeter.* -> wildfire_incident (perimeters merge to the incident)
fire.<other> -> wildfire_incident (catchall)
Registry after v0.5.7-fire:
{new_ignition, wildfire_hotspot, wildfire_incident}
Parity confirmed. No orphans, no missing.
Aligning firms.py to emit `wildfire_hotspot` (matching the central FIRMS map) means native + central FIRMS produce identical categories regardless of which feed path is enabled.
Composer (`_CATEGORY_EMOJI`, `_CATEGORY_LABEL`) and router (three source-attribution tables) updated to drop the removed categories and add the new ones.
Deferred to v0.5.8: distance_max_km field on rules for actual proximity filtering. Replaces the parametric fire_proximity registry entry with a parameterized rule field that the user CAN configure ("alert me about wildfire_incident within 30 km" instead of an opaque "Fire Near Mesh" toggle).
Tests
-----
PYTHONPATH=. pytest -q: 380 passed (was 366; +14 net).
- tests/test_fire_v057.py (new): FIRMS subject is tail-only `>` with no mid-subject placement; WFIGS subjects cover active + four tombstones; WFIGS tombstone strips `:removed(:.*)?$` for group_key; two same-IrwinID tombstones both propagate through _handle and share group_key, with the original env_id preserved on data["_central_tombstone_id"]; legacy GDACS `:removed` shape still strips cleanly; fire_proximity / wildfire_proximity absent from ALERT_CATEGORIES; no "Fire Near Mesh" name duplicates; fire-family parity (native + central emit == registry); required-fields check on the three fire entries.
- tests/test_central_region_routing.py: updated FIRMS test (tail-only `>`) and WFIGS test (includes tombstone subjects).
- tests/test_pipeline_toggle_filter.py, tests/test_adapter_firms.py, tests/test_v052_dispatcher.py, tests/test_pipeline_digest.py: bulk-migrated obsolete category references (wildfire_proximity -> wildfire_hotspot, fire_proximity -> wildfire_incident) so the existing test suites continue to exercise the same routing/digest/dispatch paths with the new category names.
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
212 lines
6.4 KiB
Python
212 lines
6.4 KiB
Python
"""Tests for FIRMS adapter Phase 2.6 — to_event() method."""
|
|
|
|
import time
|
|
from unittest.mock import MagicMock
|
|
|
|
import pytest
|
|
|
|
from meshai.env.firms import FIRMSAdapter
|
|
from meshai.notifications.events import Event
|
|
|
|
|
|
# ============================================================
|
|
# FIXTURES
|
|
# ============================================================
|
|
|
|
@pytest.fixture
|
|
def mock_config():
|
|
"""Create a mock FIRMSConfig."""
|
|
config = MagicMock()
|
|
config.map_key = "test-key"
|
|
config.source = "VIIRS_SNPP_NRT"
|
|
config.bbox = [-117, 42, -114, 44]
|
|
config.day_range = 1
|
|
config.tick_seconds = 1800
|
|
config.confidence_min = "nominal"
|
|
config.proximity_km = 10.0
|
|
return config
|
|
|
|
|
|
@pytest.fixture
|
|
def adapter(mock_config):
|
|
"""Create a FIRMSAdapter with mocked dependencies."""
|
|
return FIRMSAdapter(mock_config, region_anchors=[], fires_adapter=None)
|
|
|
|
|
|
def make_firms_event(
|
|
lat=42.5,
|
|
lon=-114.5,
|
|
new_ignition=False,
|
|
severity="routine",
|
|
headline="Test Hotspot",
|
|
frp=None,
|
|
confidence="n",
|
|
distance_km=None,
|
|
nearest_anchor=None,
|
|
near_fire=None,
|
|
):
|
|
"""Helper to create a FIRMS event dict."""
|
|
now = time.time()
|
|
return {
|
|
"source": "firms",
|
|
"event_id": f"firms_{lat:.4f}_{lon:.4f}_2026-05-15_1200",
|
|
"event_type": "Fire Hotspot",
|
|
"severity": severity,
|
|
"headline": headline,
|
|
"lat": lat,
|
|
"lon": lon,
|
|
"expires": now + 21600,
|
|
"fetched_at": now,
|
|
"properties": {
|
|
"new_ignition": new_ignition,
|
|
"confidence": confidence,
|
|
"frp": frp,
|
|
"brightness": 350.0,
|
|
"acq_date": "2026-05-15",
|
|
"acq_time": "1200",
|
|
"near_fire": near_fire,
|
|
"distance_to_fire_km": 5.0 if near_fire else None,
|
|
"distance_km": distance_km,
|
|
"nearest_anchor": nearest_anchor,
|
|
},
|
|
}
|
|
|
|
|
|
# ============================================================
|
|
# CATEGORY DECISION TESTS
|
|
# ============================================================
|
|
|
|
def test_to_event_new_ignition(adapter):
|
|
"""New ignition maps to new_ignition category."""
|
|
evt = make_firms_event(new_ignition=True)
|
|
event = adapter.to_event(evt)
|
|
assert event is not None
|
|
assert event.category == "new_ignition"
|
|
|
|
|
|
def test_to_event_near_known_fire(adapter):
|
|
"""Hotspot near known fire maps to wildfire_hotspot."""
|
|
evt = make_firms_event(new_ignition=False, near_fire="Snake River Fire")
|
|
event = adapter.to_event(evt)
|
|
assert event is not None
|
|
assert event.category == "wildfire_hotspot"
|
|
|
|
|
|
# ============================================================
|
|
# SEVERITY PASS-THROUGH TESTS
|
|
# ============================================================
|
|
|
|
def test_to_event_severity_passes_through(adapter):
|
|
"""Severity from FIRMS event passes through unchanged."""
|
|
for sev in ["routine", "priority", "immediate"]:
|
|
evt = make_firms_event(severity=sev)
|
|
event = adapter.to_event(evt)
|
|
assert event is not None
|
|
assert event.severity == sev
|
|
|
|
|
|
# ============================================================
|
|
# CONTENT TESTS
|
|
# ============================================================
|
|
|
|
def test_to_event_summary_includes_frp(adapter):
|
|
"""Summary includes FRP when present."""
|
|
evt = make_firms_event(frp=85.5)
|
|
event = adapter.to_event(evt)
|
|
assert event is not None
|
|
assert "FRP 85" in event.summary
|
|
|
|
|
|
def test_to_event_summary_handles_missing_frp(adapter):
|
|
"""Missing FRP doesn't break to_event."""
|
|
evt = make_firms_event(frp=None)
|
|
event = adapter.to_event(evt)
|
|
assert event is not None
|
|
assert "FRP" not in event.summary
|
|
|
|
|
|
def test_to_event_summary_includes_distance_when_present(adapter):
|
|
"""Summary includes distance and anchor when present."""
|
|
evt = make_firms_event(distance_km=12, nearest_anchor="TFL")
|
|
event = adapter.to_event(evt)
|
|
assert event is not None
|
|
assert "12 km" in event.summary
|
|
assert "TFL" in event.summary
|
|
|
|
|
|
def test_to_event_region_uses_nearest_anchor(adapter):
|
|
"""Region is set from nearest_anchor."""
|
|
evt = make_firms_event(nearest_anchor="MHR")
|
|
event = adapter.to_event(evt)
|
|
assert event is not None
|
|
assert event.region == "MHR"
|
|
|
|
|
|
# ============================================================
|
|
# SPATIAL KEY TESTS
|
|
# ============================================================
|
|
|
|
def test_to_event_group_key_is_spatial_grid(adapter):
|
|
"""Group key is spatial grid based on rounded lat/lon."""
|
|
evt = make_firms_event(lat=42.5678, lon=-114.3456)
|
|
event = adapter.to_event(evt)
|
|
assert event is not None
|
|
assert event.group_key == "firms:42.57:-114.35"
|
|
|
|
|
|
def test_to_event_inhibit_keys_match_group_key(adapter):
|
|
"""Inhibit keys contain the same spatial key as group_key."""
|
|
evt = make_firms_event(lat=42.5678, lon=-114.3456)
|
|
event = adapter.to_event(evt)
|
|
assert event is not None
|
|
assert event.group_key in event.inhibit_keys
|
|
|
|
|
|
def test_two_nearby_detections_share_group_key(adapter):
|
|
"""Two detections in same grid cell share group_key."""
|
|
# Both round to 42.57:-114.35
|
|
evt1 = make_firms_event(lat=42.571, lon=-114.351)
|
|
evt2 = make_firms_event(lat=42.572, lon=-114.352)
|
|
event1 = adapter.to_event(evt1)
|
|
event2 = adapter.to_event(evt2)
|
|
assert event1 is not None
|
|
assert event2 is not None
|
|
assert event1.group_key == event2.group_key
|
|
|
|
|
|
# ============================================================
|
|
# DEFENSIVE TESTS
|
|
# ============================================================
|
|
|
|
def test_to_event_missing_coords_returns_none(adapter):
|
|
"""Missing coordinates returns None."""
|
|
evt = make_firms_event()
|
|
evt["lat"] = None
|
|
event = adapter.to_event(evt)
|
|
assert event is None
|
|
|
|
|
|
def test_to_event_missing_properties_returns_event(adapter):
|
|
"""Missing properties dict defaults to wildfire_hotspot."""
|
|
evt = {
|
|
"source": "firms",
|
|
"event_id": "test",
|
|
"event_type": "Fire Hotspot",
|
|
"severity": "routine",
|
|
"headline": "Test",
|
|
"lat": 42.5,
|
|
"lon": -114.5,
|
|
"fetched_at": time.time(),
|
|
}
|
|
# No "properties" key at all
|
|
event = adapter.to_event(evt)
|
|
assert event is not None
|
|
assert event.category == "wildfire_hotspot"
|
|
|
|
|
|
def test_to_event_does_not_raise_on_corrupted_dict(adapter):
|
|
"""Corrupted dict returns None without raising."""
|
|
evt = {"garbage": True}
|
|
# Should not raise
|
|
event = adapter.to_event(evt)
|
|
assert event is None
|