meshai/tests/test_nws_dedup_relaxation.py

140 lines
4.7 KiB
Python
Raw Normal View History

feat(v0.6-phase3): reminder system + schema split + NWS dedup relaxation Third broadcast type Active: clock-driven re-broadcasts of still-live events at human-scale cadences. WFIGS fires 8h, itd_511 work zones daily 8 AM Mountain, SWPC G-storms 8h. NWS is NOT a clock reminder -- instead the per-CAP-id dedup is relaxed to allow re-broadcast if >3h since last. Schema split first_broadcast_at + last_broadcast_at on all reminder- eligible tables. Wire prefix logic: New (first sight), Update (WFIGS material change), Active (clock reminder). All cadences, channels, day- of-week patterns, timezones, and termination conditions GUI-editable from day one via the existing adapter_config editor. Termination: tombstone OR containment_100 OR end_date_passed (no max-count). Quiet hours not respected -- ripped out in Phase 2. Schema (v11.sql): - ALTER TABLE fires|nws_alerts|traffic_events|quake_events|swpc_events| gauge_readings ADD COLUMN first_broadcast_at REAL - Backfill: UPDATE ... SET first_broadcast_at = last_broadcast_at WHERE last_broadcast_at IS NOT NULL - ALTER TABLE adapter_meta ADD COLUMN reminder_enabled INTEGER NOT NULL DEFAULT 0 - UPDATE adapter_meta SET reminder_enabled=1 WHERE adapter IN ('wfigs', 'swpc') -- itd_511_work_zone is a new meta row seeded with reminder_enabled=1 - SCHEMA_VERSION 10 -> 11 Handler commit-callbacks (wfigs/nws/quake/swpc/incident): - UPDATE ... SET last_broadcast_at=?, first_broadcast_at=COALESCE( first_broadcast_at, ?) -- first_broadcast_at stamped once, never overwritten NWS handler (meshai/central/nws_handler.py): - _render() gains a prefix kwarg - After-first-broadcast branch: when (now - last_broadcast_at) >= adapter_config.nws.duplicate_allowed_after_seconds (default 10800 = 3h), allow the re-broadcast with prefix=Active. Under the window, suppress as before. The commit callback continues to update last_broadcast_at. ReminderScheduler (meshai/notifications/reminders/__init__.py): - Async loop, ticks every 60s - Each tick: SELECT adapter FROM adapter_meta WHERE reminder_enabled=1 - Per adapter, load reminders_<adapter> config from adapter_config (cadence_kind, cadence_value, channels, terminate_when, dow_mask, timezone) - Interval cadence: rows where last_broadcast_at <= now - cadence_value - Clock cadence: localizes now to configured tz, finds slots that just passed in the last tick window, gated by dow_mask - Termination conditions checked per adapter: wfigs.containment_100 -> current_contained_pct >= 100 wfigs.last_event_age_24h -> last_event_at older than 24h swpc.end_date_passed -> payload_json end_time in past itd_511_work_zone.end_date_passed -> traffic_events.end_at in past - Active: prefix on every emitted wire; dispatcher.dispatch_scheduled_ broadcast() honors cold-start grace, bypasses toggle path - On success, last_broadcast_at = now; first_broadcast_at preserved Launched from notifications/pipeline/__init__.py:start_pipeline() alongside BandConditionsScheduler. adapter_config registry (+15 new keys, 43 -> 58): - reminders_wfigs.cadence_kind/cadence_value/channels/terminate_when - reminders_swpc.cadence_kind/cadence_value/channels/terminate_when - reminders_itd_511_work_zone.cadence_kind/cadence_value/channels/ dow_mask/timezone/terminate_when - nws.duplicate_allowed_after_seconds adapter_meta (+4 rows, 15 -> 19): - reminders_wfigs, reminders_swpc, reminders_itd_511_work_zone (pseudo-adapters carrying the reminder config) - itd_511_work_zone (reminder target row; reminder_enabled=1) - reminder_enabled flag added to wfigs/swpc (existing rows updated by v11.sql) and to itd_511_work_zone seed. Tests (tests/test_reminders.py, 10 cases): - wfigs reminder fires past 8h cadence, stamps last_broadcast_at, preserves first_broadcast_at - reminder skipped within cadence - reminder skipped when containment_100, last_event_age_24h - swpc reminder fires (interval) - work_zone clock reminder fires at 08:00 Mountain on enabled DOW - work_zone reminder skipped when end_date_passed - work_zone reminder skipped outside slot window - reminder_enabled=0 suppresses all reminders for that adapter tests/test_nws_dedup_relaxation.py (5 cases): - First sighting renders without Active: prefix - Re-broadcast within 3h suppressed - Re-broadcast after 3h allowed with Active: prefix - adapter_config.nws.duplicate_allowed_after_seconds override takes effect (1h window verified) - First sighting stamps first_broadcast_at=committed_at, last_broadcast_at=committed_at; 4h later broadcast stamps last_broadcast_at only, first_broadcast_at preserved Test count: 829 -> 844 (+15 new, 0 regressions). Foundation tests updated for new counts (REGISTRY=58, ADAPTER_META=19, schema=v11).
2026-06-05 21:11:32 +00:00
"""v0.6-phase3 NWS dedup-window relaxation tests.
The same CAP id is now re-broadcast (with Active: prefix) when more than
`nws.duplicate_allowed_after_seconds` (default 10800 = 3h) have elapsed
since the last broadcast.
"""
from __future__ import annotations
import time
import pytest
from meshai.central.nws_handler import handle_nws
from meshai.persistence import get_db
def _env(*, cap_id="urn:oid:dedup.001", event="Severe Thunderstorm Warning",
severity="Severe", area="Ada County", county="Ada", state="ID",
expires="2026-06-05T03:00:00Z", lat=43.6, lon=-116.2,
category="wx.alert.severe_thunderstorm_warning"):
return {
"id": cap_id, "subject": "central.wx.alert.us.id",
"data": {
"id": cap_id, "adapter": "nws", "category": category,
"severity": 2,
"geo": {"centroid": [lon, lat], "primary_region": "US-ID"},
"data": {
"id": cap_id, "event": event, "severity": severity,
"areaDesc": area, "msgType": "Alert",
"headline": f"{event} for {area}",
"description": "X", "expires": expires,
"_enriched": {"geocoder": {"city": None,
"county": county, "state": state}},
},
},
}
def _commit(data, ts):
cb = data["_on_broadcast_committed"]
cb(float(ts))
def test_first_broadcast_no_active_prefix():
"""A first sighting renders without Active: prefix."""
env = _env()
data = {}
wire = handle_nws(env, env["subject"], data=data, now=1_000_000)
assert wire is not None
assert "Active:" not in wire
def test_repeat_within_3h_suppressed():
env = _env(cap_id="urn:oid:rep1")
data = {}
# First broadcast at t=0.
wire1 = handle_nws(env, env["subject"], data=data, now=0)
assert wire1 is not None
_commit(data, 0)
# Same CAP id again 2h later -- inside 3h window -> suppressed.
env2 = _env(cap_id="urn:oid:rep1")
data2 = {}
wire2 = handle_nws(env2, env2["subject"], data=data2, now=2 * 3600)
assert wire2 is None
def test_repeat_after_3h_allowed_with_active_prefix():
env = _env(cap_id="urn:oid:rep2")
data = {}
wire1 = handle_nws(env, env["subject"], data=data, now=0)
assert wire1 is not None
_commit(data, 0)
# Same CAP id 4h later -> allowed, Active: prefix.
env2 = _env(cap_id="urn:oid:rep2")
data2 = {}
wire2 = handle_nws(env2, env2["subject"], data=data2, now=4 * 3600)
assert wire2 is not None
assert "Active:" in wire2
def test_dedup_window_respects_config_override():
"""Changing nws.duplicate_allowed_after_seconds via adapter_config takes effect."""
from meshai.adapter_config import invalidate_cache
conn = get_db()
conn.execute(
"UPDATE adapter_config SET value_json='3600' "
"WHERE adapter='nws' AND key='duplicate_allowed_after_seconds'"
)
invalidate_cache()
env = _env(cap_id="urn:oid:tunable")
data = {}
handle_nws(env, env["subject"], data=data, now=0)
_commit(data, 0)
# 90 min later -> still suppressed (under new 1h window).
# Wait, 90 min > 60 min so it would BE allowed. Use 30 min instead.
env2 = _env(cap_id="urn:oid:tunable")
data2 = {}
wire = handle_nws(env2, env2["subject"], data=data2, now=30 * 60)
assert wire is None # 30 min < 60 min window
# 2h later -> allowed (over 1h window now).
env3 = _env(cap_id="urn:oid:tunable")
data3 = {}
wire3 = handle_nws(env3, env3["subject"], data=data3, now=2 * 3600)
assert wire3 is not None
assert "Active:" in wire3
def test_handler_stamps_first_broadcast_at():
"""The commit callback writes first_broadcast_at via COALESCE -- only on
the first commit, never overwriting it."""
env = _env(cap_id="urn:oid:stamp")
data = {}
handle_nws(env, env["subject"], data=data, now=0)
_commit(data, 100.0)
conn = get_db()
row = conn.execute(
"SELECT first_broadcast_at, last_broadcast_at FROM nws_alerts "
"WHERE event_id='urn:oid:stamp'"
).fetchone()
assert row["first_broadcast_at"] == 100.0
assert row["last_broadcast_at"] == 100.0
# Second broadcast 4h later -> last_broadcast_at updates, first_broadcast_at preserved.
env2 = _env(cap_id="urn:oid:stamp")
data2 = {}
wire2 = handle_nws(env2, env2["subject"], data=data2, now=4 * 3600)
assert wire2 is not None
_commit(data2, 4 * 3600.0)
row2 = conn.execute(
"SELECT first_broadcast_at, last_broadcast_at FROM nws_alerts "
"WHERE event_id='urn:oid:stamp'"
).fetchone()
assert row2["first_broadcast_at"] == 100.0 # unchanged
assert row2["last_broadcast_at"] == 4 * 3600.0