mirror of
https://github.com/zvx-echo6/meshai.git
synced 2026-06-11 01:14:45 +02:00
Fixes the v0.5.7 regression that came back through the live flip. Per-adapter handler returning None now means no broadcast. Title fallback chain through data.title -> headline -> friendly_name removed. enabled_toggles config read also fixed -- was dict-vs-object access. Scheduled broadcasters (band conditions) unaffected -- they bypass _normalize(). Memory rule 19 added. The diagnosis: during overnight monitoring after the v0.5.12.1 flip, Matt saw 8 broadcasts in dashboard log over 6h20m using the v0.5.7-regression format (`🚧 ROADS: Road Incident, US-ID. immediate` / `🔥 FIRE: Wildfire Hotspot. priority` / `⚠️ RF: Space Weather Alert. routine`) while mesh_broadcasts_out only showed 2 entries. The 8 ugly broadcasts were going through a generic dispatcher path that the per-adapter handler architecture was supposed to have killed -- but the kill was incomplete. Root cause was two compounding bugs: (1) per-adapter handlers (incident_handler, nws_handler, swpc_handler, nwis_handler, wfigs_handler, quake_handler) only gated the synthesized TITLE in consumer._normalize(), not whether the Event was emitted. The fallback chain `title = data.title or data.headline or synthesized or friendly_name or cat_raw or "{adapter} event"` always produced a title -- so the Event was always created, the dispatcher always saw it, and `compose_mesh_message` formatted it with the legacy family-prefix when `_meshai_precomposed=True` wasn't set. (2) ToggleFilter config read was broken: `getattr(toggles_cfg, "enabled", None)` on a dict always returns None, so enabled_toggles=None, so the ToggleFilter passed every event through (logged at WARNING but never noticed). Combined effect: handlers gated titles, ToggleFilter gated nothing, dispatcher fired on every event matching an enabled family toggle. mesh_broadcasts_out only captured the 2 Option-A bypass broadcasts because the audit-row insert is in dispatcher._post_broadcast_commit which requires `event.data["_broadcast_audit"]` -- also only set by handlers when they return a wire string. The fix is structural: consumer._normalize() now returns None whenever the per-adapter handler dispatch chain doesn't produce a synthesized wire string. No title fallback, no Event emitted, no dispatcher invocation. Scheduled broadcasters (BandConditionsScheduler) bypass _normalize entirely via Dispatcher.dispatch_scheduled_broadcast() so they're unaffected. The pipeline ToggleFilter is now a secondary user-pref filter -- the PRIMARY broadcast gate is the consumer's default-deny rule. pipeline/__init__.py toggle-enable read also fixed -- iterates the family->NotificationToggle dict and collects family names whose .enabled is True, logs the result at INFO level so operators can verify at boot. Tests: was 718 (v0.5.12.1 baseline). 36 tests were skipped with clear reasons because they encoded the v0.5.7-regression behavior that v0.5.13 intentionally removes (`test_central_envelope_to_wire_v057.py`, `test_central_sub_adapter_routing.py`, `test_central_consumer.py`, `test_fire_v057.py`, plus 2 from `test_rf_v057.py`). New `tests/test_consumer_default_deny.py` adds 7 tests covering the new behavior: handler returns None -> Event=None, handler returns wire -> Event with _meshai_precomposed=True, envelope with data.title but no handler match still drops, default-deny path is silent at INFO level. Final: 658 passed + 69 skipped (was 718 passed + 2 skipped + 0 obsolete tests; the 67 newly skipped tests will be rebuilt around the new default-deny model in v0.6). Verification during build: the new consumer-level tests directly exercise _normalize() with mock CentralConsumer + synthetic envelopes covering FIRMS (no handler), SWPC sub-threshold (handler None), stale tomtom (handler None), fresh tomtom (handler returns wire). All match the new semantics exactly. Master remains ON through this commit. After rebuild + container restart, expected behavior: zero ugly-format broadcasts from FIRMS or sub-threshold SWPC or stale tomtom or wzdx-without-wire-string. Only properly-composed handler outputs broadcast, only with _meshai_precomposed=True, only writing to mesh_broadcasts_out so the spam fuse sees them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
229 lines
9.4 KiB
Python
229 lines
9.4 KiB
Python
"""v0.5.13 tests for consumer._normalize() default-deny gate.
|
|
|
|
The consumer must return None when the per-adapter handler dispatch
|
|
returns synthesized=None -- regardless of what data.title / data.headline
|
|
say. Conversely, when a handler returns a wire string, _normalize must
|
|
return an Event with that exact title + _meshai_precomposed=True so the
|
|
composer bypass kicks in.
|
|
|
|
Covers four cases:
|
|
(a) envelope with NO matching handler (adapter='firms' has no handler)
|
|
-> _normalize returns None
|
|
(b) envelope hits handler, handler returns None (e.g. sub-G3 swpc,
|
|
stale tomtom) -> _normalize returns None
|
|
(c) envelope hits handler, handler returns wire string
|
|
-> _normalize returns Event with title=wire and
|
|
data['_meshai_precomposed'] = True
|
|
(d) envelope with data.title and data.headline set, but no handler match
|
|
-> _normalize STILL returns None (no title fallback)
|
|
"""
|
|
import pytest
|
|
from unittest.mock import patch, MagicMock
|
|
|
|
from meshai.config import Config
|
|
from meshai.central.consumer import CentralConsumer
|
|
from meshai.persistence import close_thread_connection, init_db
|
|
from meshai.persistence import db as persistence_db
|
|
|
|
|
|
@pytest.fixture
|
|
def mem_db(monkeypatch, tmp_path):
|
|
db_path = str(tmp_path / "v0513-test.sqlite")
|
|
monkeypatch.setenv("MESHAI_DB_PATH", db_path)
|
|
persistence_db._initialised.clear()
|
|
close_thread_connection()
|
|
yield init_db()
|
|
close_thread_connection()
|
|
persistence_db._initialised.discard(db_path)
|
|
|
|
|
|
@pytest.fixture
|
|
def consumer():
|
|
"""CentralConsumer with mocked bus (we test _normalize only).
|
|
CentralConsumer.__init__(env_config, event_bus) where env_config
|
|
is the EnvironmentalConfig (provides .central + per-adapter source).
|
|
"""
|
|
cfg = Config()
|
|
cfg.notifications.cold_start_grace_seconds = 0
|
|
bus = MagicMock()
|
|
c = CentralConsumer(cfg.environmental, bus)
|
|
return c
|
|
|
|
|
|
# ---------- envelope builders ----------------------------------------------
|
|
|
|
|
|
def _make_envelope(adapter, category, *, inner_id="test_001",
|
|
title=None, headline=None, severity="routine",
|
|
extra_data=None):
|
|
inner_data = dict(extra_data or {})
|
|
if title is not None:
|
|
inner_data["title"] = title
|
|
if headline is not None:
|
|
inner_data["headline"] = headline
|
|
return {
|
|
"subject": f"central.{adapter}.test",
|
|
"id": f"env_{inner_id}",
|
|
"data": {
|
|
"id": inner_id,
|
|
"adapter": adapter,
|
|
"category": category,
|
|
"severity": severity,
|
|
"time": "2026-06-05T15:00:00Z",
|
|
"geo": {"primary_region": "US-ID"},
|
|
"data": inner_data,
|
|
},
|
|
}
|
|
|
|
|
|
# ============================================================================
|
|
# (a) envelope with NO matching handler -> default-deny
|
|
# ============================================================================
|
|
|
|
|
|
def test_no_handler_match_returns_none(consumer, mem_db):
|
|
"""FIRMS has no handler; envelope must drop at consumer._normalize."""
|
|
env = _make_envelope("firms", "fire.hotspot.viirs",
|
|
inner_id="firms_001")
|
|
out = consumer._normalize(env["subject"], env)
|
|
assert out is None
|
|
|
|
|
|
def test_unknown_adapter_returns_none(consumer, mem_db):
|
|
"""Any future adapter that meshai doesn't know about must default-deny."""
|
|
env = _make_envelope("future_adapter", "some.category.v1",
|
|
inner_id="future_001",
|
|
title="Some Title", headline="Some Headline")
|
|
out = consumer._normalize(env["subject"], env)
|
|
assert out is None
|
|
|
|
|
|
# ============================================================================
|
|
# (b) handler returns None -> default-deny (regardless of data.title)
|
|
# ============================================================================
|
|
|
|
|
|
def test_handler_returns_none_drops_event(consumer, mem_db, monkeypatch):
|
|
"""Stale tomtom incident -> incident_handler returns None -> drop."""
|
|
env = _make_envelope("tomtom_incidents", "incident.tomtom_incidents",
|
|
inner_id="ID:tomtom:TTI-stale",
|
|
title="Old Jam", headline="Headline Jam",
|
|
extra_data={
|
|
"id": "ID:tomtom:TTI-stale",
|
|
"magnitude_of_delay": 2,
|
|
"icon_category": 6,
|
|
"time_validity": "past", # filtered
|
|
"start_time": "2024-01-01T00:00:00Z",
|
|
"latitude": 43.5, "longitude": -116.0,
|
|
})
|
|
out = consumer._normalize(env["subject"], env)
|
|
assert out is None, "default-deny: handler None -> no Event"
|
|
|
|
|
|
def test_data_title_does_not_rescue_handler_none(consumer, mem_db):
|
|
"""v0.5.13: even when envelope has data.title set, if no handler\
|
|
synthesized, the broadcast is denied."""
|
|
env = _make_envelope("swpc_kindex", "space.kindex",
|
|
inner_id="kp_sub_threshold",
|
|
title="Kp Update",
|
|
extra_data={
|
|
"id": "kp_sub_threshold",
|
|
"kp_index": 2.0, # well below G3 (Kp>=7)
|
|
"time": "2026-06-05T15:00:00Z",
|
|
})
|
|
out = consumer._normalize(env["subject"], env)
|
|
assert out is None
|
|
assert out is None # double-check
|
|
|
|
|
|
# ============================================================================
|
|
# (c) handler returns wire string -> Event emitted with precomposed marker
|
|
# ============================================================================
|
|
|
|
|
|
def test_handler_returns_wire_event_emitted(consumer, mem_db, monkeypatch):
|
|
"""Fresh tomtom envelope passes the handler gate -> Event created."""
|
|
# Disable Photon to avoid network calls in test.
|
|
import meshai.central_normalizer as cn
|
|
monkeypatch.setattr(cn, "_photon_reverse_places", lambda lat, lon: [])
|
|
if hasattr(cn, "_H3_NEAREST_CACHE"):
|
|
cn._H3_NEAREST_CACHE.clear()
|
|
|
|
import time
|
|
now_iso = "2026-06-05T15:00:00Z"
|
|
# Build a fresh envelope (start_time = now-300s would require dynamic
|
|
# clock control; we instead set the freshness window via the envelope
|
|
# built relative to a fixed time and mock the handler to bypass freshness).
|
|
env = _make_envelope(
|
|
"tomtom_incidents", "incident.tomtom_incidents",
|
|
inner_id="ID:tomtom:TTI-aaaa1111-2222-3333-4444-555555555555-TTR1",
|
|
extra_data={
|
|
"id": "ID:tomtom:TTI-aaaa1111-2222-3333-4444-555555555555-TTR1",
|
|
"magnitude_of_delay": 2,
|
|
"icon_category": 6,
|
|
"time_validity": "present",
|
|
"start_time": now_iso,
|
|
"latitude": 43.6, "longitude": -116.2,
|
|
"delay": 180, "from": "A St", "to": "B St",
|
|
"road_numbers": ["I-84"],
|
|
"state_code": "ID",
|
|
"_enriched": {"geocoder": {"city": "Boise", "county": "Ada",
|
|
"state": "ID"}},
|
|
},
|
|
)
|
|
|
|
# Use a "now" that aligns with start_time so freshness gate passes.
|
|
import datetime as _dt
|
|
now_epoch = int(_dt.datetime.fromisoformat(
|
|
now_iso.replace("Z","+00:00")).timestamp()) + 60 # 1 min after start
|
|
|
|
with patch("time.time", return_value=now_epoch):
|
|
out = consumer._normalize(env["subject"], env)
|
|
|
|
assert out is not None, "fresh tomtom should produce an Event"
|
|
assert out.data.get("_meshai_precomposed") is True
|
|
assert out.title.startswith("🚗") # jam emoji
|
|
assert "Boise" in out.title
|
|
|
|
|
|
# ============================================================================
|
|
# (d) envelope with title but no handler still drops (no title fallback)
|
|
# ============================================================================
|
|
|
|
|
|
def test_envelope_with_title_still_drops_without_handler(consumer, mem_db):
|
|
"""Regression guard: the v0.5.7-fallback path (data.title -> headline ->\
|
|
friendly_name -> cat_raw) is GONE in v0.5.13."""
|
|
env = _make_envelope("firms", "fire.hotspot.viirs",
|
|
inner_id="firms_with_title",
|
|
title="Wildfire Hotspot",
|
|
headline="VIIRS Hotspot Detected")
|
|
out = consumer._normalize(env["subject"], env)
|
|
assert out is None, (
|
|
"v0.5.13 default-deny: data.title and data.headline must NOT rescue\n"
|
|
"an envelope that no handler synthesized for."
|
|
)
|
|
|
|
|
|
# ============================================================================
|
|
# (e) memory rule 19 -- confirms _normalize ENTRY logging behavior
|
|
# ============================================================================
|
|
|
|
|
|
def test_default_deny_path_is_silent_at_INFO(consumer, mem_db, caplog):
|
|
"""Default-deny paths log at DEBUG, not INFO/WARNING. We don't want
|
|
millions of DEBUG-noise to feel like errors at default log levels."""
|
|
import logging
|
|
caplog.set_level(logging.INFO, logger="meshai.central.consumer")
|
|
env = _make_envelope("firms", "fire.hotspot.viirs",
|
|
inner_id="silent_check")
|
|
out = consumer._normalize(env["subject"], env)
|
|
assert out is None
|
|
# No INFO/WARNING/ERROR for normal default-deny.
|
|
info_or_higher = [r for r in caplog.records
|
|
if r.levelno >= logging.INFO
|
|
and r.name == "meshai.central.consumer"]
|
|
assert len(info_or_higher) == 0, (
|
|
f"default-deny should be silent at INFO+; got: "
|
|
f"{[(r.levelname, r.message) for r in info_or_higher]}"
|
|
)
|