mirror of
https://github.com/zvx-echo6/meshai.git
synced 2026-08-26 17:31:34 +00:00
Triaged all 20 known-red tests. 13 were stale tests asserting rotted expectations against deliberate, documented behavior changes; fixed by deriving expected values instead of hard-coding, or updating the expectation to match a documented policy change: - test_adapter_config_foundation.py / test_adapter_config_api.py: REGISTRY/API key-count and key-set guards hard-coded magic numbers (59/94/17) that rotted repeatedly. Now derive expectations from REGISTRY itself and, for the schema version, from the migrations directory, so they can't rot the same way again. - test_fire_tracker_phase4.py: two tests hardcoded a nonexistent deployment path (/opt/meshai/meshai/router.py) that matches no Dockerfile WORKDIR in this repo; resolve the module path via importlib.util.find_spec instead. - test_tombstone_broadcast.py: asserted fire severity == "immediate", which commit2f677e85deliberately downgraded to "priority" (to stop fire broadcasts bypassing the Grouper/cooldown during NATS backlog replay) without updating this test. - test_pipeline_grouper.py: test_immediate_severity_bypasses_grouper asserted an immediate-severity bypass that commit85d48ce3("fix(fire): remove immediate-severity exemption from grouper + cooldown") DELETED on purpose -- fire events carry _severity_override="immediate", and the exemption left fire with no rate control at all in normal live operation. Re-adding the bypass would re-open that fire-spam hole on a public-safety mesh, so the test moves, not the source. Renamed + inverted to assert the real contract (all severities coalesce; only a missing group_key passes through). - test_tail_followups.py: dispatcher mock was missing dispatch_scheduled_fire_broadcast (a method added alongside the generic dispatch_scheduled_broadcast; test_reminders.py already mocks both). - test_tracking_v057.py: guard required an empty tracking-family adapter list in Environment.tsx, but the frontend has long grouped the pre-existing native satpass adapter under the "Tracking" display section (its own "satpass" backend toggle, unrelated to the Phase-7 tracking family every other guard in this file confirms is still unimplemented). Narrowed the guard to allow only that known entry. - test_v052_dispatcher.py: two tests used category="wildfire_incident", which the phase3b fire migration (#33) forced onto a dedicated formatter via NATIVE_ALWAYS_DECIDE; swapped to wildfire_hotspot (same emoji/label, not in NATIVE_ALWAYS_DECIDE) to keep exercising the generic composer logic under test. Also fixes one stale COMMENT (comment-only, no logic change) in meshai/notifications/pipeline/__init__.py's start_pipeline(): it still claimed "Immediate events bypass the grouper and don't need this [periodic flush]", which has been false since85d48ce3and is precisely what makes the deleted bypass look like a missing feature. The comment now records that the removal was deliberate and must not be reverted. The remaining 6 failures are left untouched -- 2 confirmed real bugs, to be fixed deliberately in their own changes: - meshcore_transport.py defines `_resolve_contact` TWICE on MeshCoreTransport (line 171 from PR #56, line 1227 from PR #92). The second silently shadows the first, so the DM contact-resolution refetch-on-miss that #56 added is dead code in production. (3 tests) - SCHEMA_VERSION (persistence/db.py:33) is stale at 26 vs. the actual highest migration v28; v27 and v28 shipped without bumping it. (3 tests) Plus 1 environment gap, not a code defect: test_natural_language_fire_question_routes_to_llm needs the `openai` package, which is declared in requirements.txt but not installed here. Suite: 20 failed, 2240 passed, 72 skipped -> 7 failed, 2254 passed, 72 skipped. All 7 remaining failures are ones classified above; no new failures introduced elsewhere in the suite. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
285 lines
12 KiB
Python
285 lines
12 KiB
Python
"""Tests for tombstone broadcast path fix.
|
|
|
|
Validates:
|
|
T1: tombstone yields _severity_override="priority" + commit handles
|
|
T2: closure wire dispatches when New was broadcast >=10min earlier
|
|
T3: build_env_summary excludes tombstoned and 100%-contained fires
|
|
|
|
Severity note: fire broadcasts (new/update/tombstone-closure alike) were
|
|
downgraded from "immediate" to "priority" by commit 2f677e85
|
|
("fix(fire): drain-mode pacer to prevent post-reconnect broadcast spam").
|
|
After a NATS consumer outage, LAST_PER_SUBJECT delivery could flood
|
|
thousands of backlogged events at once; "immediate" severity bypassed the
|
|
Grouper and zeroed dispatcher cooldowns, so a backlog replay produced
|
|
duplicate "New" broadcasts for the same fire. "priority" routes fire
|
|
broadcasts back through the normal pipeline guards (Grouper, cooldown).
|
|
This file's expectations were written before that downgrade and never
|
|
updated -- "immediate" here would be reverting a deliberate, documented
|
|
incident fix.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import time
|
|
|
|
import pytest
|
|
|
|
from meshai.central.wfigs_handler import handle_wfigs
|
|
from meshai.notifications.env_reporter import EnvReporter
|
|
from meshai.persistence import get_db
|
|
|
|
|
|
@pytest.fixture
|
|
def reporter():
|
|
return EnvReporter()
|
|
|
|
|
|
def _seed_fire(conn, *, irwin_id, name, acres, contained=None,
|
|
last_broadcast_at=None, last_event_at=None,
|
|
tombstoned_at=None, county="Ada", state="ID"):
|
|
now = int(time.time())
|
|
conn.execute(
|
|
"INSERT OR REPLACE INTO fires(irwin_id, incident_name, incident_type, "
|
|
"current_acres, current_contained_pct, lat, lon, county, state, "
|
|
"declared_at, last_event_at, last_broadcast_at, tombstoned_at) "
|
|
"VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)",
|
|
(irwin_id, name, "WF", acres, contained, 43.6, -116.2,
|
|
county, state, now, last_event_at or now,
|
|
last_broadcast_at, tombstoned_at),
|
|
)
|
|
|
|
|
|
class TestTombstoneSeverityAndCommitHandles:
|
|
"""T1: tombstone branch sets priority severity and attaches commit handles."""
|
|
|
|
def test_severity_is_priority(self):
|
|
conn = get_db()
|
|
now = int(time.time())
|
|
_seed_fire(conn, irwin_id="FIRE-001", name="Test Fire",
|
|
acres=500, contained=80,
|
|
last_broadcast_at=now - 3600)
|
|
|
|
data = {}
|
|
wire = handle_wfigs(
|
|
normalized={"_kind": "wfigs_tombstone", "irwin_id": "FIRE-001"},
|
|
envelope={"data": {"category": "wildfire", "severity": "immediate"}},
|
|
subject="wfigs.tombstone",
|
|
data=data,
|
|
now=now,
|
|
)
|
|
assert wire is not None, "tombstone should produce wire for previously-broadcast fire"
|
|
# See module docstring: fire severity was deliberately downgraded
|
|
# from "immediate" to "priority" (commit 2f677e85) so fire
|
|
# broadcasts flow through the normal Grouper/cooldown guards.
|
|
assert data.get("_severity_override") == "priority", (
|
|
f"expected priority, got {data.get('_severity_override')}")
|
|
|
|
def test_commit_handles_attached(self):
|
|
conn = get_db()
|
|
now = int(time.time())
|
|
_seed_fire(conn, irwin_id="FIRE-002", name="Handled Fire",
|
|
acres=1000, contained=95,
|
|
last_broadcast_at=now - 7200)
|
|
|
|
data = {}
|
|
wire = handle_wfigs(
|
|
normalized={"_kind": "wfigs_tombstone", "irwin_id": "FIRE-002"},
|
|
envelope={"data": {"category": "wildfire", "severity": "immediate"}},
|
|
subject="wfigs.tombstone",
|
|
data=data,
|
|
now=now,
|
|
)
|
|
assert wire is not None
|
|
assert "_on_broadcast_committed" in data, "commit callback missing"
|
|
assert "_broadcast_audit" in data, "broadcast audit descriptor missing"
|
|
assert "_cooldown_suffix" in data, "cooldown suffix missing"
|
|
assert data["_cooldown_suffix"] == "FIRE-002"
|
|
|
|
def test_dedup_suffix_is_closed(self):
|
|
conn = get_db()
|
|
now = int(time.time())
|
|
_seed_fire(conn, irwin_id="FIRE-003", name="Dedup Fire",
|
|
acres=200, contained=100,
|
|
last_broadcast_at=now - 600)
|
|
|
|
data = {}
|
|
wire = handle_wfigs(
|
|
normalized={"_kind": "wfigs_tombstone", "irwin_id": "FIRE-003"},
|
|
envelope={"data": {"category": "wildfire", "severity": "immediate"}},
|
|
subject="wfigs.tombstone",
|
|
data=data,
|
|
now=now,
|
|
)
|
|
assert wire is not None
|
|
assert data.get("_dedup_suffix") == "closed", (
|
|
f"expected 'closed', got {data.get('_dedup_suffix')}")
|
|
|
|
def test_commit_callback_flips_handled(self):
|
|
"""The commit callback should flip event_log.handled to 1."""
|
|
conn = get_db()
|
|
now = int(time.time())
|
|
_seed_fire(conn, irwin_id="FIRE-004", name="Callback Fire",
|
|
acres=300, contained=50,
|
|
last_broadcast_at=now - 1800)
|
|
|
|
data = {}
|
|
wire = handle_wfigs(
|
|
normalized={"_kind": "wfigs_tombstone", "irwin_id": "FIRE-004"},
|
|
envelope={"data": {"category": "wildfire", "severity": "immediate"}},
|
|
subject="wfigs.tombstone",
|
|
data=data,
|
|
now=now,
|
|
)
|
|
assert wire is not None
|
|
assert "_on_broadcast_committed" in data
|
|
|
|
# Before commit: event_log row should have handled=0
|
|
row_before = conn.execute(
|
|
"SELECT id, handled FROM event_log WHERE event_id_external='FIRE-004' "
|
|
"ORDER BY id DESC LIMIT 1"
|
|
).fetchone()
|
|
assert row_before is not None, "event_log row should exist"
|
|
assert row_before["handled"] == 0, "should be unhandled before commit"
|
|
|
|
# Fire the commit callback
|
|
data["_on_broadcast_committed"](float(now))
|
|
|
|
# After commit: handled should be 1
|
|
row_after = conn.execute(
|
|
"SELECT handled FROM event_log WHERE id=?",
|
|
(row_before["id"],)
|
|
).fetchone()
|
|
assert row_after["handled"] == 1, "should be handled=1 after commit callback"
|
|
|
|
|
|
class TestTombstoneAfterNewBroadcast:
|
|
"""T2: closure dispatches when a New broadcast went out earlier."""
|
|
|
|
def test_closure_wire_after_prior_broadcast(self):
|
|
"""Fire that was broadcast 10 min ago gets a closure wire on tombstone."""
|
|
conn = get_db()
|
|
now = int(time.time())
|
|
_seed_fire(conn, irwin_id="IA-1", name="IA 1",
|
|
acres=0.1, contained=None,
|
|
last_broadcast_at=now - 600, # 10 min ago
|
|
last_event_at=now - 600)
|
|
|
|
data = {}
|
|
wire = handle_wfigs(
|
|
normalized={"_kind": "wfigs_tombstone", "irwin_id": "IA-1"},
|
|
envelope={"data": {"category": "wildfire", "severity": "immediate"}},
|
|
subject="wfigs.tombstone",
|
|
data=data,
|
|
now=now,
|
|
)
|
|
assert wire is not None, "tombstone should produce wire"
|
|
assert "✅" in wire, "closure wire should contain checkmark"
|
|
assert "IA 1" in wire, "closure wire should name the fire"
|
|
assert data["category"] == "wildfire_closed"
|
|
# See module docstring: downgraded from "immediate" to "priority"
|
|
# by commit 2f677e85 to prevent Grouper/cooldown bypass.
|
|
assert data["_severity_override"] == "priority"
|
|
assert callable(data.get("_on_broadcast_committed"))
|
|
|
|
def test_no_wire_when_never_broadcast(self):
|
|
"""Fire that was never broadcast should NOT get a closure wire."""
|
|
conn = get_db()
|
|
now = int(time.time())
|
|
_seed_fire(conn, irwin_id="SILENT-1", name="Silent Fire",
|
|
acres=5, contained=None,
|
|
last_broadcast_at=None, # never broadcast
|
|
last_event_at=now - 600)
|
|
|
|
data = {}
|
|
wire = handle_wfigs(
|
|
normalized={"_kind": "wfigs_tombstone", "irwin_id": "SILENT-1"},
|
|
envelope={"data": {"category": "wildfire", "severity": "immediate"}},
|
|
subject="wfigs.tombstone",
|
|
data=data,
|
|
now=now,
|
|
)
|
|
assert wire is None, "no closure wire for never-broadcast fire"
|
|
|
|
def test_tombstoned_at_stamped(self):
|
|
"""Tombstone should stamp tombstoned_at on the fires row."""
|
|
conn = get_db()
|
|
now = int(time.time())
|
|
_seed_fire(conn, irwin_id="STAMP-1", name="Stamp Fire",
|
|
acres=10, contained=50,
|
|
last_broadcast_at=now - 3600)
|
|
|
|
handle_wfigs(
|
|
normalized={"_kind": "wfigs_tombstone", "irwin_id": "STAMP-1"},
|
|
envelope={"data": {"category": "wildfire", "severity": "immediate"}},
|
|
subject="wfigs.tombstone",
|
|
data={},
|
|
now=now,
|
|
)
|
|
row = conn.execute(
|
|
"SELECT tombstoned_at FROM fires WHERE irwin_id='STAMP-1'"
|
|
).fetchone()
|
|
assert row is not None
|
|
assert row["tombstoned_at"] == now
|
|
|
|
|
|
class TestEnvSummaryExcludesContainedTombstoned:
|
|
"""T3: build_env_summary fire count excludes tombstoned and 100%-contained."""
|
|
|
|
def test_summary_excludes_tombstoned(self, reporter):
|
|
conn = get_db()
|
|
now = int(time.time())
|
|
# Active fire
|
|
_seed_fire(conn, irwin_id="ACTIVE-1", name="Active Fire",
|
|
acres=500, contained=20,
|
|
last_event_at=now - 3600)
|
|
# Tombstoned fire
|
|
_seed_fire(conn, irwin_id="DEAD-1", name="Dead Fire",
|
|
acres=1000, contained=100,
|
|
last_event_at=now - 7200,
|
|
tombstoned_at=now - 3600)
|
|
text = reporter.build_env_summary(now=now)
|
|
assert "Active fires (WFIGS, last 7d): 1" in text, (
|
|
f"should count 1 active fire, got: {text}")
|
|
|
|
def test_summary_excludes_100_contained(self, reporter):
|
|
conn = get_db()
|
|
now = int(time.time())
|
|
# Active fire, 50% contained
|
|
_seed_fire(conn, irwin_id="HALF-1", name="Half Contained",
|
|
acres=300, contained=50,
|
|
last_event_at=now - 3600)
|
|
# 100% contained, NOT tombstoned
|
|
_seed_fire(conn, irwin_id="FULL-1", name="Fully Contained",
|
|
acres=800, contained=100,
|
|
last_event_at=now - 7200)
|
|
text = reporter.build_env_summary(now=now)
|
|
assert "Active fires (WFIGS, last 7d): 1" in text, (
|
|
f"should count 1 active fire, got: {text}")
|
|
|
|
def test_summary_includes_null_containment(self, reporter):
|
|
conn = get_db()
|
|
now = int(time.time())
|
|
# Two fires with NULL containment
|
|
_seed_fire(conn, irwin_id="NULL-1", name="No Containment 1",
|
|
acres=10, contained=None,
|
|
last_event_at=now - 3600)
|
|
_seed_fire(conn, irwin_id="NULL-2", name="No Containment 2",
|
|
acres=20, contained=None,
|
|
last_event_at=now - 7200)
|
|
text = reporter.build_env_summary(now=now)
|
|
assert "Active fires (WFIGS, last 7d): 2" in text, (
|
|
f"should count 2 active fires, got: {text}")
|
|
|
|
def test_summary_empty_when_all_excluded(self, reporter):
|
|
conn = get_db()
|
|
now = int(time.time())
|
|
# Only tombstoned and 100%-contained fires
|
|
_seed_fire(conn, irwin_id="EX-1", name="Ex 1",
|
|
acres=100, contained=100,
|
|
last_event_at=now - 3600,
|
|
tombstoned_at=now - 1800)
|
|
_seed_fire(conn, irwin_id="EX-2", name="Ex 2",
|
|
acres=200, contained=100,
|
|
last_event_at=now - 7200)
|
|
text = reporter.build_env_summary(now=now)
|
|
assert "Active fires" not in text, (
|
|
f"should not mention fires when all excluded, got: {text}")
|