meshai/tests/test_fire_tracker_phase4.py

187 lines
6.8 KiB
Python
Raw Normal View History

feat(v0.7-fire-tracker-4): fix LLM DM path + daily fire digest + ?status queries Phase 4 of FIRMS+WFIGS fusion. Foundation: every direct LLM DM mentioning a fire/weather/quake/avalanche/flood/etc. keyword was failing silently in prod with UnboundLocalError because router.py referenced scope_type before assigning it. With that path restored, two new features land: a twice-daily fire-digest scheduled broadcast (LLM-rendered) and a ?status <fire_name> on-demand mesh-DM intent. BUG-FIX ROOT CAUSE (Job Zero): router.py:745 ("if should_inject_mesh and scope_type == 'env'") read `scope_type` -- a local variable bound only at line 761 inside an unrelated `if self.source_manager and self.mesh_reporter` block. Python's lexical scoping made scope_type a local of the whole generate_llm_response function, so reading it before the assignment raised UnboundLocalError on every env-keyword DM. The exception propagated to main.py's outer except, no response went out, bot appeared dead on fire/weather/quake/avalanche/flood queries. Evidence (synthetic in-process trace against the live container's config + GoogleBackend): "are there any fires near me?" -> UnboundLocalError (pre-fix) -> real LLM answer (post-fix) "Yes, there are a few active fires reported in the region. Salmon River: 4,200 acres, 78% contained. Cache Peak: 1,847 acres, 23% contained. ..." "what's the weather?" -> UnboundLocalError (pre-fix) -> "I do not have current weather information. I can tell you about active fires, stream gauge levels, space weather, or band conditions if you'd like." (post-fix) "hi there" -> normal LLM answer in both cases Fix: hoist `scope_type, scope_value = self._detect_mesh_scope(query)` to right after `should_inject_mesh` is computed; remove the now-duplicate detection inside the source_manager block. Secondary mitigation: tightened the "do not invent commands" prompt with an explicit "if no list appears above, you have NO commands" clause. The prior prompt told the LLM "answer based on the command list provided below" without always providing one, so the LLM hallucinated plausible-sounding !commands (the "use ! commands" canned-looking response Matt was seeing on non-env queries). PHASE 4 FEATURES: 1. Fire-digest scheduler (meshai/notifications/scheduled/fire_digest.py). Modeled after BandConditionsScheduler. Runs in the pipeline's start_pipeline coroutine alongside band_conditions + reminders. On each slot (default 06:00 + 18:00 America/Boise): - Queries active fires (tombstoned_at IS NULL) + last 24h passes. - Builds a prompt asking for a single mesh-wire summary <= 200 chars. - Calls the LLM (Google/Anthropic/OpenAI per config). - Falls back to a terse "Fires today (N): Cache Peak 1847 ac; Twin Peaks 320 ac; +N more" line when the LLM is unavailable. - Dispatches via dispatcher.dispatch_scheduled_broadcast (same path band_conditions uses). Idempotency: v16.sql adds fire_digest_broadcasts(slot_epoch PK, sent_at, summary, source). INSERT OR IGNORE pattern blocks the same slot firing twice (matters when container restarts mid-day). 2. ?status <fire_name> on-demand intent (router.py). Before falling through to the LLM, route() now checks for a leading "?status" / "status:" sigil or natural-language triggers like "how is X fire?". On match: - _lookup_fire_fuzzy walks fires by exact -> startswith -> contains -> word-overlap (skipping a trailing " fire" word so "cache peak fire" matches "Cache Peak"). Active fires rank above tombstoned ones. - _build_fire_status_context composes a small context block (name, acres, containment, county/state, last 3 passes with drift). - The query is REWRITTEN into an LLM prompt with that context inlined; the rest of the normal LLM path (chunking, history, summary persistence) runs unchanged. Live verification: "?status Cache Peak" -> "The Cache Peak fire is 1,847 acres and 23% contained. It's located in Probe / ID."; "?status Salmon" -> word-overlap matches "Salmon River" -> "The Salmon River fire is 4,200 acres and 78% contained, located in Probe / ID." 3. adapter_config rows (GUI-editable per CONFIG-vs-CODE rule): fires.digest_enabled = true (master toggle) fires.digest_schedule = ["06:00", "18:00"] fires.digest_timezone = "America/Boise" fires.digest_max_chars = 200 Schema (v16.sql): - fire_digest_broadcasts(slot_epoch INTEGER PK, sent_at, summary, source) with source in {'llm', 'fallback_terse', 'skipped_no_fires'}. - Index on sent_at for ops queries. Tests (tests/test_fire_tracker_phase4.py, 10 cases all green): - Regression guard: scope_type appears as an assignment BEFORE the env_reporter check (prevents the UnboundLocalError from coming back). - adapter_config seeds all 4 digest keys with expected defaults. - render_digest returns ('', 'no_fires') when no active fires. - render_digest falls back to terse line when LLM is None; wire fits cap. - render_digest with a stub LLM returns ('<llm text>', 'llm'). - _lookup_fire_fuzzy: exact, "X fire" trim, word-overlap, no-match. - _maybe_rewrite_status_query: builds context-bearing prompt; returns None on non-status queries. Combined suite: 60 passed in 3.81s across phase1+phase2+phase3+phase4 +or-arch+include-roundtrip. Live verification on CT108 after rebuild: - v16 migration applied (schema_meta=16, no Traceback in 3 min). - FireDigestScheduler started: enabled=True schedule=['06:00','18:00'] tz=America/Boise. - LLM DM probe (real Gemini) returns real answers on env queries (Bug A fixed end-to-end). - ?status Cache Peak + ?status Salmon return fire-specific summaries. - render_digest with real LLM returns source=llm + non-empty wire. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 07:13:17 +00:00
"""v0.7-fire-tracker-4 tests."""
from __future__ import annotations
import asyncio
import time
import uuid
import pytest
@pytest.fixture(autouse=True)
def _isolate_db(tmp_path, monkeypatch):
db_path = str(tmp_path / f"meshai-{uuid.uuid4().hex}.sqlite")
monkeypatch.setenv("MESHAI_DB_PATH", db_path)
from meshai.persistence import db as pdb
pdb.close_thread_connection()
pdb._initialised.discard(db_path)
from meshai.persistence import init_db
init_db(db_path)
yield db_path
pdb.close_thread_connection()
pdb._initialised.discard(db_path)
def _seed_fire(*, irwin_id, name, lat, lon, acres=None,
contained=None, county="Test", state="ID"):
from meshai.persistence import get_db
get_db().execute(
"INSERT INTO fires(irwin_id, incident_name, current_acres, "
"current_contained_pct, lat, lon, county, state, last_event_at) "
"VALUES (?,?,?,?,?,?,?,?,?)",
(irwin_id, name, acres, contained, lat, lon, county, state,
int(time.time())),
)
# ===========================================================================
# Bug A regression: scope_type defined before use
# ===========================================================================
def test_router_scope_type_defined_before_env_check():
"""The env_reporter check at the top of generate_llm_response reads
scope_type. Pre-fix it was UnboundLocalError on every env query."""
import re
from pathlib import Path
src = Path("/opt/meshai/meshai/router.py").read_text()
# Find the "if should_inject_mesh and scope_type" line + the
# nearest preceding `scope_type, scope_value = ` assignment.
env_use_line = None
for i, line in enumerate(src.splitlines(), start=1):
if "should_inject_mesh and scope_type" in line:
env_use_line = i
break
assert env_use_line is not None
# There must be an assignment on or before this line.
preceding = "\n".join(src.splitlines()[: env_use_line - 1])
assert re.search(r"scope_type[, ]+scope_value\s*=", preceding) \
or "scope_type:" in preceding
# ===========================================================================
# adapter_config seed + categories registration
# ===========================================================================
def test_adapter_config_seeds_digest_keys():
from meshai.persistence import get_db
rows = {
(r["adapter"], r["key"]): r["default_json"]
for r in get_db().execute(
"SELECT adapter, key, default_json FROM adapter_config "
"WHERE adapter='fires' AND key LIKE 'digest%'"
)
}
assert rows[("fires", "digest_enabled")] == "true"
assert rows[("fires", "digest_schedule")] == '["06:00", "18:00"]'
assert rows[("fires", "digest_timezone")] == '"America/Boise"'
assert rows[("fires", "digest_max_chars")] == "200"
# ===========================================================================
# Digest renderer
# ===========================================================================
def test_render_digest_returns_no_fires_when_table_empty():
from meshai.notifications.scheduled.fire_digest import render_digest
async def _run():
return await render_digest(llm_backend=None, max_chars=200)
wire, source = asyncio.run(_run())
assert wire == ""
assert source == "no_fires"
def test_render_digest_terse_fallback_when_no_llm():
_seed_fire(irwin_id="ID-A", name="Cache Peak",
lat=42.0, lon=-114.0, acres=1847, contained=23)
_seed_fire(irwin_id="ID-B", name="Twin Peaks",
lat=43.0, lon=-115.0, acres=320, contained=5)
from meshai.notifications.scheduled.fire_digest import render_digest
async def _run():
return await render_digest(llm_backend=None, max_chars=200)
wire, source = asyncio.run(_run())
assert source == "fallback_terse"
assert wire
assert "Cache Peak" in wire
assert len(wire) <= 200
def test_render_digest_uses_llm_when_available():
"""When the LLM backend returns a string, that string IS the wire."""
_seed_fire(irwin_id="ID-A", name="Cache Peak",
lat=42.0, lon=-114.0, acres=1847)
class StubLLM:
async def generate(self, *, messages, system_prompt, max_tokens):
# The renderer must give us a single-line wire derived from
# the LLM output, with markdown stripped + cap applied.
return "Cache Peak 1847 ac stable; no spotting today."
from meshai.notifications.scheduled.fire_digest import render_digest
async def _run():
return await render_digest(llm_backend=StubLLM(), max_chars=200)
wire, source = asyncio.run(_run())
assert source == "llm"
assert wire == "Cache Peak 1847 ac stable; no spotting today."
# ===========================================================================
# Fuzzy fire lookup for ?status
# ===========================================================================
def test_status_lookup_exact_name():
from meshai.router import _lookup_fire_fuzzy
_seed_fire(irwin_id="ID-A", name="Cache Peak",
lat=42.0, lon=-114.0, acres=1847)
f = _lookup_fire_fuzzy("Cache Peak")
assert f is not None
assert f["incident_name"] == "Cache Peak"
def test_status_lookup_trims_trailing_fire_word():
from meshai.router import _lookup_fire_fuzzy
_seed_fire(irwin_id="ID-A", name="Cache Peak",
lat=42.0, lon=-114.0, acres=1847)
f = _lookup_fire_fuzzy("cache peak fire")
assert f is not None
assert f["incident_name"] == "Cache Peak"
def test_status_lookup_word_overlap_fallback():
from meshai.router import _lookup_fire_fuzzy
_seed_fire(irwin_id="ID-A", name="Cache Peak",
lat=42.0, lon=-114.0, acres=1847)
f = _lookup_fire_fuzzy("how is peak doing")
assert f is not None
assert f["incident_name"] == "Cache Peak"
def test_status_lookup_returns_none_on_no_match():
from meshai.router import _lookup_fire_fuzzy
_seed_fire(irwin_id="ID-A", name="Cache Peak",
lat=42.0, lon=-114.0, acres=1847)
assert _lookup_fire_fuzzy("nonexistent ranger station") is None
def test_status_query_rewrite_includes_fire_context():
from meshai.router import _maybe_rewrite_status_query
_seed_fire(irwin_id="ID-A", name="Cache Peak",
lat=42.0, lon=-114.0, acres=1847, contained=23)
out = _maybe_rewrite_status_query("?status Cache Peak", router=None)
assert out is not None
assert "Cache Peak" in out
assert "1847" in out
# Must instruct the LLM to be terse mesh format.
assert "mesh" in out.lower()
def test_status_query_rewrite_returns_none_when_not_status():
from meshai.router import _maybe_rewrite_status_query
out = _maybe_rewrite_status_query("how's the weather?", router=None)
assert out is None