mirror of
https://github.com/zvx-echo6/meshai.git
synced 2026-06-10 17:04:45 +02:00
3 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| dcb53ae30c |
test: update stale assertions post feature/mesh-intelligence merge
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
|||
| 89640f624d |
fix(v0.7-fire-tracker-4-revised): rip ?status; LLM DM 7-path verification 3 of 7 pass (NOT verified)
Matt review caught a scope error: ?status was a hypothetical sketch
in the design doc ("a node could ping ?status cache peak") treated as
authorization without asking. Ripping the structured-command path
entirely. The LLM DM path with env_reporter injection is the natural-
language interface; ?status was redundant infrastructure parallel to
the path the design depends on.
What landed:
- router.py: _maybe_rewrite_status_query + _lookup_fire_fuzzy +
_build_fire_status_context removed. route() restored to:
bang -> IGNORE-empty -> LLM with verbatim query.
- tests/test_fire_tracker_phase4.py: 5 ?status tests removed; replaced
with two regression guards:
test_natural_language_fire_question_routes_to_llm -- "how's the
cache peak fire?" returns RouteType.LLM with the verbatim query
(no in-router rewriting).
test_status_helpers_removed_from_router -- hard-block on
_maybe_rewrite_status_query / _lookup_fire_fuzzy / "?status"
appearing anywhere in router.py source. If anyone adds a
structured-command path for fires, this test fails and the
author has to talk to Matt first.
- 56 passed in 3.80s across phase1+phase2+phase3+phase4+or-arch+
include-roundtrip.
What stays (NOT ripped):
- Daily fire digest -- scheduled broadcaster, not a command. Its 4
adapter_config rows (fires.digest_enabled / digest_schedule /
digest_timezone / digest_max_chars) stay GUI-editable.
- Bug A fix (UnboundLocalError at router.py:745) -- independent of
?status. Confirmed still in effect.
LLM DM 7-path verification result -- 3 of 7 pass, INCOMPLETE:
| # | query | env_reporter | verdict |
|---|-----------------------------------------------|----------------------|---------|
| 1 | "are there any fires near me?" | build_fires_detail | PASS |
| 2 | "any weather alerts?" | build_alerts_detail | FAIL |
| 3 | "any earthquakes nearby?" | build_quakes_detail | FAIL |
| 4 | "how's traffic on I-84?" | build_traffic_detail | FAIL |
| 5 | "what's the snake river level?" | build_gauges_detail | PASS |
| 6 | "what are the band conditions?" | build_swpc_detail | PASS |
| 7 | "why didn't I hear about anything today?" | build_drop_audit | FAIL |
Two distinct failure classes:
Class A -- routing miss (#4 traffic, #7 drop):
_ENV_KEYWORDS_TO_SUBTYPE lacks "traffic" (only road/jam/crash/
closure/511/incident map to "traffic"), so a query literally
mentioning "traffic" never triggers env scope -> build_traffic_detail
never runs even though traffic_events has 9 rows on disk. The LLM
fell back to training data and hallucinated I-84 conditions.
build_drop_audit has no natural-language trigger phrase at all;
"why didn't I hear about anything today?" has no env keyword.
Class B -- empty data + LLM hallucination (#2 alerts, #3 quakes):
Env scope IS detected, build_alerts_detail and build_quakes_detail
DO run, but return empty because nws_alerts has 0 rows and
quake_events 24h-window has 0 rows (legitimate empty state). The
LLM has no env block to ground on and hallucinated "144 earthquakes
worldwide" -- sounds authoritative, is fabricated.
Not fixed in this commit -- needs Matt's call on:
(a) keyword additions to _ENV_KEYWORDS_TO_SUBTYPE for traffic +
drop_audit triggers (risk: false-positive env-scope triggers
for unrelated phrases).
(b) anti-hallucination prompt clamp: "If a topic's env block is
missing/empty, say you don't have live data instead of
answering from general knowledge." (risk: bot apologizes
every other message.)
Per the "STOP if any path fails" instruction, this commit does NOT
claim verification done; the report at
v0.7-firetracker-phase4.md has the full table + per-row mesh-receiver
wire + per-failure root cause analysis.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| f69a05dd6d |
feat(v0.7-fire-tracker-4): fix LLM DM path + daily fire digest + ?status queries
Phase 4 of FIRMS+WFIGS fusion. Foundation: every direct LLM DM
mentioning a fire/weather/quake/avalanche/flood/etc. keyword was
failing silently in prod with UnboundLocalError because router.py
referenced scope_type before assigning it. With that path restored,
two new features land: a twice-daily fire-digest scheduled broadcast
(LLM-rendered) and a ?status <fire_name> on-demand mesh-DM intent.
BUG-FIX ROOT CAUSE (Job Zero):
router.py:745 ("if should_inject_mesh and scope_type == 'env'") read
`scope_type` -- a local variable bound only at line 761 inside an
unrelated `if self.source_manager and self.mesh_reporter` block.
Python's lexical scoping made scope_type a local of the whole
generate_llm_response function, so reading it before the assignment
raised UnboundLocalError on every env-keyword DM. The exception
propagated to main.py's outer except, no response went out, bot
appeared dead on fire/weather/quake/avalanche/flood queries.
Evidence (synthetic in-process trace against the live container's
config + GoogleBackend):
"are there any fires near me?" -> UnboundLocalError (pre-fix)
-> real LLM answer (post-fix)
"Yes, there are a few active
fires reported in the region.
Salmon River: 4,200 acres, 78%
contained. Cache Peak: 1,847
acres, 23% contained. ..."
"what's the weather?" -> UnboundLocalError (pre-fix)
-> "I do not have current weather
information. I can tell you
about active fires, stream gauge
levels, space weather, or band
conditions if you'd like." (post-fix)
"hi there" -> normal LLM answer in both cases
Fix: hoist `scope_type, scope_value = self._detect_mesh_scope(query)`
to right after `should_inject_mesh` is computed; remove the
now-duplicate detection inside the source_manager block.
Secondary mitigation: tightened the "do not invent commands" prompt
with an explicit "if no list appears above, you have NO commands"
clause. The prior prompt told the LLM "answer based on the command
list provided below" without always providing one, so the LLM
hallucinated plausible-sounding !commands (the "use ! commands"
canned-looking response Matt was seeing on non-env queries).
PHASE 4 FEATURES:
1. Fire-digest scheduler (meshai/notifications/scheduled/fire_digest.py).
Modeled after BandConditionsScheduler. Runs in the pipeline's
start_pipeline coroutine alongside band_conditions + reminders.
On each slot (default 06:00 + 18:00 America/Boise):
- Queries active fires (tombstoned_at IS NULL) + last 24h passes.
- Builds a prompt asking for a single mesh-wire summary <= 200
chars.
- Calls the LLM (Google/Anthropic/OpenAI per config).
- Falls back to a terse "Fires today (N): Cache Peak 1847 ac;
Twin Peaks 320 ac; +N more" line when the LLM is unavailable.
- Dispatches via dispatcher.dispatch_scheduled_broadcast (same
path band_conditions uses).
Idempotency: v16.sql adds fire_digest_broadcasts(slot_epoch PK,
sent_at, summary, source). INSERT OR IGNORE pattern blocks the same
slot firing twice (matters when container restarts mid-day).
2. ?status <fire_name> on-demand intent (router.py).
Before falling through to the LLM, route() now checks for a leading
"?status" / "status:" sigil or natural-language triggers like
"how is X fire?". On match:
- _lookup_fire_fuzzy walks fires by exact -> startswith ->
contains -> word-overlap (skipping a trailing " fire" word so
"cache peak fire" matches "Cache Peak"). Active fires rank
above tombstoned ones.
- _build_fire_status_context composes a small context block
(name, acres, containment, county/state, last 3 passes with
drift).
- The query is REWRITTEN into an LLM prompt with that context
inlined; the rest of the normal LLM path (chunking, history,
summary persistence) runs unchanged.
Live verification: "?status Cache Peak" -> "The Cache Peak fire is
1,847 acres and 23% contained. It's located in Probe / ID.";
"?status Salmon" -> word-overlap matches "Salmon River" ->
"The Salmon River fire is 4,200 acres and 78% contained, located
in Probe / ID."
3. adapter_config rows (GUI-editable per CONFIG-vs-CODE rule):
fires.digest_enabled = true (master toggle)
fires.digest_schedule = ["06:00", "18:00"]
fires.digest_timezone = "America/Boise"
fires.digest_max_chars = 200
Schema (v16.sql):
- fire_digest_broadcasts(slot_epoch INTEGER PK, sent_at, summary,
source) with source in {'llm', 'fallback_terse', 'skipped_no_fires'}.
- Index on sent_at for ops queries.
Tests (tests/test_fire_tracker_phase4.py, 10 cases all green):
- Regression guard: scope_type appears as an assignment BEFORE the
env_reporter check (prevents the UnboundLocalError from coming back).
- adapter_config seeds all 4 digest keys with expected defaults.
- render_digest returns ('', 'no_fires') when no active fires.
- render_digest falls back to terse line when LLM is None; wire fits cap.
- render_digest with a stub LLM returns ('<llm text>', 'llm').
- _lookup_fire_fuzzy: exact, "X fire" trim, word-overlap, no-match.
- _maybe_rewrite_status_query: builds context-bearing prompt; returns
None on non-status queries.
Combined suite: 60 passed in 3.81s across phase1+phase2+phase3+phase4
+or-arch+include-roundtrip.
Live verification on CT108 after rebuild:
- v16 migration applied (schema_meta=16, no Traceback in 3 min).
- FireDigestScheduler started: enabled=True schedule=['06:00','18:00']
tz=America/Boise.
- LLM DM probe (real Gemini) returns real answers on env queries
(Bug A fixed end-to-end).
- ?status Cache Peak + ?status Salmon return fire-specific summaries.
- render_digest with real LLM returns source=llm + non-empty wire.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
