mirror of
https://github.com/zvx-echo6/meshai.git
synced 2026-06-10 17:04:45 +02:00
Matt Johnson (via Claude)
89640f624d
Matt review caught a scope error: ?status was a hypothetical sketch
in the design doc ("a node could ping ?status cache peak") treated as
authorization without asking. Ripping the structured-command path
entirely. The LLM DM path with env_reporter injection is the natural-
language interface; ?status was redundant infrastructure parallel to
the path the design depends on.
What landed:
- router.py: _maybe_rewrite_status_query + _lookup_fire_fuzzy +
_build_fire_status_context removed. route() restored to:
bang -> IGNORE-empty -> LLM with verbatim query.
- tests/test_fire_tracker_phase4.py: 5 ?status tests removed; replaced
with two regression guards:
test_natural_language_fire_question_routes_to_llm -- "how's the
cache peak fire?" returns RouteType.LLM with the verbatim query
(no in-router rewriting).
test_status_helpers_removed_from_router -- hard-block on
_maybe_rewrite_status_query / _lookup_fire_fuzzy / "?status"
appearing anywhere in router.py source. If anyone adds a
structured-command path for fires, this test fails and the
author has to talk to Matt first.
- 56 passed in 3.80s across phase1+phase2+phase3+phase4+or-arch+
include-roundtrip.
What stays (NOT ripped):
- Daily fire digest -- scheduled broadcaster, not a command. Its 4
adapter_config rows (fires.digest_enabled / digest_schedule /
digest_timezone / digest_max_chars) stay GUI-editable.
- Bug A fix (UnboundLocalError at router.py:745) -- independent of
?status. Confirmed still in effect.
LLM DM 7-path verification result -- 3 of 7 pass, INCOMPLETE:
| # | query | env_reporter | verdict |
|---|-----------------------------------------------|----------------------|---------|
| 1 | "are there any fires near me?" | build_fires_detail | PASS |
| 2 | "any weather alerts?" | build_alerts_detail | FAIL |
| 3 | "any earthquakes nearby?" | build_quakes_detail | FAIL |
| 4 | "how's traffic on I-84?" | build_traffic_detail | FAIL |
| 5 | "what's the snake river level?" | build_gauges_detail | PASS |
| 6 | "what are the band conditions?" | build_swpc_detail | PASS |
| 7 | "why didn't I hear about anything today?" | build_drop_audit | FAIL |
Two distinct failure classes:
Class A -- routing miss (#4 traffic, #7 drop):
_ENV_KEYWORDS_TO_SUBTYPE lacks "traffic" (only road/jam/crash/
closure/511/incident map to "traffic"), so a query literally
mentioning "traffic" never triggers env scope -> build_traffic_detail
never runs even though traffic_events has 9 rows on disk. The LLM
fell back to training data and hallucinated I-84 conditions.
build_drop_audit has no natural-language trigger phrase at all;
"why didn't I hear about anything today?" has no env keyword.
Class B -- empty data + LLM hallucination (#2 alerts, #3 quakes):
Env scope IS detected, build_alerts_detail and build_quakes_detail
DO run, but return empty because nws_alerts has 0 rows and
quake_events 24h-window has 0 rows (legitimate empty state). The
LLM has no env block to ground on and hallucinated "144 earthquakes
worldwide" -- sounds authoritative, is fabricated.
Not fixed in this commit -- needs Matt's call on:
(a) keyword additions to _ENV_KEYWORDS_TO_SUBTYPE for traffic +
drop_audit triggers (risk: false-positive env-scope triggers
for unrelated phrases).
(b) anti-hallucination prompt clamp: "If a topic's env block is
missing/empty, say you don't have live data instead of
answering from general knowledge." (risk: bot apologizes
every other message.)
Per the "STOP if any path fails" instruction, this commit does NOT
claim verification done; the report at
v0.7-firetracker-phase4.md has the full table + per-row mesh-receiver
wire + per-failure root cause analysis.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| fixtures/central_envelopes | ||
| conftest.py | ||
| test_adapter_avalanche.py | ||
| test_adapter_config_api.py | ||
| test_adapter_config_foundation.py | ||
| test_adapter_ducting.py | ||
| test_adapter_fires.py | ||
| test_adapter_firms.py | ||
| test_adapter_nws.py | ||
| test_adapter_roads511.py | ||
| test_adapter_swpc.py | ||
| test_adapter_traffic.py | ||
| test_adapter_usgs.py | ||
| test_adapter_usgs_quake.py | ||
| test_avalanche_v057.py | ||
| test_band_conditions.py | ||
| test_central_consumer.py | ||
| test_central_envelope_to_wire_v057.py | ||
| test_central_normalizer.py | ||
| test_central_region_routing.py | ||
| test_central_sub_adapter_routing.py | ||
| test_channel_rendering.py | ||
| test_cold_start_grace.py | ||
| test_config_loader.py | ||
| test_config_source_field.py | ||
| test_consumer_default_deny.py | ||
| test_curation.py | ||
| test_dashboard_config_save.py | ||
| test_dispatcher_persistence.py | ||
| test_env_reporter.py | ||
| test_fire_tracker_phase1.py | ||
| test_fire_tracker_phase2.py | ||
| test_fire_tracker_phase3.py | ||
| test_fire_tracker_phase4.py | ||
| test_fire_v057.py | ||
| test_firms_handler.py | ||
| test_incident_handler.py | ||
| test_include_roundtrip.py | ||
| test_itd_511_work_zone.py | ||
| test_notification_toggles.py | ||
| test_nwis_handler.py | ||
| test_nws_dedup_relaxation.py | ||
| test_nws_handler.py | ||
| test_or_arch_continuous.py | ||
| test_persistence.py | ||
| test_pipeline_digest.py | ||
| test_pipeline_grouper.py | ||
| test_pipeline_inhibitor_grouper.py | ||
| test_pipeline_persistence.py | ||
| test_pipeline_scheduler.py | ||
| test_pipeline_skeleton.py | ||
| test_pipeline_toggle_filter.py | ||
| test_quake_handler.py | ||
| test_reminders.py | ||
| test_renderers.py | ||
| test_rf_v057.py | ||
| test_router_env_scope.py | ||
| test_save_section_secret_preserve.py | ||
| test_seismic_v057.py | ||
| test_swpc_handler.py | ||
| test_tail_followups.py | ||
| test_tracking_v057.py | ||
| test_traffic_v057.py | ||
| test_v052_dispatcher.py | ||
| test_water_v057.py | ||
| test_weather_v057.py | ||
| test_wfigs_handler.py | ||
| test_work_zone_renderer.py | ||