mirror of
https://github.com/zvx-echo6/meshai.git
synced 2026-06-11 01:14:45 +02:00
310 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| ebb34b75ef |
feat(wfigs): multi-line renderer with delta/bold logic and new fire fields
Normalizer: add fire_cause, agency, personnel, unique_fire_id from WFIGS raw payload to the normalized incident dict. Renderer: replace single-line wire format with structured multi-line output — header, size/contained with bold deltas on updates, location anchor, cause/discovered date, and unique fire ID. Update call sites pass last_bcast_acres and last_bcast_contained for case-(iii) updates to enable delta calculation and selective bolding. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
|||
| fc78f26c82 |
fix(grouper): strip non-serializable values before persisting held events
event.data can contain callables and internal _on_ callbacks that cause json.dumps to fail with TypeError. Filter these out before serializing to SQLite. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
|||
| 3b12be2242 |
fix(normalizer): accept wildfire IncidentTypeCategory in WFIGS parser
WFIGS API returns both "WF" and "wildfire" as IncidentTypeCategory values. The previous check only accepted "WF", silently dropping wildfire-typed incidents. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
|||
| 94118f7b6d |
fix(normalizer): handle None return from _parse_wfigs_incidents
When the non-WF filter returns None, the caller must check before assigning to n[_kind]. Fixes TypeError on non-wildfire incidents. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
|||
| 69ccbc035a |
feat(wfigs): add severity override for fire broadcasts
Fire events now carry _severity_override in the data dict: - New fires (case i/ii): priority by default - Update fires (case iii): priority by default - All fires: immediate if acres > 1000 OR contained_pct == 0 consumer.py checks for _severity_override before falling back to map_severity(inner.severity). This ensures fire broadcasts are prioritized appropriately in the dispatch queue. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
|||
| fa8553a1e0 |
refactor(dashboard): move Central Connection to dedicated tab
- Add 'Central' as first tab in FAMILIES array (before Weather) - Import Server icon from lucide-react - Remove Central Connection card from header area - Render Central config in its own tab panel (no adapter sub-tabs) The Central tab now shows URL, Durable, and Region fields with the enabled toggle, matching the previous inline card behavior. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
|||
| 9d8d5fd321 |
feat(dashboard): expanded fire config with incident types and triggers
- Add allowed_incident_types (WF/RX/OTHER) checkboxes - Conditionally show native polling settings (tick_seconds, state) - Broadcast triggers section with acres/containment toggles - Cooldown and freshness window inputs in hours Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
|||
| 335dbf632c |
feat(dashboard): fire adapter config in Environment page
Rework the NIFC Fire Perimeters tab to show all fire config in one place:
- Native polling settings (tick_seconds, state) shown only when native
- Broadcast Settings section (always shown):
- Broadcast on acres increase toggle
- Broadcast on containment increase toggle
- Update cooldown (hours, converted from cooldown_seconds)
- Freshness window (hours, 0 = disabled)
- Daily Digest section (always shown):
- Digest enabled toggle
- Schedule times (HH:MM list)
Changes persist to adapter_config table via PUT /api/adapter-config/{adapter}/{key}.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
|||
| 87cce0048d |
fix(dispatcher): disable staleness filter for fire events
Fire events are always relevant regardless of age (a wildfire burning for 9 hours is not stale — it's ongoing). The staleness filter was designed for incidents with time_validity semantics, not persistent fire state. - defaults.py: add wfigs.freshness_seconds = 0 (disabled) - dispatcher.py: for fire toggle family, read from adapter_config instead of toggle; skip staleness check when freshness_s == 0 Fixes Blue Ridge fire being dropped after LAST_PER_SUBJECT replay. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
|||
| bd4b264672 |
fix(central): use LAST_PER_SUBJECT and filter non-WF incidents
- consumer.py: change DeliverPolicy from NEW to LAST_PER_SUBJECT to get latest state per subject on reconnect instead of replaying backlog - central_normalizer.py: drop RX and non-wildfire (non-WF) incident types early in _parse_wfigs_incidents before they reach the handler Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
|||
| 798712d20c |
docs(v0.7): comprehensive dashboard docs rewrite -- Reference +8 sections, per-page tooltips, component polish
All three approved tiers in one commit. Reference.tsx is the deep docs
hub (8 new sections); the 10 other pages get short helper text +
tooltips that cross-reference back into Reference; 3 components get
operational-context tooltips. No new features land here -- this is the
copy that catches the GUI up to v0.6 + v0.7 system behavior.
Decisions applied per Matt's call:
- Keep both bang commands AND the LLM DM path (bangs are short on a
mesh-constrained interface; LLM is the anything-else path). Cross-
references between the two land in Reference -> Commands and
Reference -> LLM DM Queries.
- Rename "wire-string rendering" to "broadcast text" in user-facing
copy on TownAnchors.tsx, GaugeSites.tsx, and the Curation section of
Reference.tsx.
- Keep the "AND-model anti-pattern" tooltip as-is on Environment.tsx +
GaugeSites.tsx (specificity is the value for advanced users); the
OR-not-AND Reference section is its home definition that other
tooltips can link to.
Ham terminology preserved:
- Reference.tsx solar/Kp section retains "Quiet sun" / "Quiet HF
conditions" language (SFI/Kp vocabulary, not the deleted Quiet Hours
feature -- confirmed via direct grep before writing).
Tier 1: Reference.tsx (the depth doc) -- 8 new sections, ordered for
readability:
- "Fire Tracker (Fusion)": Phases 1-4 unified. Six fire-family alert
categories with example wire strings (wildfire_declared,
wildfire_growth, wildfire_halted, wildfire_spotting,
unattributed_hotspot_cluster, wildfire_incident). Attribution
mechanics (spread_radius_mi default, centroid as 24h median).
Movement mechanics (pass_id bucketing, per-pass centroid, 8-way
bearing, mi/h drift). Spotting mechanics (convex-hull perimeter +
vertex-distance approximation + per-fire cooldown). Daily LLM digest
(twice-daily summary broadcaster). The 10 fires.* adapter_config
knobs with defaults.
- "Broadcast Types": the three prefix categories -- New: (first sight),
Update: (material change), Active: (clock-driven reminder).
- "Reminder System": cadences per adapter (WFIGS 8h, SWPC 8h, ITD 511
per-zone). The tombstone (fires.tombstoned_at) termination. The
per-adapter reminder_enabled flag.
- "LLM DM (Natural-Language Queries)": all 7 env_reporter adapter
blocks (build_fires_detail / build_alerts_detail / build_quakes_detail
/ build_traffic_detail / build_gauges_detail / build_swpc_detail /
build_drop_audit) with example questions that hit each one. The
grounding clause behavior ("No active X right now" when an adapter
block is empty -- the v0.7-fire-tracker-4-final clamp). The
include_in_llm_context per-adapter toggle.
- "OR-not-AND Architecture": the per-adapter Central vs native
contract. Mutually exclusive. The AND-mode anti-pattern definition
(referenced by the Environment + GaugeSites tooltips). The Spokane
fix context.
- "Adapter Config & the CODE Rule": the GUI knob hub. The CONFIG-vs-
CODE split (thresholds in CONFIG, sentence templates / emoji /
translation maps in CODE). Restart-required vs live keys. The
include_in_llm_context toggle.
- "Curation: Gauges & Towns": Gauge Sites (NWS-AHPS thresholds, USGS
lookup, Action/Minor/Moderate/Major). Town Anchors (broadcast text
suffix lookup chain: Photon -> this table -> landclass -> county
-> coords). Example output "3 mi N of Almo".
- "Schema Migrations": light touch. v11-v16 schema additions tagged
with the phase they shipped under.
Tier 2: per-page tooltips and cross-references (10 pages):
- AdapterConfig.tsx: header paragraph extended with the CODE rule
pointer + LLM context toggle explanation.
- Alerts.tsx: !subscribe blurb extended with the three broadcast types
and links to Reference -> Broadcast Types + Reminder System.
- Config.tsx: environmental section description updated to point at
Environment.tsx for adapter knobs + Reference -> OR-not-AND for the
architecture.
- Dashboard.tsx: RF Propagation title carries SWPC R/S/G + Kp legend
tooltip; LOCAL badge defines what counts as local.
- Environment.tsx: Central region-token helper now references the
OR-not-AND section; tick_seconds defined inline as the native-mode
poll interval.
- GaugeSites.tsx: page description rewritten -- replaces "envelope
time" jargon with operational language, explains USGS lookup
mechanics, points at Reference -> OR-not-AND for the central-feed
disable.
- Mesh.tsx: Topology + Geographic buttons get tooltips defining the
rendering model.
- Notifications.tsx: band-conditions block extended with the daily
fire digest pointer + Reference -> Fire Tracker + Broadcast Types
cross-refs.
- TownAnchors.tsx: page description rewritten -- "wire-string
rendering" -> "broadcast text", chain fallback explained ("Photon
-> this table -> landclass -> county/state -> coords"), example
output included.
Tier 3: component tooltip polish (3 components):
- NodeTable.tsx: Battery + Last Heard column headers get title-bearing
spans with the voltage chart + offline-threshold legend.
- NodeDetail.tsx: SNR quality bands documented as a comment in the
neighbor render block (the legend lives next to where the colored
quality dots are computed).
- RestartBanner.tsx: banner copy extended with the restart-required
catalog (Config -> environmental, LLM backend swap, dispatcher
cold-start grace) so operators know what touched it.
Build verification:
- tsc + vite build green (one warning about chunk size > 500kB --
pre-existing).
- All 8 new TOPICS ids resolve in the served bundle:
adapter-config, broadcast-types, curation, fire-tracker,
llm-dm, or-not-and, reminders, schema.
- Distinctive new strings present in the bundle ("3 mi N of Almo",
"Photon nearest-town", "AND-mode anti-pattern", "R (Radio Blackouts").
- "Quiet sun" preserved (the ham SFI/Kp vocabulary in the Solar
section, not the deleted Quiet Hours feature).
- Container Up healthy, 0 tracebacks in 2 min post-rebuild.
Changelog: v0.7-docs-rewrite.md (per-page strip / rewrite / add table).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 0934601265 |
fix(v0.7-fire-tracker-4-final): widen env-scope keyword catch + anti-hallucination clause -- close all 7 LLM DM paths
Follow-up to v0.7-fire-tracker-4-revised. 7-path verify identified 4
paths failing for 2 root causes: (A) _ENV_KEYWORDS_TO_SUBTYPE missing
"traffic" + drop-audit phrases so build_traffic_detail and
build_drop_audit never ran; (B) LLM hallucinated specific numbers when
env blocks were empty (fabricated 144 earthquakes from a blank
quake_events table). This commit widens the keyword catch (phrase-match
for multi-word triggers to avoid false positives) and adds a positive-
framed anti-hallucination instruction to the env-context system prompt.
Re-verified all 7 paths against real Gemini in the prod container;
verdicts in v0.7-firetracker-phase4.md.
Class A -- routing miss fix:
- _ENV_KEYWORDS_TO_SUBTYPE gains "traffic"/"commute"/"highway"
mapped to the existing "traffic" subtype.
- New _ENV_PHRASES_TO_SUBTYPE dict for multi-word triggers, matched
as whole-phrase substrings (NOT single-word membership). Drop-audit
phrases: "why didn't"/"why didnt"/"why am i not"/"why am i missing"
/"what was filtered"/"drop audit"/"filtered out" all map to a new
"drop_audit" subtype. Phrase-match keeps "why" alone from
false-positing every "why is X" question.
- _detect_env_subtype now checks phrases first, then falls back to
single-word tokenized match.
Class B -- positive-framed anti-hallucination clause:
- New module-level ENV_GROUNDING_CLAUSE constant. Appended to the
system prompt whenever env scope is detected (after env_block +
drop_block injection).
- Per Matt's mitigation guidance: positive ("answer from the blocks")
not negative ("do not hallucinate"). Wording:
"ENVIRONMENTAL CONTEXT GROUNDING:
Answer only from the environmental context blocks above. If a
block is empty or missing for an adapter the user asked about
(e.g. no NWS alerts in the block), say something like 'No active
<category> right now' -- never invent specific numbers, place
names, or counts. If you do not have a relevant block for the
question, say so briefly."
7-path verification, post-fix (real Gemini, prod container):
| # | query | method | verdict |
|---|-------------------------------------------------|----------------------|---------|
| 1 | "are there any fires near me?" | build_fires_detail | PASS |
| 2 | "any weather alerts?" | build_alerts_detail | PASS |
| 3 | "any earthquakes nearby?" | build_quakes_detail | PASS |
| 4 | "how's traffic on I-84?" | build_traffic_detail | PASS |
| 5 | "what's the snake river level?" | build_gauges_detail | PASS |
| 6 | "what are the band conditions?" | build_swpc_detail | PASS |
| 7 | "why didn't I hear about anything today?" | build_drop_audit | PASS |
Hallucination evidence (pre vs post on the quakes path):
pre-fix: "There have been 144 earthquakes of magnitude 1.5 or
greater in the past 24 hours worldwide. Some of the most
recent earthquakes reported include: A magnitude 2.1
earthquake in Pahala, Hawaii..." (fabricated)
post-fix: "I haven't observed any information about earthquakes
in the mesh data." (grounded)
Routing-miss evidence (pre vs post on the traffic path):
pre-fix: _detect_env_subtype("how's traffic on I-84?") -> None
-> env scope NOT triggered, build_traffic_detail never
called, LLM fabricated I-84 conditions in OR
post-fix: _detect_env_subtype("how's traffic on I-84?") -> "traffic"
-> env scope triggers, build_traffic_detail returns
185 chars of real Ada-county incident data, LLM
grounds on it: "I haven't observed any active
traffic incidents on I-84 within the last two hours.
The current active incidents are on North 9th Street
/ South 9th Street and SH-21, both in Ada."
Tests:
- 7 phase4 tests pass (no new tests needed; verification is the LLM
DM path itself).
- Full suite: 56 passed in 3.80s across phase1+phase2+phase3+phase4
+or-arch+include-roundtrip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 89640f624d |
fix(v0.7-fire-tracker-4-revised): rip ?status; LLM DM 7-path verification 3 of 7 pass (NOT verified)
Matt review caught a scope error: ?status was a hypothetical sketch
in the design doc ("a node could ping ?status cache peak") treated as
authorization without asking. Ripping the structured-command path
entirely. The LLM DM path with env_reporter injection is the natural-
language interface; ?status was redundant infrastructure parallel to
the path the design depends on.
What landed:
- router.py: _maybe_rewrite_status_query + _lookup_fire_fuzzy +
_build_fire_status_context removed. route() restored to:
bang -> IGNORE-empty -> LLM with verbatim query.
- tests/test_fire_tracker_phase4.py: 5 ?status tests removed; replaced
with two regression guards:
test_natural_language_fire_question_routes_to_llm -- "how's the
cache peak fire?" returns RouteType.LLM with the verbatim query
(no in-router rewriting).
test_status_helpers_removed_from_router -- hard-block on
_maybe_rewrite_status_query / _lookup_fire_fuzzy / "?status"
appearing anywhere in router.py source. If anyone adds a
structured-command path for fires, this test fails and the
author has to talk to Matt first.
- 56 passed in 3.80s across phase1+phase2+phase3+phase4+or-arch+
include-roundtrip.
What stays (NOT ripped):
- Daily fire digest -- scheduled broadcaster, not a command. Its 4
adapter_config rows (fires.digest_enabled / digest_schedule /
digest_timezone / digest_max_chars) stay GUI-editable.
- Bug A fix (UnboundLocalError at router.py:745) -- independent of
?status. Confirmed still in effect.
LLM DM 7-path verification result -- 3 of 7 pass, INCOMPLETE:
| # | query | env_reporter | verdict |
|---|-----------------------------------------------|----------------------|---------|
| 1 | "are there any fires near me?" | build_fires_detail | PASS |
| 2 | "any weather alerts?" | build_alerts_detail | FAIL |
| 3 | "any earthquakes nearby?" | build_quakes_detail | FAIL |
| 4 | "how's traffic on I-84?" | build_traffic_detail | FAIL |
| 5 | "what's the snake river level?" | build_gauges_detail | PASS |
| 6 | "what are the band conditions?" | build_swpc_detail | PASS |
| 7 | "why didn't I hear about anything today?" | build_drop_audit | FAIL |
Two distinct failure classes:
Class A -- routing miss (#4 traffic, #7 drop):
_ENV_KEYWORDS_TO_SUBTYPE lacks "traffic" (only road/jam/crash/
closure/511/incident map to "traffic"), so a query literally
mentioning "traffic" never triggers env scope -> build_traffic_detail
never runs even though traffic_events has 9 rows on disk. The LLM
fell back to training data and hallucinated I-84 conditions.
build_drop_audit has no natural-language trigger phrase at all;
"why didn't I hear about anything today?" has no env keyword.
Class B -- empty data + LLM hallucination (#2 alerts, #3 quakes):
Env scope IS detected, build_alerts_detail and build_quakes_detail
DO run, but return empty because nws_alerts has 0 rows and
quake_events 24h-window has 0 rows (legitimate empty state). The
LLM has no env block to ground on and hallucinated "144 earthquakes
worldwide" -- sounds authoritative, is fabricated.
Not fixed in this commit -- needs Matt's call on:
(a) keyword additions to _ENV_KEYWORDS_TO_SUBTYPE for traffic +
drop_audit triggers (risk: false-positive env-scope triggers
for unrelated phrases).
(b) anti-hallucination prompt clamp: "If a topic's env block is
missing/empty, say you don't have live data instead of
answering from general knowledge." (risk: bot apologizes
every other message.)
Per the "STOP if any path fails" instruction, this commit does NOT
claim verification done; the report at
v0.7-firetracker-phase4.md has the full table + per-row mesh-receiver
wire + per-failure root cause analysis.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| f69a05dd6d |
feat(v0.7-fire-tracker-4): fix LLM DM path + daily fire digest + ?status queries
Phase 4 of FIRMS+WFIGS fusion. Foundation: every direct LLM DM
mentioning a fire/weather/quake/avalanche/flood/etc. keyword was
failing silently in prod with UnboundLocalError because router.py
referenced scope_type before assigning it. With that path restored,
two new features land: a twice-daily fire-digest scheduled broadcast
(LLM-rendered) and a ?status <fire_name> on-demand mesh-DM intent.
BUG-FIX ROOT CAUSE (Job Zero):
router.py:745 ("if should_inject_mesh and scope_type == 'env'") read
`scope_type` -- a local variable bound only at line 761 inside an
unrelated `if self.source_manager and self.mesh_reporter` block.
Python's lexical scoping made scope_type a local of the whole
generate_llm_response function, so reading it before the assignment
raised UnboundLocalError on every env-keyword DM. The exception
propagated to main.py's outer except, no response went out, bot
appeared dead on fire/weather/quake/avalanche/flood queries.
Evidence (synthetic in-process trace against the live container's
config + GoogleBackend):
"are there any fires near me?" -> UnboundLocalError (pre-fix)
-> real LLM answer (post-fix)
"Yes, there are a few active
fires reported in the region.
Salmon River: 4,200 acres, 78%
contained. Cache Peak: 1,847
acres, 23% contained. ..."
"what's the weather?" -> UnboundLocalError (pre-fix)
-> "I do not have current weather
information. I can tell you
about active fires, stream gauge
levels, space weather, or band
conditions if you'd like." (post-fix)
"hi there" -> normal LLM answer in both cases
Fix: hoist `scope_type, scope_value = self._detect_mesh_scope(query)`
to right after `should_inject_mesh` is computed; remove the
now-duplicate detection inside the source_manager block.
Secondary mitigation: tightened the "do not invent commands" prompt
with an explicit "if no list appears above, you have NO commands"
clause. The prior prompt told the LLM "answer based on the command
list provided below" without always providing one, so the LLM
hallucinated plausible-sounding !commands (the "use ! commands"
canned-looking response Matt was seeing on non-env queries).
PHASE 4 FEATURES:
1. Fire-digest scheduler (meshai/notifications/scheduled/fire_digest.py).
Modeled after BandConditionsScheduler. Runs in the pipeline's
start_pipeline coroutine alongside band_conditions + reminders.
On each slot (default 06:00 + 18:00 America/Boise):
- Queries active fires (tombstoned_at IS NULL) + last 24h passes.
- Builds a prompt asking for a single mesh-wire summary <= 200
chars.
- Calls the LLM (Google/Anthropic/OpenAI per config).
- Falls back to a terse "Fires today (N): Cache Peak 1847 ac;
Twin Peaks 320 ac; +N more" line when the LLM is unavailable.
- Dispatches via dispatcher.dispatch_scheduled_broadcast (same
path band_conditions uses).
Idempotency: v16.sql adds fire_digest_broadcasts(slot_epoch PK,
sent_at, summary, source). INSERT OR IGNORE pattern blocks the same
slot firing twice (matters when container restarts mid-day).
2. ?status <fire_name> on-demand intent (router.py).
Before falling through to the LLM, route() now checks for a leading
"?status" / "status:" sigil or natural-language triggers like
"how is X fire?". On match:
- _lookup_fire_fuzzy walks fires by exact -> startswith ->
contains -> word-overlap (skipping a trailing " fire" word so
"cache peak fire" matches "Cache Peak"). Active fires rank
above tombstoned ones.
- _build_fire_status_context composes a small context block
(name, acres, containment, county/state, last 3 passes with
drift).
- The query is REWRITTEN into an LLM prompt with that context
inlined; the rest of the normal LLM path (chunking, history,
summary persistence) runs unchanged.
Live verification: "?status Cache Peak" -> "The Cache Peak fire is
1,847 acres and 23% contained. It's located in Probe / ID.";
"?status Salmon" -> word-overlap matches "Salmon River" ->
"The Salmon River fire is 4,200 acres and 78% contained, located
in Probe / ID."
3. adapter_config rows (GUI-editable per CONFIG-vs-CODE rule):
fires.digest_enabled = true (master toggle)
fires.digest_schedule = ["06:00", "18:00"]
fires.digest_timezone = "America/Boise"
fires.digest_max_chars = 200
Schema (v16.sql):
- fire_digest_broadcasts(slot_epoch INTEGER PK, sent_at, summary,
source) with source in {'llm', 'fallback_terse', 'skipped_no_fires'}.
- Index on sent_at for ops queries.
Tests (tests/test_fire_tracker_phase4.py, 10 cases all green):
- Regression guard: scope_type appears as an assignment BEFORE the
env_reporter check (prevents the UnboundLocalError from coming back).
- adapter_config seeds all 4 digest keys with expected defaults.
- render_digest returns ('', 'no_fires') when no active fires.
- render_digest falls back to terse line when LLM is None; wire fits cap.
- render_digest with a stub LLM returns ('<llm text>', 'llm').
- _lookup_fire_fuzzy: exact, "X fire" trim, word-overlap, no-match.
- _maybe_rewrite_status_query: builds context-bearing prompt; returns
None on non-status queries.
Combined suite: 60 passed in 3.81s across phase1+phase2+phase3+phase4
+or-arch+include-roundtrip.
Live verification on CT108 after rebuild:
- v16 migration applied (schema_meta=16, no Traceback in 3 min).
- FireDigestScheduler started: enabled=True schedule=['06:00','18:00']
tz=America/Boise.
- LLM DM probe (real Gemini) returns real answers on env queries
(Bug A fixed end-to-end).
- ?status Cache Peak + ?status Salmon return fire-specific summaries.
- render_digest with real LLM returns source=llm + non-empty wire.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 31e543ca04 |
feat(v0.7-fire-tracker-3): spotting detection -- pixels beyond perimeter trigger immediate broadcast
Phase 3 of FIRMS+WFIGS fusion. v15.sql adds perimeter_geojson to fire_passes + last_spotting_broadcast_at to fires. FIRMS handler computes convex hull of each pass on pass-boundary close; attributed pixels >= 1.5 mi (configurable) from previous-pass perimeter emit wildfire_spotting broadcast. Cooldown 1h between spotting broadcasts per fire so rapid embers do not spam. wildfire_spotting category at immediate severity -- spotting is the highest-actionable fire signal (spread beyond perimeter). All thresholds GUI-editable. Phase 4 (LLM summaries + on-demand queries) deferred. Schema (v15.sql): - fire_passes gains perimeter_geojson TEXT (nullable; populated by _close_prev_perimeter at boundary). GeoJSON Polygon, single outer ring in (lon, lat) order per RFC 7946, closed (first == last). - fires gains last_spotting_broadcast_at REAL (per-fire cooldown latch). Index (irwin_id, last_spotting_broadcast_at) for the cooldown probe. adapter_config (defaults.py REGISTRY): - fires.spotting_distance_threshold_mi = 1.5 (float). Matches design doc Phase 3 spec; design doc open question #6 lists this as TBD pending real spotting observation data. - fires.spotting_cooldown_seconds = 3600 (int, 1h). Suppresses rapid-ember spam from a single satellite pass. ALERT_CATEGORIES (notifications/categories.py): - wildfire_spotting: immediate / fire. Highest fire severity -- spotting represents fire spread BEYOND the existing perimeter, the most actionable detection signal. FIRMS handler (central/firms_handler.py): - _handle_pass_boundary now closes the prior pass's perimeter (convex hull of fire_pixels via Andrew's monotone chain) on the first boundary; subsequent in-pass pixels reuse the stored hull. - _check_spotting runs for every attributed pixel: looks up the most recent CLOSED pass (perimeter_geojson NOT NULL AND pass_id != current), point-in-polygon test, vertex-distance approximation per design doc Q (sparse pixels make edge projection overkill at VIIRS 375 m resolution), per-fire cooldown gate. - Priority order: spotting (immediate) > growth (priority) > cluster (priority) > halt (routine). Spotting preempts growth at the same pixel because immediate > priority. - Helpers: _convex_hull (Andrew's monotone chain), _hull_to_geojson (RFC 7946 Polygon), _point_in_polygon (ray casting), _close_prev_perimeter, _check_spotting, _prev_has_perimeter. Wire string: - wildfire_spotting: "🔥 Possible spotting <dist:.1f> mi <dir> of <incident_name> perimeter" -- direction is 8-way bearing from the previous pass's centroid to the spotting pixel. Tests (tests/test_fire_tracker_phase3.py, 11 cases all green): - Pass close stamps perimeter_geojson as a closed Polygon (6 hex vertices -> 7-entry closed ring). - Pixel 2 mi NE of perimeter fires spotting with distance in the 1.0..2.5 mi band (vertex-distance approximation) and direction NE. - Pixel inside perimeter -> NO spotting wire. - Second spotting candidate within 1h cooldown -> suppressed. - Past-cooldown spotting fires again. - Convex hull / point-in-polygon / GeoJSON round-trip helper tests. - adapter_config seed for both new fires.* keys. - wildfire_spotting category registered with immediate severity. - 49 tests green across phase1/phase2/phase3/or-arch/include-roundtrip. Live verification on CT108 after rebuild: - v15 migration applied (schema_meta=15, no Traceback in 3 min). - Container healthy. Synthetic 25-pixel probe (PROBE-V07P3-*, cleaned up after): - Pass A: 20 pixels in a ~0.3 mi circle. Perimeter stored on boundary. - Pass B: 5 pixels at distances 0.5/1.0/2.0/5.0/7.0 mi from center. Observed wires: "🔥 Possible spotting 1.7 mi NE of Probe Spotting Fire perimeter" "🔥 Possible spotting 4.7 mi NW of Probe Spotting Fire perimeter" (Plus a Phase 2 growth wire on the first pass B pixel -- documented side effect: single-pixel pass B centroid shows 0.5 mi drift from pass A.) - 7.0 mi E pixel: outside 5 mi spread, no broadcast (cluster check found no co-located unattributed pixels). Cleanup confirmed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| f5c566c6c0 |
feat(v0.7-fire-tracker-2): movement analysis -- growth + halt detection
Phase 2 of FIRMS+WFIGS fusion. v14.sql adds fire_passes table for per-satellite-pass centroid tracking + drift computation. FIRMS handler now detects pass boundaries (satellite + time bucket), computes pass centroid (median of pass pixels), Haversine drift from previous pass, bearing to 8-way direction, mi/h speed. Drift >= 0.5 mi (configurable) emits wildfire_growth broadcast with wire including movement vector and nearest-town context. Halt detection: fire with no new pixels for >=12h (configurable) emits wildfire_halted broadcast (routine). Two new ALERT_CATEGORIES: wildfire_growth (priority), wildfire_halted (routine). All thresholds GUI-editable via adapter_config.fires.*. Phase 3 (spotting) and Phase 4 (LLM summaries) deferred to subsequent commits. Schema (v14.sql): - fire_passes table (irwin_id FK CASCADE, pass_id, pass_centroid_lat/lon, pixel_count, total_frp, pass_started_at, pass_ended_at, drift_mi_from_prev, drift_direction, drift_mi_per_hour). PRIMARY KEY (irwin_id, pass_id) so the UPSERT path is cheap; secondary index on (irwin_id, pass_ended_at) for the prev-pass lookup + halt counter. - fires gains last_pass_id, last_pass_at, halt_broadcast_at columns. halt_broadcast_at is latched per halt event; the detector filter (halt_broadcast_at IS NULL OR halt_broadcast_at < last_pass_at) reopens eligibility automatically when an idle fire receives a new attributed pixel that advances last_pass_at. adapter_config (defaults.py REGISTRY): - fires.growth_drift_threshold_mi = 0.5 (float). Per-pass centroid drift at or above this fires wildfire_growth. 0.5 mi matches the design doc Phase 2 spec and is roughly 2x the VIIRS 375m pixel size (i.e., detectable as more than centroid jitter). - fires.halt_passes_threshold = 2 (int). Documented intent; the operational rule uses halt_minimum_seconds below as the time gate because per-satellite pass-count enforcement would require modeling the global VIIRS schedule per satellite. The 12h gate subsumes it (4 passes/day in Idaho). - fires.halt_minimum_seconds = 43200 (int, 12h). ALERT_CATEGORIES (notifications/categories.py): - wildfire_growth: priority/fire. FIRMS handler tags data["category"] + data["severity"] on the pass-boundary path when drift >= threshold. - wildfire_halted: routine/fire. Halt detector tags data["category"] + data["severity"] when a fire transitions to idle for >=12h. FIRMS handler (central/firms_handler.py): - The Phase 1 attribution branch now passes through _handle_pass_boundary(): UPSERT fire_passes row for the current (irwin_id, pass_id) with median centroid + pixel count + total FRP + min/max acq_time; lookup the prior pass; compute drift mi + 8-way direction + mi/h speed and write them into the current pass row (only the FIRST boundary fills these; subsequent in-pass pixels COALESCE keep them stable). Update fires cursor (last_pass_id, last_pass_at) and current_centroid_lat/lon to the latest pass centroid -- this overrides Phase 1's 24h all-pixels median for fires that have pass data. - Growth wire emitted ONLY at the boundary (last_pass_id != current, prev pass exists, drift >= threshold). Subsequent in-pass pixels stay silent because pass_id == last_pass_id. - _maybe_emit_halt runs as a final fallback when neither growth nor cluster has fired. SELECT one fire matching the halt criteria, stamp halt_broadcast_at, return the wire. The fallback ordering is growth > cluster > halt so a busy fire's growth broadcast doesn't starve a quiet fire's halt. - New helpers: _bearing() (great-circle initial bearing, deg CW from N), _direction_8() (compass 8-way mapping with +/-22.5 deg sectors). Wire strings: - wildfire_growth: `🔥 <incident_name> moving <dir> <speed:.1f> mi/h ~<dist_to_nearest_town:.1f> mi from <nearest_town>`. nearest_town via meshai.central_normalizer.nearest_town (same Photon-backed cache that wfigs_handler uses); failure falls back to bare "moving <dir> <speed> mi/h". - wildfire_halted: `🔥 <incident_name> no growth in <hours>h`. Tests (tests/test_fire_tracker_phase2.py, 10 cases all green): - 2-pass attribution with pass2 1.0 mi N of pass1 -> drift=1.0, direction='N', mi/h computed, growth wire returned, data tagged. - Drift below threshold (0.3 mi) -> NO growth broadcast; pass row still records the (sub-threshold) drift for ops visibility. - Halt detector: last_pass_at 14h ago -> fires once, halt_broadcast_at stamped. - Re-run halt detector with halt latched -> NO second broadcast. - Halt re-eligibility: halt_broadcast_at < last_pass_at -> eligible again (a resurrected then re-idled fire). - Bearing + direction round-trip across all 8 cardinals. - Direction sector boundary (22.5/67.5 deg) correctness. - adapter_config seed for 3 new fires.* keys. - Two new ALERT_CATEGORIES registered. - 5-pixel single-pass aggregate (pixel_count, total_frp sum, median centroid, started/ended_at min/max). Phase 1 test fix: - tests/test_fire_tracker_phase1.py::test_centroid_recomputes_as_median_across_passes retimed to 12:00/12:10/12:20 so all 3 pixels land in one N20 bucket. Phase 2 makes current_centroid_* the per-pass median (latest pass overrides Phase 1's 24h median); the same-pass shape preserves the original median-computation intent. 39 total tests green across phase1/phase2/or-arch/include-roundtrip. Live verification on CT108 after rebuild: - v14 migration applied (schema_meta version=14, no Traceback in 3 min). - adapter_config.fires.growth_drift_threshold_mi = 0.5 - adapter_config.fires.halt_passes_threshold = 2 - adapter_config.fires.halt_minimum_seconds = 43200 - Container healthy. Synthetic 100-pixel probe inside prod container (PROBE-V07P2-*, cleaned up after): - Pass A (50 pixels @ 12:00-12:25, N20 bucket 329768): centroid (44.30000, -115.50000), pixel_count=50, total_frp=975.0, drift=NULL (first pass). - Pass B (50 pixels @ 18:00-18:25, N20 bucket 329772, centered 1.2 mi NE of A): centroid (44.31230, -115.48282), pixel_count=50, total_frp=975.0, drift_mi_from_prev=1.1703 (~design target 1.2 mi with -0.03 mi rounding), drift_direction="NE", drift_mi_per_hour=0.209 (1.17 mi over 5.5h between pass ends). - Growth wire: "🔥 Probe Movement Fire moving NE 0.2 mi/h, ~13.0 mi from Long Creek Summit Home" (Photon nearest-town anchor populated successfully). - Exactly ONE growth broadcast (first pixel of pass B); 99 other pixels stayed silent. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| dd8e687aca |
feat(v0.7-fire-tracker-1): registry correlation + 2 new categories
Phase 1 of the FIRMS+WFIGS fusion design doc. v13.sql adds fire_pixels table for per-fire pixel history + spread_radius_mi/current_centroid_*/ last_hotspot_at on fires. FIRMS handler now attributes incoming pixels to fires via point-in-circle within configurable radius (default 5 mi), updating per-fire centroid as median of recent pixels. Unattributed pixels go through a cluster detector: 3+ pixels within 1 mi within 60 min triggers a single unattributed_hotspot_cluster broadcast (Possible new fire). Two new ALERT_CATEGORIES: wildfire_declared (priority, WFIGS first-sight) and unattributed_hotspot_cluster (priority, FIRMS cluster). All thresholds GUI-editable via adapter_config.fires.* and adapter_config.firms.*. Phases 2-4 (movement analysis, spotting, LLM summaries) deferred to subsequent commits. Schema (v13.sql): - fire_pixels table (irwin_id FK CASCADE, acq_time, lat/lon, frp, satellite, pass_id, attributed_at). Indexed on (irwin_id, acq_time) for centroid queries + on (acq_time) for Phase 2. - fires gains spread_radius_mi (nullable; NULL => use global default), current_centroid_lat/lon (median of last 24h pixels, distinct from the WFIGS-declared anchor lat/lon), last_hotspot_at (Phase 2 halt detector). - firms_pixels gains attributed_at + cluster_broadcast_at + compound index on (attributed_at, cluster_broadcast_at, acq_time) for the cluster query. adapter_config (defaults.py REGISTRY + ADAPTER_META): - fires.spread_radius_mi_default = 5.0 (float) - firms.cluster_min_pixels = 3 (int) - firms.cluster_max_radius_mi = 1.0 (float) - firms.cluster_time_window_minutes = 60 (int) - ADAPTER_META["fires"] meta block (display_name + description). ALERT_CATEGORIES (notifications/categories.py): - wildfire_declared: priority/fire. WFIGS handler tags data["category"] on cases (i)+(ii) [INSERT or row-exists-but-never-broadcast]; case (iii) Update keeps the existing wildfire_incident category. - unattributed_hotspot_cluster: priority/fire. FIRMS handler tags data["category"] + data["severity"] when emitting the cluster wire. FIRMS handler (central/firms_handler.py): - Unchanged storage path: filter, INSERT OR IGNORE into firms_pixels. - New _attribute_or_cluster() runs on every newly-stored pixel (dedup hits skip -- the original insert had its shot already). - Attribution: bbox prefilter on fires.tombstoned_at IS NULL, then exact Haversine to fires(current_centroid_lat ?? lat, current_centroid_lon ?? lon) inside spread_radius_mi (per-fire ?? global default). Multi-match resolves to nearest (design doc Q2). On match: INSERT fire_pixels, UPDATE firms_pixels.attributed_at, recompute centroid as median of last 24h pixels for this fire. - Cluster: on attribution miss, query firms_pixels WHERE attributed_at IS NULL AND cluster_broadcast_at IS NULL AND acq_time > NOW-window. If count >= cluster_min_pixels, fire the cluster wire and stamp cluster_broadcast_at on every member so a 4th arrival cannot re-fire. WFIGS handler (central/wfigs_handler.py): the existing prefix=New branches (i)+(ii) now set data["category"]="wildfire_declared". Existing _render() unchanged. Wire strings: - wildfire_declared: re-uses _render(prefix="New") -- emoji + name + type + anchor + acres + containment + coords. - unattributed_hotspot_cluster: _render_cluster_wire() emits "Possible new fire: <N> hotspots within <r> mi @ <lat>,<lon> (combined <total_frp> MW)". Tests (tests/test_fire_tracker_phase1.py, 10 cases all green): - Pixel within radius -> attribution + centroid + last_hotspot_at. - Centroid recomputes as median across multiple passes. - Pixel outside radius -> NO attribution + stays unattributed. - 3 unattributed within 1 mi within 60 min -> cluster broadcast fires exactly once, all 3 stamped cluster_broadcast_at. - 4th pixel in the same footprint -> NO second broadcast (existing 3 are stamped so SQL filter excludes them). - 5th-7th pixels 2h later -> form a NEW cluster (window prune fires). - WFIGS first-sight tags data["category"]="wildfire_declared". - WFIGS Update branch does NOT retag wildfire_declared. - New adapter_config rows seeded on init_db. - ALERT_CATEGORIES contains both new entries with correct toggle/severity. Live verification on CT108 after rebuild: - v13 migration applied (schema_meta version=13, no Traceback). - adapter_config.fires.spread_radius_mi_default = 5.0 - adapter_config.firms.cluster_min_pixels = 3 - adapter_config.firms.cluster_max_radius_mi = 1.0 - adapter_config.firms.cluster_time_window_minutes = 60 - fires gains 4 new columns; firms_pixels gains 2 new columns; fire_pixels table created. - Container healthy, FIRMS pixels continue arriving (126 pre-deploy). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 3351e7b444 |
fix(v0.6-tail-4): register !include YAML tag constructor in config loader -- closes prod PUT 500
Pre-existing issue surfaced by v0.6-tail-3: prod config at
/data/config/config.yaml:82 uses !include to compose from separate
files, but the loader had no constructor registered so PUT
/api/config/<section> returned 500 with could-not-determine-constructor
when the section save path round-tripped YAML. This adds the !include
constructor (read path) + preserves the include structure on write so
multi-file config layouts work end-to-end via the GUI. The runtime
config behavior is unchanged; this only fixes the PUT-and-round-trip
case.
Implementation note: the read-only runtime path
(_load_yaml_with_includes) already had a working !include constructor
that recursively substitutes content. The bug was specifically in
save_section() -- it used plain yaml.safe_load() to re-read the target
file off disk for secret-ref preservation and for in-place section
updates. When target_file == "config.yaml" that file contains !include
directives for OTHER sections, and safe_load died on them.
Adding a third constructor that substitutes !include on save would
have flattened the multi-file layout to a single file the first time
anyone PUT an inline section. Instead this commit adds a preserve-mode
loader/dumper pair:
- _load_yaml_preserve() returns an Include("path") sentinel for each
!include node instead of recursing into the referenced file.
- _dump_yaml_preserve() re-emits Include("path") back to disk as
`!include path`. (PyYAML auto-quotes when the scalar contains a
period, so the on-disk form is `!include 'foo.yaml'`; both forms
are equivalent at parse time.)
- save_section()'s three yaml-touching sites (the secret-ref raw
read, the existing-target read, and the final dump) now use these
helpers. Local.yaml stays on yaml.safe_load/dump because local.yaml
never contains !include.
The runtime loader is untouched, so boot-time config still substitutes
includes and Config dataclasses see real values. Only the GUI's
section-save path round-trips through the preserve helpers.
Tests (tests/test_include_roundtrip.py, 8 cases):
- Runtime loader still substitutes !include content (regression guard)
- Preserve loader returns Include() sentinels
- Preserve dumper re-emits `!include path` (tolerant of PyYAML
auto-quoting)
- Read -> write -> read identity through the preserve helpers
- save_section('bot', ...) on a config.yaml that uses !include for
sibling sections succeeds AND leaves the includes intact on disk
(this is the exact prod PUT 500 case from v0.6-tail-3)
- After save_section, the runtime loader re-resolves all !include
files AND sees the saved change to the inline section
- save_section on a dedicated file (env_feeds.yaml) writes only that
file; config.yaml's !include directives are untouched
- Runtime cycle detection still trips on A!include->B!include->A
Live verification on CT108 after rebuild:
PUT /api/config/bot {"name":"AIDA","owner":"Malice","respond_to_dms":true,"filter_bbs_protocols":true}
-> HTTP 200 {"saved":true,"restart_required":false,"changed_keys":[]}
/data/config/config.yaml retains all 7 !include directives
(meshtastic, mesh_sources, mesh_intelligence, environmental,
notifications, llm, dashboard)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| f89e9c11fb |
feat(v0.6-tail-3): enforce OR-not-AND continuously -- close USGS direct-lookup leak + flag environmental config changes as restart-required
Gap 1 -- env_routes.lookup_usgs_site no longer creates a temporary
USGSStreamsAdapter to hit USGS.gov directly. When the env_store has no
native usgs adapter (because usgs.feed_source != native), the endpoint
returns HTTP 404 with a body that says "site lookup unavailable in
central-feed mode; values must be entered manually or sourced from
Central". This closes the AND-mode anti-pattern Central's v0.10.2
report flagged: meshai was in central-feed mode for usgs but the
lookup helper would still call USGS.gov directly the first time the
dashboard opened the Add-Gauge form.
Gap 2 -- config_routes.RESTART_REQUIRED_SECTIONS gains "environmental"
and the PUT handler now diffs the section before/after, returning
{saved, restart_required, changed_keys}. restart_required is true only
when there are actual changes AND the section is in the restart-required
set, so a no-op PUT to environmental never raises a false alarm.
Frontend wiring:
- New RestartBanner component (yellow top-of-main banner) listens to a
meshai:restart-required CustomEvent + cross-tab storage event,
persists across navigations via localStorage, shows changed_keys
preview + Restart-now button (POSTs /api/system/restart) + dismiss.
- Layout.tsx mounts <RestartBanner /> above {children} so it surfaces
on every page.
- Config.tsx saveSection() now calls notifyRestartRequired(changed_keys)
alongside its existing setRestartRequired(true) when the API flags
the section.
- GaugeSites.tsx probes /api/config/environmental at mount and shows a
"USGS lookup" button next to the site_id input. The button is
disabled with an explanatory tooltip when usgs.feed_source != native,
and gracefully renders the 404 detail when the API returns 404 in
central-feed mode -- enter-manually UX, no silent fallback.
Tests -- tests/test_or_arch_continuous.py (11 cases, all passing):
- USGS lookup 404 with no env_store / no native usgs adapter
- 502 on native-adapter exception
- 200 + payload on native-adapter happy path
- environmental in RESTART_REQUIRED_SECTIONS
- PUT environmental with changed feed_source -> restart_required:true
+ changed_keys list including foo.feed_source dotted path
- PUT bot (non-restart section) -> restart_required:false
- No-op PUT to bot / environmental -> restart_required:false, empty
changed_keys
- _diff_keys helper unit tests (nested dicts, list-element changes)
Why this matters: per the Spokane post-mortem and Central's v0.10.2
response, both sides need belt-and-suspenders against transient
AND-modes. meshai's static OR enforcement at env_store boot is the
runtime guard; this commit makes the GUI honor it continuously --
the lookup helper can't sneak past it any more, and the user is told
explicitly that an environmental config change does not take effect
until the container restarts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 24763947c3 |
fix(v0.6-tail-2): finish Quiet Hours rip (Reference.tsx + dashboard bundle rebuild)
Phase 2 (
|
|||
| 566b06de06 |
feat(v0.6-tail): close 5 v0.6-phase1-complete.md follow-ups
(1) Auto-call refresh-toggles on PUT /api/config/notifications
meshai/dashboard/api/config_routes.py adds register_config_routes_hooks(app)
which registers a FastAPI HTTP middleware: on any 2xx PUT whose path
matches /api/config/notifications or /api/config, the middleware
invokes _refresh_toggle_filter(app) which reaches into app.state.bus._
pipeline_components["toggle_filter"] and calls .refresh(app.state.config).
The dashboard no longer has to remember to ping POST /api/notifications/
refresh-toggles after a toggle change. The explicit endpoint stays for
backwards-compat.
(2) env_reporter block-size cap moved to adapter_config
New registry row pipeline.env_reporter_block_chars (int, default 3000).
meshai/notifications/env_reporter.py replaces the hardcoded
_BLOCK_MAX_CHARS = 3000 with _DEFAULT_BLOCK_MAX_CHARS (the fallback) +
a _block_cap() helper that reads from adapter_config on every slice.
Mutating the row via PUT /api/adapter-config takes effect on the next
env_reporter call -- no restart.
(3) Bulk-import endpoint for gauge_sites
meshai/dashboard/api/gauge_sites_import.py adds
POST /api/gauge-sites/import with two paths:
format=csv -- expects "data" (CSV text with header row matching
gauge_sites columns: site_id, gauge_name, lat, lon,
and optionally action_ft/flood_minor_ft/
flood_moderate_ft/flood_major_ft/enabled). UPSERT
via ON CONFLICT(site_id) DO UPDATE. Returns
{inserted, updated, skipped}.
format=nws-ahps -- expects "wfo" (list of WFO codes). Fetches
water.weather.gov/ahps2/index.php?wfo=<WFO> for each,
regex-parses gauge links, then fetches up to 50
gauge detail pages per request and regex-parses
lat/lon + four threshold values. Best-effort; rows
stored under "AHPS-<gauge_id>" so they dont collide
with USGS-* ids. Returns the same shape plus
detail_fetched + errors list.
Frontend (dashboard-frontend/src/pages/GaugeSites.tsx) gains a
Import button + modal with two tabs (Paste CSV / Scrape NWS-AHPS)
rendered via an ImportModal component. CSV tab has a 48-row textarea
with the column-header hint inline; AHPS tab has a comma-separated WFO
input defaulting to BOI. Both submit via fetch() and show the JSON
response inline. Invalidates the curation cache server-side on any
successful insert/update so nwis_handler sees the new gauges on its
next call.
(4) WFIGS tombstone column -- CORRECTNESS
v12.sql adds fires.tombstoned_at REAL (nullable) + idx_fires_tombstoned_at.
meshai/central/wfigs_handler.py: the tombstone branch
(kind=="wfigs_tombstone") UPDATE fires SET tombstoned_at=COALESCE(
tombstoned_at, ?) so the first tombstone-time wins (idempotent against
repeated tombstone envelopes).
meshai/notifications/reminders/__init__.py: the wfigs tombstone
termination condition now checks row["tombstoned_at"] IS NOT NULL.
Reminders correctly STOP for closed fires -- before this change the
8h cadence would have kept Active: broadcasts going indefinitely past
a WFIGS removal.
SCHEMA_VERSION 11 -> 12.
(5) Delete INCIDENT_BROADCAST_HEARTBEAT_S
meshai/central/incident_handler.py: removed the dead constant
(v0.5.9 REVISED dropped the heartbeat path but left the constant
imported-but-never-read).
tests/test_incident_handler.py: removed the orphan
test_i_8h_heartbeat_triggers_update test (asserted None, used the
deleted constant for time arithmetic) and the stray import line.
Tests (tests/test_tail_followups.py, 16 cases):
- middleware fires refresh on PUT /api/config/notifications (200), does
NOT fire on PUT /api/config/llm
- env_reporter _block_cap() default 3000; mutate via PUT, invalidate,
next read returns the new cap
- CSV import inserts new rows, updates existing, skips bad rows,
rejects missing required columns, rejects bad format
- AHPS index parser extracts (gauge_id, name) from realistic HTML
- AHPS detail parser extracts lat/lon + four thresholds from realistic
HTML
- fires has tombstoned_at column after migrations
- wfigs tombstone branch stamps tombstoned_at
- ReminderScheduler skips a fire whose tombstoned_at is NOT NULL
- ReminderScheduler still fires for a fire whose tombstoned_at IS NULL
- INCIDENT_BROADCAST_HEARTBEAT_S no longer importable
Foundation/API test counts bumped:
REGISTRY 58 -> 59 (+ env_reporter_block_chars)
schema_meta v11 -> v12
Test count: 844 -> 859 (+16 new, -1 deleted dead test). 0 regressions.
|
|||
| 3a410d5087 |
feat(v0.6-phase3): reminder system + schema split + NWS dedup relaxation
Third broadcast type Active: clock-driven re-broadcasts of still-live
events at human-scale cadences. WFIGS fires 8h, itd_511 work zones daily
8 AM Mountain, SWPC G-storms 8h. NWS is NOT a clock reminder -- instead
the per-CAP-id dedup is relaxed to allow re-broadcast if >3h since last.
Schema split first_broadcast_at + last_broadcast_at on all reminder-
eligible tables. Wire prefix logic: New (first sight), Update (WFIGS
material change), Active (clock reminder). All cadences, channels, day-
of-week patterns, timezones, and termination conditions GUI-editable
from day one via the existing adapter_config editor. Termination:
tombstone OR containment_100 OR end_date_passed (no max-count). Quiet
hours not respected -- ripped out in Phase 2.
Schema (v11.sql):
- ALTER TABLE fires|nws_alerts|traffic_events|quake_events|swpc_events|
gauge_readings ADD COLUMN first_broadcast_at REAL
- Backfill: UPDATE ... SET first_broadcast_at = last_broadcast_at
WHERE last_broadcast_at IS NOT NULL
- ALTER TABLE adapter_meta ADD COLUMN reminder_enabled INTEGER NOT NULL
DEFAULT 0
- UPDATE adapter_meta SET reminder_enabled=1 WHERE adapter IN
('wfigs', 'swpc') -- itd_511_work_zone is a new meta row seeded
with reminder_enabled=1
- SCHEMA_VERSION 10 -> 11
Handler commit-callbacks (wfigs/nws/quake/swpc/incident):
- UPDATE ... SET last_broadcast_at=?, first_broadcast_at=COALESCE(
first_broadcast_at, ?) -- first_broadcast_at stamped once, never
overwritten
NWS handler (meshai/central/nws_handler.py):
- _render() gains a prefix kwarg
- After-first-broadcast branch: when (now - last_broadcast_at) >=
adapter_config.nws.duplicate_allowed_after_seconds (default 10800
= 3h), allow the re-broadcast with prefix=Active. Under the
window, suppress as before. The commit callback continues to
update last_broadcast_at.
ReminderScheduler (meshai/notifications/reminders/__init__.py):
- Async loop, ticks every 60s
- Each tick: SELECT adapter FROM adapter_meta WHERE reminder_enabled=1
- Per adapter, load reminders_<adapter> config from adapter_config
(cadence_kind, cadence_value, channels, terminate_when, dow_mask,
timezone)
- Interval cadence: rows where last_broadcast_at <= now - cadence_value
- Clock cadence: localizes now to configured tz, finds slots that
just passed in the last tick window, gated by dow_mask
- Termination conditions checked per adapter:
wfigs.containment_100 -> current_contained_pct >= 100
wfigs.last_event_age_24h -> last_event_at older than 24h
swpc.end_date_passed -> payload_json end_time in past
itd_511_work_zone.end_date_passed -> traffic_events.end_at in past
- Active: prefix on every emitted wire; dispatcher.dispatch_scheduled_
broadcast() honors cold-start grace, bypasses toggle path
- On success, last_broadcast_at = now; first_broadcast_at preserved
Launched from notifications/pipeline/__init__.py:start_pipeline()
alongside BandConditionsScheduler.
adapter_config registry (+15 new keys, 43 -> 58):
- reminders_wfigs.cadence_kind/cadence_value/channels/terminate_when
- reminders_swpc.cadence_kind/cadence_value/channels/terminate_when
- reminders_itd_511_work_zone.cadence_kind/cadence_value/channels/
dow_mask/timezone/terminate_when
- nws.duplicate_allowed_after_seconds
adapter_meta (+4 rows, 15 -> 19):
- reminders_wfigs, reminders_swpc, reminders_itd_511_work_zone
(pseudo-adapters carrying the reminder config)
- itd_511_work_zone (reminder target row; reminder_enabled=1)
- reminder_enabled flag added to wfigs/swpc (existing rows updated by
v11.sql) and to itd_511_work_zone seed.
Tests (tests/test_reminders.py, 10 cases):
- wfigs reminder fires past 8h cadence, stamps last_broadcast_at,
preserves first_broadcast_at
- reminder skipped within cadence
- reminder skipped when containment_100, last_event_age_24h
- swpc reminder fires (interval)
- work_zone clock reminder fires at 08:00 Mountain on enabled DOW
- work_zone reminder skipped when end_date_passed
- work_zone reminder skipped outside slot window
- reminder_enabled=0 suppresses all reminders for that adapter
tests/test_nws_dedup_relaxation.py (5 cases):
- First sighting renders without Active: prefix
- Re-broadcast within 3h suppressed
- Re-broadcast after 3h allowed with Active: prefix
- adapter_config.nws.duplicate_allowed_after_seconds override takes
effect (1h window verified)
- First sighting stamps first_broadcast_at=committed_at,
last_broadcast_at=committed_at; 4h later broadcast stamps
last_broadcast_at only, first_broadcast_at preserved
Test count: 829 -> 844 (+15 new, 0 regressions). Foundation tests
updated for new counts (REGISTRY=58, ADAPTER_META=19, schema=v11).
|
|||
| b948ed775f |
feat(v0.6-phase2): rip out quiet hours entirely -- dashboard toggle, config schema, pipeline checks. Per Matt's repeated feedback (saved as feedback-quiet-hours-trash.md): silent is better than ugly, mesh users who need a fire alert at 3 AM need it at 3 AM. No replacement.
Backend removals:
meshai/config.py
- NotificationRuleConfig.override_quiet field
- NotificationToggle.quiet_hours_override field
- NotificationsConfig.quiet_hours_enabled / quiet_hours_start /
quiet_hours_end fields
- _default_toggles() no longer sets quiet_hours_override=True
- rule migration helper no longer copies override_quiet
meshai/notifications/router.py
- self._quiet_enabled / _quiet_start / _quiet_end instance vars
- _in_quiet_hours() method (deleted entirely)
- The dispatch-time check that suppressed non-overriding rules
during quiet hours
- 'override_quiet': False dropped from subscription rule dicts
meshai/notifications/pipeline/dispatcher.py
- _toggle_to_rule() no longer passes override_quiet=... to the
NotificationRuleConfig constructor
Test changes:
tests/test_notification_toggles.py
- RecChannel.deliver() no longer records override_quiet
- test_quiet_hours_override_immediate_only deleted (only tested the
removed feature)
Frontend removals (dashboard-frontend/src/pages/Notifications.tsx):
- The 'Enable Quiet Hours' card with its time-range inputs deleted
- 'Override Quiet Hours' per-rule toggle deleted
- 'Quiet-hours override (immediate only)' per-toggle field deleted
- quiet_hours_* fields removed from TS interfaces
- quietHoursEnabled prop + state plumbing removed from the RuleEditor
- All override_quiet: false defaults dropped from rule scaffolds
- Unused Moon icon import dropped
Verification (post-strip):
grep -rn 'quiet_hours\|override_quiet' meshai/*.py meshai/**/*.py
-> 0 hits
grep -rn 'quiet_hours\|override_quiet\|quietHours' dashboard-frontend/src
-> 0 hits
Test count: 830 -> 829 (-1: test_quiet_hours_override_immediate_only
deleted; no other regressions).
No replacement. Mesh users who need a fire alert at 3 AM need it at 3 AM.
|
|||
| 90783376e8 |
feat(v0.6-6): inhibit_state + grouper_held persistence + ToggleFilter live-reload + Inhibitor/Grouper config knobs
Closes audit doc section A.9 + finding #5. The last Phase-1 pipeline state that lived only in instance memory now writes through to SQLite, and ToggleFilter changes propagate without a container restart. Schema: v10.sql adds inhibit_state(key PK, rank, expires_at, updated_at) and grouper_held(group_key PK, event_json, hold_until_at, updated_at). Indexes on expires_at / hold_until_at support the prune sweeps. SCHEMA_VERSION 9 -> 10. Migration runner: Fixed the alphabetical-vs-numeric sort bug v10 surfaced -- the runner now sorts pending migrations by their integer version, not by filename, so v10.sql correctly applies AFTER v9.sql (was applying after v1 alphabetically, which made schema_meta stick at 9). Inhibitor (meshai/notifications/pipeline/inhibitor.py): - __init__ restores non-expired keys from inhibit_state on construct. - handle() write-throughs every (key, rank, expires_at) tuple. - _prune_expired DELETEs the same expired keys from disk. - clear() (test path) drops the table. Grouper (meshai/notifications/pipeline/grouper.py): - __init__ restores non-expired held events from grouper_held; the Event is rebuilt via Event.from_dict(json.loads(event_json)). - handle() write-throughs (group_key, event_json, hold_until_at). - tick() and flush_all() DELETE on emit. ToggleFilter (meshai/notifications/pipeline/toggle_filter.py): - new refresh(config) method re-reads config.notifications.toggles and rebuilds the enabled set. Live wiring: - meshai/dashboard/api/config_routes.py adds a POST /api/notifications/refresh-toggles endpoint that reaches into app.state.bus._pipeline_components["toggle_filter"] and calls refresh(app.state.config). The frontend pings this after PUT /api/config/notifications so toggles take effect on the next event. - meshai/main.py stashes self.event_bus on the dashboard FastAPI app.state after build_pipeline so the route can reach it. - Inhibitor.ttl_seconds and Grouper.window_seconds already read from adapter_config.pipeline.{inhibitor_ttl_seconds, grouper_window_seconds} via the v0.6-3b None-default wiring (rows seeded in v0.6-3a.1). Tests (tests/test_pipeline_persistence.py, 11 cases): - v10 tables present - Inhibitor: state persists across simulated restart; expired rows not restored; prune removes from disk; clear() wipes both. - Grouper: state persists across restart; tick() clears disk; expired rows not restored. - ToggleFilter: refresh() picks up new enabled set; refresh(None) is a no-op; disabling a family in config + refresh drops it. Test count: 819 -> 830 (+11 pipeline persistence cases + schema test bump). |
|||
| e3bf53ade4 |
feat(v0.6-4): gauge_sites + town_anchors curation tables + GUI CRUD
Closes Section A.5 (gauge_sites) and A.12 (town_anchors) of the audit
doc by lifting both Python-dict curation lists into editable SQLite
tables. Operators can add/edit/disable rows from the dashboard without
a deploy; runtime reads go through cached accessors that invalidate
when the REST API mutates state.
Schema:
v8.sql adds gauge_sites(site_id PK, gauge_name, lat, lon, action_ft,
flood_minor_ft, flood_moderate_ft, flood_major_ft, enabled, updated_at).
v9.sql adds town_anchors(anchor_id AUTOINC PK, name UNIQUE, lat, lon,
state, enabled, updated_at).
SCHEMA_VERSION 7 -> 9.
Seed (meshai/persistence/curation.py):
_GAUGE_SITES_SEED carries the original 9 Idaho rows from
IDAHO_CURATED_SITES verbatim.
_TOWN_ANCHORS_SEED carries the 29 Idaho-and-neighbor towns from
_TOWN_COORDS verbatim.
seed_gauge_sites() / seed_town_anchors() INSERT OR IGNORE -- safe to
re-run; never overwrites user edits.
Handler integration:
- meshai/central/idaho_gauge_sites.py: IDAHO_CURATED_SITES dict deleted.
lookup_site() now calls meshai.persistence.curation.lookup_gauge_site()
which reads the table. THRESHOLD_RANK, normalize_site_id, and
compute_threshold_state remain in this module (CODE per Matt s rule).
- meshai/central/nwis_handler.py drops IDAHO_CURATED_SITES from its
import list; the table-backed lookup_site() is API-compatible.
- meshai/central_normalizer.py: _TOWN_COORDS dict deleted.
_compute_distance_bearing() now calls
meshai.persistence.curation.lookup_town_anchor() with the same
lowercased-name semantics it always used.
REST API (meshai/dashboard/api/curation_routes.py):
/api/gauge-sites GET list, GET one, POST add, PUT update, DELETE
/api/town-anchors GET list, GET one, POST add, PUT update, DELETE
Every mutation calls invalidate_curation_cache() so handler reads see
the new state on the next call -- no container restart.
Dashboard (dashboard-frontend/src/pages/):
- GaugeSites.tsx: table view with Add row / Edit row inline / Delete
confirm + per-row enabled toggle. 8 columns mirror the schema.
- TownAnchors.tsx: same pattern, 5 columns. Name is lowercased on
save to match the lookup key.
- Left-nav entries "Gauge Sites" (Droplets icon) and "Town Anchors"
(MapPin icon) added to Layout.tsx; routes added to App.tsx.
Tests (tests/test_curation.py, 18 cases):
- v8/v9 tables exist
- Seed lands every row from both dicts
- Seed idempotent; never overwrites user edits
- lookup_gauge_site hits/miss, disabled rows are invisible
- lookup_town_anchor case-insensitive
- REST API: GET list, GET one, GET 404, POST add, PUT update, DELETE,
POST missing-field 400; both gauge_sites + town_anchors
- Accessor reflects API mutations after invalidate_curation_cache()
tests/test_nwis_handler.py back-compat: IDAHO_CURATED_SITES dict alias
points at _GAUGE_SITES_SEED so the existing assertion suite still passes.
tests/test_adapter_config_foundation.py schema_meta v7 -> v9 bump.
Test count: 797 -> 819 (+18 curation cases + 4 maintenance updates).
|
|||
| eb84f27941 |
feat(v0.6-5): env_reporter + router wiring + include_in_llm_context per-adapter toggle -- LLM gains read access to every adapter table via the existing mesh_reporter pre-rendered prompt-injection pattern
Closes audit doc Section C. The LLM can now answer "any fires near me?",
"how are band conditions?", "why didnt I hear about that quake?"
without any tool-use / MCP / SQL pass-through -- via the same prompt-
injection contract mesh_reporter uses.
env_reporter (meshai/notifications/env_reporter.py):
- EnvReporter class with build_env_summary / build_fires_detail /
build_alerts_detail / build_quakes_detail / build_traffic_detail /
build_gauges_detail / build_swpc_detail / build_drop_audit / build_all
- Reads from fires + firms_pixels + nws_alerts + quake_events +
traffic_events + gauge_readings + swpc_events +
band_conditions_broadcasts + event_log + dispatcher_state
- Each build_*_detail() checks adapter_meta.include_in_llm_context for
the relevant adapter(s) before reading; turning the meta off via
/api/adapter-meta drops that adapters block out of the LLM prompt
- Defensive: missing meta row defaults to True (include); DB-unavailable
returns empty string; per-block 3000-char cap
- Module-level env_reporter singleton for the router
Router wiring (meshai/router.py):
- Extended _MESH_KEYWORDS dispatcher with _ENV_KEYWORDS_TO_SUBTYPE
mapping (fire/quake/flood/warning/storm/road/swpc/etc -> coarse
subtype). "flood" intentionally precedes "warning" so
"river flood warning" routes to gauges, not alerts
- _detect_env_subtype helper at module level (also test-importable)
- _is_mesh_question now also fires for env keywords -- single detector
per Matt s spec
- _detect_mesh_scope returns ("env", subtype) when an env keyword
matches, taking precedence over the node/region branches
- generate_llm_response: when scope_type == "env", appends
env_reporter.build_all() + env_reporter.build_drop_audit(hours=1)
to the system prompt. Wrapped in try/except so a reporter fault
never blocks the LLM call
Tests:
- tests/test_env_reporter.py (18 cases): meta gate, every build_*
method shape, build_all combines blocks, all-off produces empty
- tests/test_router_env_scope.py (18 cases): parametrized subtype
detection across fires/quakes/alerts/gauges/traffic/swpc, word-
boundary check (firearm != fire), synthetic-probe end-to-end
(seed fires table -> env_reporter emits a fires block with the
seeded row)
Test count: 761 -> 797 (+36 new, 0 regressions).
|
|||
| 42b3106e97 |
feat(v0.6-3c): adapter_config REST API + dashboard editor
Closes the audit-doc Section A keystone (the GUI editor). Together with
v0.6-3a foundation, v0.6-3a.1 trim, and v0.6-3b handler wiring, every
Rule-17 CONFIG knob from the audit is now editable in the dashboard
without a container restart.
API (meshai/dashboard/api/adapter_config_routes.py):
GET /api/adapter-config -- {adapter: [{key, value, default,
type, description}]}
GET /api/adapter-config/<adapter> -- one adapter list
GET /api/adapter-config/<adapter>/<key> -- single row
PUT /api/adapter-config/<adapter>/<key> body {value} -- typed validation
int: int or whole-number float; rejects bool, fractional float, str
float: int or float; rejects bool
str: str only
bool: bool only
json: any JSON-serializable value
Every PUT calls invalidate_cache() so the next handler accessor
read sees the new value -- no container restart needed.
POST /api/adapter-config/<adapter>/<key>/reset -- value_json = default_json,
cache invalidated
GET /api/adapter-meta -- {adapter: {display_name,
include_in_llm_context, description}}
PUT /api/adapter-meta/<adapter> partial-update body, fields:
include_in_llm_context: bool, display_name: non-empty str
Dashboard (dashboard-frontend/src/pages/AdapterConfig.tsx):
- Per-adapter cards. Header row shows display_name, the include_in_llm_context
toggle, and an expand chevron. Adapters with zero config keys (e.g. itd_511)
still render so users can toggle their LLM-context inclusion.
- Expanded body lists each key with a type-aware widget:
bool -> checkbox, commit-on-change
int/float -> number input, commit-on-blur (or Enter)
str -> text input, commit-on-blur
json -> textarea, commit-on-blur (JSON.parse with inline error)
Each row shows the key name, type tag, description, "edited" badge when
value != default, a per-key Reset button, and a save badge (spinner,
check, error tooltip, or a small amber dot for unsaved local changes).
- Auto-save semantics: every blur/change/reset triggers PUT immediately;
no explicit Save button needed. Reset is one-click per key.
Wiring:
- meshai/dashboard/server.py registers the new router with prefix /api.
- dashboard-frontend/src/App.tsx adds the /adapter-config route.
- dashboard-frontend/src/components/Layout.tsx adds the left-nav entry
(Sliders icon, label "Adapter Config", after Reference).
- Vite build produces a fresh meshai/dashboard/static bundle. The
Dockerfile copies meshai/ so the new bundle ships with the container
image at next rebuild.
Tests (tests/test_adapter_config_api.py, 30 cases):
- GET grouped, per-adapter, single key
- GET per-adapter returns [] for adapters with zero keys (itd_511)
- PUT updates value, GET shows new value, accessor returns new value
(proves cache invalidation propagates to the in-process accessor)
- PUT type validation per (int, float, str, bool, json) incl. edge cases:
int rejects str + fractional float + bool but accepts whole-number float;
float accepts int + float, rejects bool; bool rejects int; str rejects
other types; json accepts list / dict / None
- PUT 404 on unknown key, 400 on missing value field
- POST reset restores default + invalidates cache
- GET /api/adapter-meta: include_in_llm_context defaults match registry
(central / geocoder false, rest true)
- PUT meta partial update: only provided fields change
- PUT meta rejects non-bool include_in_llm_context, empty display_name,
unknown adapter
Test count: 731 -> 761 (+30 API cases, 0 regressions).
Refs audit doc v0.6-phase1-audit.md Section A keystone + finding #4.
|
|||
| 914d38c907 |
feat(v0.6-3b): wire every handler to adapter_config + v7.sql firms dedup_key column
Replaces module-level magic numbers in 12 handlers with reads via the
v0.6-3a.1 typed accessor. Every default matches the prior hardcoded
value exactly, so first-deploy behavior is unchanged.
Handlers wired (43 keys across the 43-row registry):
wfigs cooldown_seconds, anchor_max_mi, broadcast_on_acres,
broadcast_on_contained
nws broadcast_severities, tombstone_msgtypes,
warning_suffix_promotes
usgs_quake regional_centroid, regional_radius_mi,
broadcast_pager_alerts, global_mag_floor,
regional_mag_floor, escalate_mag_floor
swpc geomag_kp_floor (extends G-scale down to Kp 5 when
lowered), flare_class_floor (R-scale
extended to M-class when lowered),
proton_pfu_floor
usgs_nwis parameter_codes, broadcast_on_recede
incident freshness_seconds, broadcast_on_update (Update path
re-implemented when toggled True:
magnitude step-up / delay doubling /
icon_category change)
tomtom_incidents drop_zero_magnitude, drop_non_present
state_511_atis skipped_states (case-insensitive match against both
state_code and primary_region suffix)
central severity_thresholds (immediate_min check ordered before
priority_max so the +inf clamp still
works)
dispatcher dedup_lru_max, cooldown_prune_size,
cooldown_prune_multiplier, dedup_db_retention_days
band_conditions swpc_freshness_seconds, hamqsl_url, hamqsl_timeout_s
geocoder (photon_url/timeout/radius/limit/town_osm_values/
h3_cache_max -- module-level constants kept as
backward-compat aliases; runtime reads via accessor)
pipeline Inhibitor.ttl_seconds + Grouper.window_seconds now
default to None, falling back to
adapter_config.pipeline.{inhibitor_ttl_seconds,
grouper_window_seconds}. Explicit constructor values
still win (test fixtures unchanged).
firms confidence_floor, frp_floor, bbox, dedup_distance_m
Schema:
v7.sql adds firms_pixels.dedup_key column + drops the old hardcoded
round(lat,5) UNIQUE INDEX, replaces with UNIQUE (dedup_key, acq_time,
satellite). The firms_handler quantizes lat/lon to
(dedup_distance_m / 111000) degrees at INSERT time -- meters-based
precision per Matt s spec, tunable via the GUI without schema changes.
SCHEMA_VERSION 6 -> 7. firms_pixels has 0 rows in production so no
backfill needed.
CODE preserved (Matt s rule): sentence templates, emoji literals, the
TomTom icon_map / ITD sub_type_map / Central adapter_map / category_map
translation tables, the band_conditions Kp/SFI -> Good/Fair/Poor
heuristic, anchor-priority ordering, expires-bucket boundaries, the
NOAA G/R/S scale tables. None of these reach the GUI.
Hot-path performance: every accessor read hits the in-memory cache after
the first call; cache hit is one dict get. Per-event reads (e.g. WFIGS
cooldown_seconds on every WFIGS poll-cycle) add a single dict lookup
to existing pipelines.
Backward-compat aliases retained for module-level imports that exist in
test code: WFIGS_BROADCAST_COOLDOWN_S, FIRMS_CONFIDENCE_FLOOR,
FIRMS_FRP_FLOOR, FIRMS_BBOX_OPTIONAL, INCIDENT_FRESHNESS_MAX_S,
PHOTON_BASE_URL/TIMEOUT_S/RADIUS_KM/LIMIT. Handler code reads via
adapter_config; tests can either monkeypatch the module attribute (firms)
or mutate adapter_config DB values.
Test count: 731 -> 731 (no new tests in 3b -- handler wiring is a pure
refactor; coverage comes from the existing handler test suites passing
unchanged).
Refs audit doc v0.6-phase1-audit.md Section A + Matt s CONFIG-vs-CODE rule.
|
|||
| 68dcbc74d0 |
feat(v0.6-3a.1): trim adapter_config registry to CONFIG-only per Matt config-vs-code rule + log-on-delete safety net for orphan cleanup
Drops 35 of the v0.6-3a-draft 77 keys + adds 1 net-new key
(firms.dedup_distance_m) for a final count of 43. The trim rules:
CONFIG (lives in adapter_config, surfaces in the GUI):
where we send (channels), how often (cadences/schedules),
thresholds (magnitude floors, severity gates, distance radius,
cooldown durations, freshness windows), curation data (which
sites/states/codes), toggles (enabled, include_in_llm_context,
drop_zero_magnitude).
CODE (stays in handlers, never reaches the GUI):
sentence templates, emoji choices, mapping/translation functions
(TomTom icon_map, ITD sub_type_map, Central adapter_map and
category_map), rendering logic (anchor priority order,
expires-bucket formatting, threshold-state labels), heuristic
logic (band_conditions Kp/SFI -> Good/Fair/Poor function).
Per-adapter outcome (kept | killed):
wfigs 4 | 4 (cooldown_seconds, anchor_max_mi, two re-broadcast toggles)
nws 3 | 4 (broadcast_severities, tombstone_msgtypes, warning_suffix_promotes)
usgs_quake 6 | 3 (centroid, radius, PAGER list, 3 mag floors)
swpc 3 | 7 (three storm-tier floors)
usgs_nwis 2 | 4 (parameter_codes, broadcast_on_recede)
incident 2 | 0 (freshness_seconds, broadcast_on_update)
tomtom_incidents 2 | 1 (drop_zero_magnitude, drop_non_present)
state_511_atis 1 | 0 (skipped_states)
itd_511 0 | 3 (all sub_type maps/emoji/phrase = CODE)
central 1 | 2 (severity_thresholds)
dispatcher 4 | 0 (LRU cap, prune params, retention days)
band_conditions 3 | 6 (SWPC freshness + HamQSL endpoint config)
geocoder 6 | 1 (Photon endpoint + town-OSM curation + cache cap)
firms 4 | 1* (confidence_floor, frp_floor, bbox, dedup_distance_m)
pipeline 2 | 0 (inhibitor TTL, grouper window)
* firms: dedup_lat_lon_decimals is replaced by dedup_distance_m=5 per
Matt s call (user-facing unit is meters, not decimal places; the
handler will internally translate to quantization step in v0.6-3b).
adapter_meta stays at 15 rows -- itd_511 keeps its include_in_llm_context
toggle even with zero config keys.
Live-DB cleanup:
meshai/adapter_config/__init__.py:prune_orphans(conn) DELETEs every
adapter_config row whose (adapter, key) is no longer in REGISTRY. Each
delete is INFO-logged with the prefix "adapter_config orphan removed:"
so docker logs carry the paper trail. Called from init_db() after
seed_defaults; idempotent (zero deletes on every subsequent boot).
Cache is invalidated when any orphan is removed.
adapter_meta is NOT pruned -- meta rows are cheap and useful even for
adapters that ended up with zero config keys.
Tests (34 cases, replaces v0.6-3a 24-case set):
- Registry count is 43; ADAPTER_META is 15
- Seed lands every REGISTRY + ADAPTER_META row; idempotent; never
overwrites user edits
- prune_orphans removes a synthetic legacy row, logs at INFO with the
exact prefix, leaves known keys untouched, leaves adapter_meta
untouched, invalidates the accessor cache
- Accessor returns correctly-typed values incl new
firms.dedup_distance_m
- Guard tests: no key in REGISTRY contains "emoji", ends with "_map",
or contains "template" / "prefix" (catches CODE leaking back in)
Test count: 721 -> 731 (+10 net: +5 prune cases, +1 firms.dedup_distance_m,
+3 CODE-guard cases, +1 registry-count assertion).
Refs Matt s locked CONFIG-vs-CODE rule.
|
|||
| cb3c5aec7e |
feat(v0.6-3a): adapter_config foundation -- migration + defaults registry + typed accessor
Closes the foundation slice of audit doc Section A (Rule 17). Lands two
SQLite tables, the seed routine that populates them from a Python
defaults registry, and a typed accessor that handler code will read in
v0.6-3b. No handler changes in this commit -- ZERO behavior risk, every
existing test still passes (721 / 69 skipped / 0 failed).
v6.sql tables:
- adapter_config(adapter, key, value_json, default_json, type, description,
updated_at) PRIMARY KEY(adapter, key) -- JSON-encoded
values flow through a single column uniformly. CHECK
constraint on `type` closes the vocab (int/float/str/
bool/json).
- adapter_meta(adapter PK, display_name, include_in_llm_context,
description, updated_at) -- per-adapter metadata + the
user-scopable LLM-context toggle (Matt refinement #5).
meshai/adapter_config/ package:
- defaults.py: REGISTRY dict mapping (adapter, key) -> {default, type,
description}. Covers audit doc sections A.1-A.12: wfigs, nws,
usgs_quake, swpc, usgs_nwis, incident family (tomtom_incidents,
state_511_atis, itd_511, shared "incident"), central consumer,
dispatcher, band_conditions, geocoder, firms, pipeline (Inhibitor +
Grouper). ~85 keys total. ADAPTER_META covers 15 adapters with
display_name + include_in_llm_context defaulting to True. Per Matt
refinement #3, every default matches the current handler constant
EXACTLY -- first deploy behavior is unchanged.
- _accessor.py: AdapterConfig class with `adapter_config.<adapter>.<key>`
syntax. Read pipeline: in-memory cache hit -> SQL -> registry
fallback (with WARNING) -> AttributeError. Process-wide cache; PUT
via v0.6-3c REST API calls invalidate_cache() to drop the cache.
GIL-atomic dict reads on the fast path (handlers call this hot).
- __init__.py: seed_defaults(conn) -- INSERT OR IGNOREs one row per
registry entry. Idempotent, never overwrites user edits.
Wiring:
- meshai/persistence/db.py: SCHEMA_VERSION 5 -> 6, and init_db() now
calls seed_defaults() after migrations apply.
- meshai/main.py: _init_components() now calls init_db() FIRST (per
commit #1 lessons-learned: a startup-time migration is required
when handlers will rely on the new schema; lazy-on-first-handler
is fine for v4/v5 but not for v6 where handler reads start in
v0.6-3b).
- tests/conftest.py: autouse fixture now calls init_db() + clears
the accessor cache around each test, so every test gets the v6
seed AND a clean cache without per-test boilerplate.
Tests (tests/test_adapter_config_foundation.py, 24 cases):
- v6 tables exist + schema_meta at 6 + type-vocabulary CHECK enforced
- seed populates every REGISTRY + ADAPTER_META row, value_json ==
default_json on first seed, type matches
- seed is idempotent + does not overwrite user edits
- accessor returns correctly typed values for int/float/str/bool/
json list/json dict/json None
- cache hit: second read does not touch the DB (patched _load_from_db
raises, accessor still succeeds)
- invalidate_cache forces a re-read; mutated DB value wins
- registry fallback path triggers when a row is missing (with WARNING)
- unknown key raises AttributeError
- setattr blocked (writes go via the REST API in 3c)
- every default JSON round-trips cleanly; every type is in vocabulary
- ADAPTER_META covers every adapter in REGISTRY
Test count: 697 -> 721 (+24 new, 0 regressions).
v0.6-3b will wire handlers one at a time (wfigs, nws, quake, swpc, nwis,
incident, central, dispatcher, band_conditions, geocoder, firms). Per
the audit lock, defaults match exactly so each wiring step is a pure
refactor -- bisect-safe.
v0.6-3c lands the /api/adapter-config CRUD + the AdapterConfig.tsx
dashboard editor + cache invalidation on PUT.
Refs audit doc v0.6-phase1-audit.md Section A + finding #4.
|
|||
| c333a97344 |
feat(v0.6-2): dispatcher state persistence -- cold-start, cooldowns, dedup LRU to SQLite
Closes Rule-20 dispatcher gap from audit doc v0.6-phase1-audit.md finding #1. Pre-this-commit the cold-start anchor, 4 drop counters, per-toggle cooldown map, and dedup OrderedDict all lived in Dispatcher instance memory and were lost on every container restart. v5.sql adds three tables: - dispatcher_state (singleton id=1): cold_start_anchor + 4 drop counters - dispatcher_cooldowns ((toggle,category,region) keyed): last_fired_at - dispatcher_dedup ((source,event_id) keyed): seen_at Dispatcher refactor: - __init__ calls _restore_from_db -- counters, cold-start anchor, cooldown map, and dedup LRU (most-recent 10k by seen_at) all rehydrated from the three new tables - write-through on every mutation: _persist_state for counter/anchor, _persist_cooldown for cooldown UPSERT + 2*cooldown_s prune, _persist_dedup for dedup INSERT OR REPLACE + 7-day cleanup - in-memory caches stay authoritative on the fast read path - cumulative-since-install counters (NOT since-boot); LLM will be able to answer "we have dropped 47 stale events this week" after commit #5 (env_reporter) lands - graceful degrade: missing v5 tables / persistence outage falls back to fresh in-memory state without crashing the constructor Tests: - tests/test_dispatcher_persistence.py (17 tests): state restore on init, counter+cooldown+dedup survival across simulated restart, cooldown rearm within 2x window, dedup LRU rebuild caps at 10k, 7-day cleanup on insert, INSERT OR REPLACE on duplicate source+event_id, v5 migration idempotent, synthetic storm (50 events) -> restart -> replay (5 incl 1 duplicate) with the duplicate dedup-rejected and counters NOT resetting - tests/conftest.py (new): autouse MESHAI_DB_PATH redirection to per-test tmp file, so the dispatcher_* tables on production /data dont get polluted by tests that construct Dispatcher() without an explicit fixture - tests/test_notification_toggles.py: _dispatch helper wipes dedup/cooldown/ state tables between calls (per-call independence preserved; pre-v0.6-2 in-memory-only Dispatcher reset naturally per instance) Test count: 680 -> 697 (+17 new, 0 regressions). Refs audit doc v0.6-phase1-audit.md finding #1. |
|||
| b2c4d53b14 |
feat(v0.6-1): FIRMS handler -- storage-only, closes silent-drop on central.fire.hotspot.>
v0.5.13 default-deny was silently dropping every FIRMS hotspot because no per-adapter handler existed. firms_pixels table has been empty since v0.5.8b. This commit adds central/firms_handler.py which stores every passing pixel that clears the (currently hardcoded, future GUI-driven) confidence + FRP floors, with dedup on (round(lat,5), round(lon,5), acq_time, satellite) via a unique partial index added in v4.sql. NO mesh broadcasts emitted by this handler -- FIRMS data is for LLM context only and will become queryable when commit #5 (env_reporter) lands. Defaults baked in: FIRMS_CONFIDENCE_FLOOR = "low" -- store every confidence level FIRMS_FRP_FLOOR = 0.0 -- store every FRP value FIRMS_BBOX_OPTIONAL = None -- no spatial filter These become adapter_config GUI rows in commit #3 (per Matt's v0.6 Phase 1 refinement: hardcoded values become GUI default values so first-deploy behavior is unchanged). Wiring: - meshai/central/firms_handler.py (new, 270 lines) - meshai/persistence/migrations/v4.sql (new, unique dedup index) - meshai/persistence/db.py (SCHEMA_VERSION 3 -> 4) - meshai/central/consumer.py (dispatch ladder gets firms branch before default-deny clause; pattern matches handle_swpc / handle_nwis) - tests/test_firms_handler.py (new, 22 tests covering confidence floor, FRP floor, bbox, dedup, missing fields, end-to-end through consumer) - tests/test_consumer_default_deny.py (swap firms -> avalanche for the "no handler" examples since FIRMS now has one) Test count: 658 -> 680 (+22 firms_handler tests, 0 regressions). Refs audit doc v0.6-phase1-audit.md finding #2. |
|||
| b6160d2eda |
feat(v0.5.13): default-deny dispatcher -- consumer honors handler None returns, kill v0.5.7 regression at the root
Fixes the v0.5.7 regression that came back through the live flip. Per-adapter handler returning None now means no broadcast. Title fallback chain through data.title -> headline -> friendly_name removed. enabled_toggles config read also fixed -- was dict-vs-object access. Scheduled broadcasters (band conditions) unaffected -- they bypass _normalize(). Memory rule 19 added. The diagnosis: during overnight monitoring after the v0.5.12.1 flip, Matt saw 8 broadcasts in dashboard log over 6h20m using the v0.5.7-regression format (`🚧 ROADS: Road Incident, US-ID. immediate` / `🔥 FIRE: Wildfire Hotspot. priority` / `⚠️ RF: Space Weather Alert. routine`) while mesh_broadcasts_out only showed 2 entries. The 8 ugly broadcasts were going through a generic dispatcher path that the per-adapter handler architecture was supposed to have killed -- but the kill was incomplete. Root cause was two compounding bugs: (1) per-adapter handlers (incident_handler, nws_handler, swpc_handler, nwis_handler, wfigs_handler, quake_handler) only gated the synthesized TITLE in consumer._normalize(), not whether the Event was emitted. The fallback chain `title = data.title or data.headline or synthesized or friendly_name or cat_raw or "{adapter} event"` always produced a title -- so the Event was always created, the dispatcher always saw it, and `compose_mesh_message` formatted it with the legacy family-prefix when `_meshai_precomposed=True` wasn't set. (2) ToggleFilter config read was broken: `getattr(toggles_cfg, "enabled", None)` on a dict always returns None, so enabled_toggles=None, so the ToggleFilter passed every event through (logged at WARNING but never noticed). Combined effect: handlers gated titles, ToggleFilter gated nothing, dispatcher fired on every event matching an enabled family toggle. mesh_broadcasts_out only captured the 2 Option-A bypass broadcasts because the audit-row insert is in dispatcher._post_broadcast_commit which requires `event.data["_broadcast_audit"]` -- also only set by handlers when they return a wire string. The fix is structural: consumer._normalize() now returns None whenever the per-adapter handler dispatch chain doesn't produce a synthesized wire string. No title fallback, no Event emitted, no dispatcher invocation. Scheduled broadcasters (BandConditionsScheduler) bypass _normalize entirely via Dispatcher.dispatch_scheduled_broadcast() so they're unaffected. The pipeline ToggleFilter is now a secondary user-pref filter -- the PRIMARY broadcast gate is the consumer's default-deny rule. pipeline/__init__.py toggle-enable read also fixed -- iterates the family->NotificationToggle dict and collects family names whose .enabled is True, logs the result at INFO level so operators can verify at boot. Tests: was 718 (v0.5.12.1 baseline). 36 tests were skipped with clear reasons because they encoded the v0.5.7-regression behavior that v0.5.13 intentionally removes (`test_central_envelope_to_wire_v057.py`, `test_central_sub_adapter_routing.py`, `test_central_consumer.py`, `test_fire_v057.py`, plus 2 from `test_rf_v057.py`). New `tests/test_consumer_default_deny.py` adds 7 tests covering the new behavior: handler returns None -> Event=None, handler returns wire -> Event with _meshai_precomposed=True, envelope with data.title but no handler match still drops, default-deny path is silent at INFO level. Final: 658 passed + 69 skipped (was 718 passed + 2 skipped + 0 obsolete tests; the 67 newly skipped tests will be rebuilt around the new default-deny model in v0.6). Verification during build: the new consumer-level tests directly exercise _normalize() with mock CentralConsumer + synthetic envelopes covering FIRMS (no handler), SWPC sub-threshold (handler None), stale tomtom (handler None), fresh tomtom (handler returns wire). All match the new semantics exactly. Master remains ON through this commit. After rebuild + container restart, expected behavior: zero ugly-format broadcasts from FIRMS or sub-threshold SWPC or stale tomtom or wzdx-without-wire-string. Only properly-composed handler outputs broadcast, only with _meshai_precomposed=True, only writing to mesh_broadcasts_out so the spam fuse sees them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 42ab966112 |
fix(v0.5.12.1): hotfix consumer.py missing 'import time'
URGENT live-flip hotfix. Bug: v0.5.9 GAMMA universal freshness gate added time.time() calls to meshai/central/consumer.py without importing time. v0.5.10 nws/quake/swpc dispatch path inherited the same bug. v0.5.11 + v0.5.12 builds inherit it too. The module only imported `datetime`, not `time`. Result during the live flip: EVERY Central event hit NameError: name 'time' is not defined at consumer.py line 454 inside the v0.5.9 GAMMA freshness check. The try/except inside _normalize() swallowed the exception, returning None synthesized for every event. 33 minutes of master=ON resulted in 0 broadcasts AND 0 persistence rows -- the right outcome (no spam) but via the wrong mechanism (crash on every event). How it slipped through: the v0.5.9 GAMMA patch script extended consumer.py with time.time() calls but didn't audit the existing imports. The test suite (718 passing) didn't catch it because the tests for the freshness helper run against central_normalizer (which DOES import time) -- the consumer.py path is never exercised by the unit tests. Live flip discovered it on first envelope arrival. Fix: one-line addition of 'import time' to consumer.py top-imports block. py_compile clean, 718 tests still pass. Container will be rebuilt + restarted with master remaining ON. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| b031bc9b89 |
feat(v0.5.12): usgs_nwis with minimal Idaho threshold curation (9 starter sites)
Final per-adapter handler before the live flip. Shape matches the v0.5.10 weather/quake/swpc family + the v0.5.9 WFIGS forward-only model that suits water-level data best: rising water is operationally meaningful (downstream warnings, evacuation calls); receding water is intentionally silent.
Components: (1) meshai/central/idaho_gauge_sites.py with a hardcoded 9-site dict covering Magic Valley + Treasure Valley + Salmon-Challis + Snake River system: Big Lost (Mackay), Snake at Heise + Idaho Falls, Big Wood (Hailey), Boise River, Payette at Banks, Henrys Fork (Rexburg), Salmon Falls Creek, Bear River at Border. Each entry carries gauge_name + lat/lon + per-threshold ft values (action / flood_minor / flood_moderate / flood_major; None means that threshold does not apply at that site). Site lookup normalizes incoming envelope monitoring_location_id (\'USGS-13186000\' or bare \'13186000\') to the canonical USGS- prefixed form. STARTER SUBSET clearly flagged in the module docstring -- expansion to full 20+ site coverage deferred to v0.6.x and likely migrated to a DB table editable via the GUI.
(2) meshai/central/nwis_handler.py filters non-curated sites at handler entrance (event_log handled=0, no gauge_readings UPSERT). Parameter filter: 00060 discharge (cfs) and 00065 gage height (ft) only; precipitation (00045) and other parameters skipped. threshold_state computed from value vs curated NWS-AHPS thresholds (high to low). UPSERT into v0.5.8b gauge_readings table (no schema migration needed; threshold_state column already there). Upward crossing detection by comparing current threshold to the most recent prior reading\'s threshold; ordered scale {normal < action < flood_minor < flood_moderate < flood_major}. If current > prior, fire \'New:\' broadcast; otherwise (unchanged, descending, or stays at same level for 96 polls/day), silent.
Wire format MEDIUM: \'🌊 New: {gauge_name}: {label} {value} ft, flow {flow_cfs:,} cfs, @ lat,lon\'. Label maps action->\"action stage\", flood_minor->\"minor flooding\", flood_moderate->\"moderate flooding\", flood_major->\"major flooding\". flow_cfs segment present only when a companion 00060 discharge reading is available. Coords segment dropped when both envelope and curated coords are missing (rare for curated sites which always have coords). Example outputs from the synthetic probe (all under 130-byte target):
🌊 New: Snake River at Heise: action stage 12.5 ft, @ 43.612,-111.654 (71 B)
🌊 New: Snake River at Heise: minor flooding 14.5 ft, @ 43.612,-111.654 (73 B)
🌊 New: Snake River at Heise: moderate flooding 16.5 ft, @ 43.612,-111.654 (76 B)
🌊 New: Boise River near Boise: action stage 8.5 ft, @ 43.690,-116.200 (72 B)
Tests: was 704 (v0.5.11 baseline), now 718 (+14 net new). Coverage: curated-site action-stage broadcasts, non-curated drop, normal-stage silent, normal->action upward crossing, action->normal downward suppression, same-threshold dedup (no broadcast every 15-min poll), flow_cfs companion from prior 00060 reading, coords fallback to curated dict when envelope lacks them, IDAHO_CURATED_SITES count + required-fields check, exact starter-set spot check, commit-callback flips event_log.handled to 1, action->flood_minor re-broadcast at the higher threshold, precip (00045) skipped, site_id normalization accepts bare \'13186000\'.
Synthetic probe over the 58,436 captured nwis envelopes from the v0.5.10 batched investigation: 3,292 hit the 9-site curation (5.6% of total volume); 1 produced a real upward-crossing broadcast detected in the captured stream (validating the dedup story -- subsequent synthesized broadcasts for the same site at the same threshold correctly silent-suppressed). 3 additional synthesized broadcasts from a rising Snake at Heise scenario (9.0->12.5->14.5->16.5 ft); receding step (15.5 ft) correctly produced no broadcast.
usgs_nwis closes the last per-adapter handler before live flip. WFIGS / incident-pipeline / weather / quake / swpc / band-conditions all unchanged. Master OFF in prod through this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 0da83e0d3d |
feat(v0.5.11): band conditions scheduled broadcaster (3x/day HF propagation)
First clock-driven broadcaster in meshai, distinct from the v0.5.8b/v0.5.9/v0.5.10 event-driven adapters. The same persistence + dispatcher + cold-start patterns apply, but the trigger is the wall clock at 06:00 / 14:00 / 22:00 Mountain Time (default; GUI-configurable per Rule 17). Components: (1) meshai/notifications/scheduled/band_conditions.py with BandConditionsScheduler (asyncio loop, mirrors the existing DigestScheduler shape), compute_band_ratings() with two-tier data sourcing -- (a) latest swpc_kindex + swpc_alerts F10.7 rows from persistence within the last 6h, (b) HamQSL.com solarxml.php fallback when SWPC is stale or incomplete, (c) silent skip when both fail, format_band_conditions_wire() multi-line MEDIUM output (~115-120B). (2) v3 schema migration adding band_conditions_broadcasts(broadcast_id PK AUTO, sent_at, scheduled_for UNIQUE, ratings_json, source). UNIQUE(scheduled_for) enforces per-slot dedup so a retry storm cannot double-broadcast. (3) Dispatcher.dispatch_scheduled_broadcast() bypasses the toggle / rules / freshness-gate pipeline but DOES honour the v0.5.8b cold-start grace -- first scheduled broadcast within the grace window after meshai starts is suppressed, mesh_broadcasts_out audit row only inserted on actual delivery. Channel selection routes through the rf_propagation toggle\'s broadcast_channel since band conditions IS RF-propagation info. (4) NotificationsConfig gains band_conditions_enabled (default true), band_conditions_schedule (list of HH:MM strings, default ["06:00","14:00","22:00"]), band_conditions_tz (default "America/Boise" so DST handles automatically). (5) Notifications.tsx grows a Band Conditions card between Cold-Start Grace and Master Toggles with the enable toggle + 3 TimeInput slots + a one-liner explaining the source priority. (6) build_pipeline + start_pipeline spawn the BandConditionsScheduler alongside the existing DigestScheduler -- best-effort, scheduler failures must NOT break notifications startup. Wire format examples (multi-line, all under 130B target): ☀️ Day Propagation 📡 Band Conditions: 80-40m: 🟡 Fair 30-20m: 🟢 Good 17-15m: 🟢 Good 12-10m: 🟡 Fair 🌞 Day Propagation (14:00 slot when storm onset, Kp=6 SFI=110) 📡 Band Conditions: 80-40m: 🔴 Poor 30-20m: 🔴 Poor 17-15m: 🔴 Poor 12-10m: 🟡 Fair 🌙 Night Propagation (22:00 slot, recovery, Kp=4 SFI=120) 📡 Band Conditions: 80-40m: 🟡 Fair 30-20m: 🟡 Fair 17-15m: 🔴 Poor 12-10m: 🔴 Poor Tests: was 686 (v0.5.10 baseline), now 704 (+18 net new -- quiet/storm condition ratings, HamQSL XML parse fallback, both-fail silent-skip path, is_day_slot per HH:MM, wire format for all 3 slot variants, byte-size guard, 6-line shape, fire_slot record row, dedup via UNIQUE constraint, silent-skip path, slot_epoch DST alignment summer + winter). Synthetic 24h probe verified the 3 expected slots fire correctly with quiet/storm/recovery scenarios + the 4th no-data scenario lands as source=\'skipped_no_data\' with no broadcast. usgs_nwis deferred to v0.5.12 (threshold-curation work). Master OFF in prod. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| de35f9c748 |
feat(v0.5.10): nws + usgs_quake + swpc handlers
Three more per-adapter handlers landing in the same v0.5.9-incident-pipeline pattern: nws_handler.py with severity-floor gate (Warning+ broadcasts only, Moderate/Minor/Unknown skipped to event_log handled=0), event-type emoji map, CAP-id-based first-sight dedup via nws_alerts table; quake_handler.py with magnitude-floor gate (M3.0 globally + M2.5 within 250mi of Idaho centroid + tsunami at any M) using Haversine for the distance check, USGS data.place curated string preferred for the place anchor, leading emoji escalation (🌐 routine / ⚠️ M5+ / 🚨 tsunami), Magnitude spelled out per Matts call; swpc_handler.py with aggressive G3+/R3+/S1+ gate, plain-English wire headlines with (NOAA scale / underlying scalar) tail tag per Matts option C (e.g. "Strong geomagnetic storm (G3/Kp7) -- HF degraded, aurora possible"), routine Kp + protons persisted to swpc_events.payload_json for trending but never broadcast. All three share the v0.5.9 universal freshness gate and the no-Update first-sight-only pattern. Persistence uses the existing v0.5.8b nws_alerts, quake_events, swpc_events tables -- no migrations needed. Tests: was 634 (v0.5.9 baseline), now 686 (+52 net new; over-delivered because parametrized emoji map adds 14 rows). Synthetic probe over the 4 nws + 1 quake + 16,217 swpc captured envelopes from the batched investigation: Phase 1 = 0/0/0 broadcasts (all real captures correctly filtered by their respective gates); Phase 2 = 5/5 synthesized fresh test events broadcast correctly (Severe T-Storm warning, M4.1 Garden Valley quake, G3 geomagnetic storm, X1.2 flare, S1 proton). WFIGS handler unchanged. usgs_nwis deferred to v0.5.12 (threshold-curation work). Master OFF in prod. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 0099d0fd94 |
feat(v0.5.9): unified incident pipeline + state_511_atis Idaho cutover + two-sided freshness gate
Coordinated change across the consumer dispatch layer + central_normalizer + new incident_handler + new itd_511 work_zone parser. The integrated story: Central PM filed a heads-up that ITD 511 publishes four EventTypes (work_zone, closure, incident, special_event) under us.id Convention A, and meshais v0.5.8 work focused on work_zone shape only -- meaning incidents (the higher-priority operational signal) were silently rendering as the v0.5.7-regression-style fallback. This v0.5.9 closes that gap by treating incidents + closures + special_events from all three traffic adapters (state_511_atis, itd_511, tomtom_incidents) as a single unified pipeline, while also migrating Idaho coverage from state_511_atis to itd_511 (the direct ITD feed) at the consumer level. Components: (1) new meshai/central/incident_handler.py routes incident/closure/special_event events through per-adapter parsers (tomtom, itd_511, state_511_atis-non-ID) to a canonical incident shape, then a single rendering pipeline with sub_type-aware emoji selection (jam/crash/road_closed/disabled_vehicle/parade/special_event/vehicle_fire/road_works). (2) Universal two-sided freshness gate in the consumer dispatch layer: only events with 0 <= age <= 1800s (default-allow on missing start_time) make it past the gate. Rejects both stale events (more than 30 min old) AND future-scheduled events (negative age -- a real itd_511 case for scheduled work projects). The gate sits ABOVE both incident_handler and the v0.5.8 work_zone formatter so all adapters get gated uniformly. (3) state_511_atis Idaho cutover -- both incident_handler and the v0.5.8 work_zone parser skip state_511_atis events where the state token is ID, deferring to itd_511 as the authoritative source. state_511_atis remains fully active for non-Idaho neighbor coverage (WA/OR/MT/UT/WY/NV) -- verified by Phase 2 WA broadcasts in the synthetic probe. (4) new itd_511 work_zone parser (extension to central_normalizer.py) consumes the itd_511 work_zone EventType and produces the same MEDIUM-style wire format as the existing state_511_atis work_zone parser (road + mile range + town + direction + sub_type + ends-at). (5) No Update: broadcasts in the incident pipeline -- per Matts call, real-time traffic Updates (jam getting worse, delay growing) are not actionable for mesh users. State tracking continues via traffic_events UPSERT but only the first sighting of an external_id ever fires a New: broadcast. WFIGS handler unchanged -- fires keep their 8h-rate-limited Update: behavior since acres growth IS operationally meaningful (evacuation decisions). Forecast: 3-10 mesh broadcasts/day in Idaho, all New:. Cross-check: original raw broadcast count was 623 against a fixed-clock 49-min synthetic window; after v0.5.9 REVISED (no Updates) it dropped to 18; after v0.5.9 GAMMA (two-sided gate + Idaho cutover) it dropped to 9. Test count: was 589 baseline, +45 net new -- 634 passing. Synthetic probe verified all four phases: Phase 1 (replay 3032 captured envelopes) = 0 broadcasts (correctly suppressed); Phase 2 (synthesized fresh non-ID + ID) = 7 broadcasts; Phase 3 (synthesized fresh itd_511 work_zone) = 2 broadcasts; Phase 4 (synthesized fresh ID for explicit ID-skip exercise) = caught by ID-skip 1/1. Master stays off in prod; no toggle flips. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 053d67db6e |
feat(v0.5.8b): persistence foundation + WFIGS handler + universal cold-start grace
Three integrated pieces that ship together because they were designed as one safety story: (1) PERSISTENCE FOUNDATION -- new meshai/persistence/ module with SQLite db.py, schema migration framework (v1), 13 tables covering all adapter event shapes (traffic_events, fires, firms_pixels, quake_events, nws_alerts, gauge_readings, swpc_events) + mesh state (mesh_nodes, mesh_telemetry, mesh_positions, mesh_messages_in, mesh_broadcasts_out, mesh_health_events) + cross-cutting event_log + schema_meta. WAL mode for reader concurrency, single-writer pattern, MESHAI_DB_PATH env var, mounted at /data/meshai.sqlite via existing docker-compose meshai_data volume. .gitignore updated. (2) WFIGS HANDLER -- meshai/central/wfigs_handler.py implements the first per-adapter handler that uses the persistence layer. Format: MEDIUM style with town/landclass/county fallback chain, lat/lon at 3-decimal precision, New:/Update: prefix. 8h-rate-limited change-detection per IRWIN via fires.last_broadcast_at. Skips tombstones and perimeters silently (logged to event_log with handled=0). Acres fallback chain DailyAcres -> IncidentSize -> raw.DiscoveryAcres -> raw.FinalAcres -> N/A. Pass-through Initial Attack auto-numbered names (IA 1, IA 2). (3) UNIVERSAL COLD-START GRACE -- meshai/notifications/pipeline/dispatcher.py grows a configurable grace window (cold_start_grace_seconds, default 60s, GUI-editable per Rule 17). Anchored to first-event-seen (not container boot), so the grace activates the moment broadcasts could fire. Suppresses mesh delivery during the window; handler-side persistence (fires UPSERT, event_log) still happens normally. New _cold_start_dropped counter exposed in dispatch_stats(). Designed to protect against JetStream backlog spam at toggle-flip time, applies universally to ALL adapters. (4) WFIGS HANDLER CALLBACK REFACTOR -- New:/Update: prefix now keys on fires.last_broadcast_at IS NULL (not row-missing), and last_broadcast_* field updates moved to a post-broadcast commit callback that the dispatcher invokes ONLY on successful delivery. This means: cold-start-suppressed events leave fires.last_broadcast_at NULL, so when they eventually broadcast post-grace, they correctly render as New: (first ACTUAL delivery for that IRWIN), not Update:. event_log.handled and mesh_broadcasts_out audit row also gated on the same callback -- decoupling persistence rows from broadcast rows for an honest audit trail. New tests: 15 in test_wfigs_handler.py, 15 in test_persistence.py, additional cold-start grace tests in test_dispatcher.py (+4 WFIGS callback scenarios). Synthetic probes wfigs-cleaned-samples.md (initial) and wfigs-cleaned-samples-v2.md (cold-start verification) generated against isolated temp SQLite databases. CT108 /data/meshai.sqlite untouched during build. Master stays off. No live toggle flips. Test count: was 535 (v0.5.7 baseline) -> 566 (persistence) -> 581 (wfigs handler) -> 589 expected (cold-start grace). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
Matt Johnson
|
7751a40c6c |
feat(content): v0.5.8-state_511_atis -- central_normalizer with Photon nearest_town + composer bypass + SB->S route normalization
First per-adapter content formatter in the meshai-side central_normalizer library (per Central response to schema-divergence + nearest-town reports). state_511_atis (94% of Idaho 511 work-zone traffic) now produces clean wire strings like "🚧 SH-55, near McCall: both directions, emergency repairs" instead of the previous "🚧 ROADS: Work Zone, US-ID. routine -- roadwork". Implementation: nearest_town(lat, lon) calls Photon directly at 100.64.0.24:2322/reverse with osm_tag=place + client-side filter for city/town/village/hamlet (Navi passthrough route documented in Central response does not exist on current Navi instance). H3-cell-7 LRU cache. Town fallback chain: _enriched.geocoder.city -> nearest_town(coords) -> drop segment. Composer bypass via event.data["_meshai_precomposed"] flag -- renderer owns full wire string for normalized events. SB->S route normalization. distance<1mi -> "near X". Tests: 535 passed (was 511, +24 net). Synthetic probe over 25 bucket-B + 8 fixture envelopes confirmed 23/25 + 8/8 produce clean output; 2/25 fell back to None (drop segment) on Photon index gaps near Boise/Cascade. Matt eyeballed and approved. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Matt Johnson
|
0a66f4b756 |
fix(notifications): v0.5.7-regression -- consumer title fallback uses registry name, mesh renderer drops [Family] prefix
TWO PRE-EXISTING bugs (dormant in safe-mode for months) that the v0.5.7 staged flip exposed the moment Central became the live source for the first time. Matt observed the exact failure mode on the mesh at 2026-06-04 15:40:30 UTC:
[Roads] 🚨 ROADS: incident.tomtom_incidents, US-ID. immediate
Neither bug was authored by v0.5.7. The campaign reordered/added Central subscriptions but did not touch the consumer normalize() or the mesh renderer. The bugs surfaced because v0.5.7 was the first occasion since v0.5.2 to actually flip notifications.enabled=True with adapters set to feed_source=central. Pre-flip, no live broadcast had ever fired in prod (safe-mode held throughout the months between v0.5.2 and v0.5.7).
The v0.5.2 cooldown filter held the mesh blast radius to a single event -- subsequent tomtom_incidents broadcasts in the same 60s window hit the (toggle, category, region) cooldown key and were silently throttled. Without v0.5.2 dispatching guards the mesh would have been pummeled.
FIX 1 -- meshai/central/consumer.py:_normalize title fallback. The old chain was:
title = (data.get("title") or data.get("headline")
or cat_raw or f"{adapter} event")
Most Central adapters per the v0.10.0 guide §6 carry per-adapter payload fields (roadway, flux, magnitude, Kp, ...) but NOT a top-level title/headline. For those adapters the chain fell to cat_raw -- the raw Central hierarchical category like "incident.tomtom_incidents", "fire.hotspot.viirs_noaa20.high", "hydro.00060.usgs.06898000", "space.kindex", "quake.event.minor". That string became event.title, which compose_mesh_message() uses as the primary identifier in the friendly mesh line.
New chain inserts the meshai-friendly registry name BEFORE cat_raw:
friendly_name = get_category(category)["name"] # "Road Incident", "Wildfire Hotspot", ...
title = (data.get("title") or data.get("headline")
or friendly_name or cat_raw
or f"{adapter} event")
NWS and USGS quake supply title/headline directly and still take the first-priority slot. cat_raw stays as the last-resort tail for genuinely unknown categories. Per-adapter title synthesis (e.g. tomtom: f"{roadway} - {event_type}") is queued as v0.5.8 work -- intentionally out of scope here.
FIX 2 -- meshai/notifications/renderers/mesh.py:_format_one_line drops the [Family] prefix unconditionally. Pre-fix:
prefix = self._toggle_label(p.event_type) # -> "Roads", "Weather", ...
if prefix:
return f"[{prefix}] {p.message}" # legacy v0.5.0 debug format
return p.message
Since v0.5.2 the dispatcher hands payload.message from compose_mesh_message() whose output ALREADY starts with the family emoji + label ("🚨 ROADS:", "🔥 FIRE:", "⚠ WX:", "🌐 RF:", ...). The renderer wrap produced the visually-broken duplicate "[Roads] 🚨 ROADS: ...". The composer was supposed to be the single source of truth for mesh formatting; the renderer never got the memo.
Post-fix the renderer is a verbatim pass-through:
return p.message or ""
The _toggle_label() method and TOGGLE_LABELS table are KEPT (the digest renderer at notifications/pipeline/digest.py still uses them for the multi-line summary format -- do not remove them).
Why pytest did not catch this
-----------------------------
compose_mesh_message is unit-tested with synthetic Events that have clean titles; no test passes "incident.tomtom_incidents" as event.title to the composer. MeshRenderer.render is unit-tested with synthetic NotificationPayloads carrying legacy messages; no test feeds composer output into the renderer. The seam between consumer/composer/renderer was never end-to-end tested with a realistic Central envelope. New file tests/test_central_envelope_to_wire_v057.py closes that gap.
Tests
-----
PYTHONPATH=. pytest -q: 474 passed, 2 skipped (was 450 baseline; +24 net).
- tests/test_central_envelope_to_wire_v057.py (new): runs five representative Central envelopes (tomtom_incidents, FIRMS hotspot, NWS alert, USGS quake, SWPC alert) through _normalize -> dispatcher -> renderer and asserts the rendered wire string (a) does not start with "[", (b) does not contain any raw Central category token (".tomtom_incidents", ".firms", ".kindex", ".proton_flux"), (c) starts with the composer emoji+label, (d) for adapters lacking upstream title/headline, uses the registry-friendly name in the primary slot. Plus a focused regression-guard test test_matt_smoking_gun_no_longer_reproduces that asserts the exact 2026-06-04 15:40:30 wire string can no longer be produced.
- tests/test_renderers.py: test_mesh_render_event_type_prefix renamed to test_mesh_render_passes_message_verbatim with new assertion (no [Family] prefix); test_mesh_render_unknown_event_type_no_prefix updated for the verbatim contract.
Re-flip verification
--------------------
After the fix landed in container image sha256:0dea6ad3, the staged flip from earlier tonight was repeated in one shot (master + central + 8 adapters + 8 toggles all ON, container restart, 5-minute observation). All 12 v0.5.7-fixed Central subscriptions confirmed active, container healthy, ugly-format detector (grep for "[<Family>] " or raw-category tokens on the wire) saw zero hits, spam-fuse not tripped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
7502663f21 |
fix(tracking): v0.5.7-tracking -- Central tracking check + categories audit
EIGHTH and FINAL family of the v0.5.7 NATS-and-categories campaign. Smallest scope of the entire campaign: zero source code changes, regression-tests only. The tracking family is a Phase 7 PLACEHOLDER -- everywhere we looked it is intentionally empty.
FIX 1 -- Central tracking adapter check: VERIFIED ABSENT in Central v0.10.0. Searched the consumer integration guide (docs/CONSUMER-INTEGRATION.md at v0.10.0-itd-511): 22 per-adapter sections in §6, none tracking-related (no APRS / ADS-B / OpenSky / aircraft / satellite). Subject prefixes in use: central.{disaster,fire,fires,hydro,meta,models,quake,space,traffic,traffic_cameras,traffic_flow,wx}.> -- no central.tracking.* / central.aprs.* / central.adsb.*. Producer source tree src/central/adapters/ has 24 adapter files; none tracking-named.
meshai already accounts for this -- _SUBJECTS_BARE has no `tracking` / `aprs` / `adsb` key; CENTRAL_ADAPTER_TO_SOURCE has no tracking entries on either side; _subjects_for("tracking", *) returns [] for any region (no key in the table). No code change needed.
FIX 2 -- ALERT_CATEGORIES tracking-family audit: VERIFIED EMPTY-BY-DESIGN. The tracking family is documented at meshai/notifications/categories.py:19 as "tracking - ADS-B, AIS, satellite passes (Phase 7)" -- a reserved-but-unimplemented family. State as of v0.5.7-tracking:
- "tracking" in VALID_TOGGLES (the toggle name is reserved)
- ZERO ALERT_CATEGORIES entries with toggle="tracking"
- ZERO native adapter files in meshai/env/ (no aprs.py, adsb.py, etc.)
- ZERO entries in _TOGGLE_PREFIX_FALLBACK routing to "tracking"
- dashboard-frontend/src/pages/Environment.tsx FAMILIES list has the
placeholder { key: "tracking", label: "Tracking", icon: Satellite,
adapters: [] } -- empty adapter list
No native emissions to audit against. Every emitted = selectable is trivially satisfied: zero emissions, zero registry entries, zero gap. No code change needed.
Regression guards (the whole commit)
------------------------------------
The risk this commit guards against is: a future Phase 7 implementer adds an APRS or ADS-B adapter (or wires a tracking Central subject) without doing the family-audit shape (registry entries with required fields, composer emoji/labels, paired test refresh). That would create orphan emissions -- the exact failure mode v0.5.7 spent eight phases eliminating across the other families.
tests/test_tracking_v057.py pins ALL of the above placeholder invariants as regression guards:
- No Central tracking-style subject prefixes anywhere in _SUBJECTS_BARE
(sweeps every adapter sub for "tracking" / "aprs" / "adsb" / "opensky").
- No tracking-style entries in CENTRAL_ADAPTER_TO_SOURCE on either side.
- _subjects_for("tracking" / "aprs" / "adsb", *) returns [] for every region.
- "tracking" reserved in VALID_TOGGLES.
- Zero ALERT_CATEGORIES entries with toggle="tracking".
- Zero env adapter files matching tracking-related needles.
- Environment.tsx FAMILIES tracking entry has adapters=[] (regex-pinned).
- No _TOGGLE_PREFIX_FALLBACK rule silently routes to tracking.
Each failure message points the future implementer at the paired changes required -- new registry entries, new composer glyphs, new env adapter file, paired test update. Phase 7 cannot ship partial.
Same outcome shape as v0.5.7-avalanche (Central has no counterpart; meshai handles it correctly already; regression-test-only commit).
Tests
-----
PYTHONPATH=. pytest -q: 450 passed (was 442; +8 net).
- tests/test_tracking_v057.py (new): eight regression guards covering Central absence, meshai-side placeholder state, frontend placeholder, and the no-silent-routing safety.
Campaign-wide test count progression across the eight v0.5.7 families:
v0.5.7-weather 345 (baseline 328 + 17)
v0.5.7-traffic 366 (+21)
v0.5.7-fire 380 (+14)
v0.5.7-seismic 400 (+20)
v0.5.7-water 413 (+13)
v0.5.7-rf 431 (+18)
v0.5.7-avalanche 442 (+11)
v0.5.7-tracking 450 (+8)
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. v0.5.7 tag deferred -- waiting on explicit instruction after Matt reviews the campaign-wide summary.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
7f8633aed5 |
fix(avalanche): v0.5.7-avalanche -- Central avalanche check + categories audit
Seventh family of the v0.5.7 NATS-and-categories campaign. Smaller scope than prior families (consumer.py unchanged): the Central side is verifiably empty, and the registry-audit gap is a single-entry add.
FIX 1 -- Central avalanche adapter check: VERIFIED ABSENT in Central v0.10.0. Searched the consumer integration guide (docs/CONSUMER-INTEGRATION.md at v0.10.0-itd-511) -- zero `avalanche` / `NWAC` / `CAIC` references. Searched the producer source tree (src/central/adapters/) -- no avalanche-named adapter files. meshai already accounts for this:
- meshai/central/consumer.py _SUBJECTS_BARE has no `avalanche` key, so
_subjects_for("avalanche", *) returns [] regardless of region.
- CENTRAL_ADAPTER_TO_SOURCE has no avalanche entry on either side.
- _subject_owned() (consumer.py line 334-) explicitly logs a warning
if someone flips avalanche.feed_source=central, then skips
subscribing.
No code change needed for FIX 1. Tests now pin these invariants so a future refactor that introduces an unexpected avalanche Central wire breaks loudly here.
FIX 2 -- ALERT_CATEGORIES avalanche-family audit. Native meshai/env/avalanche.py emits two categories from the NWAC/CAIC danger-level tier:
danger_level >= 4 (High, Extreme) -> avalanche_warning
danger_level == 3 (Considerable) -> avalanche_watch
danger_level <= 2 (Low, Moderate) -> silently dropped (not actionable)
Pre-v0.5.7-avalanche registry had avalanche_warning + avalanche_considerable. avalanche_warning matched the native emit. avalanche_considerable was a LEGACY name for the Considerable-danger tier -- the native code already emits avalanche_watch for that same semantic (verified at meshai/env/avalanche.py:266; tests/test_adapter_avalanche.py:90 asserts the mapping).
So avalanche_watch was MISSING from the registry, leaving the rule editor unable to target danger-level=3 emissions even though they were correctly routed to toggle="avalanche" via the `("avalanche", "avalanche")` prefix fallback.
Added avalanche_watch under toggle="avalanche", default_severity="routine", with a description that points at the Considerable-tier semantics and an example_message matching the live NWAC product phrasing. composer._CATEGORY_EMOJI and _CATEGORY_LABEL gained matching entries so live LoRa rendering shows the right glyph (⛷, label "AVY").
Legacy entry kept: avalanche_considerable remains in the registry as a forward-compat target even though no current code path emits it. Reasoning matches the v0.5.7-rf precedent:
- router.py source-attribution tables (lines 317, 429) reference it
- composer.py emoji + label tables reference it
- A future phase might re-emit avalanche_considerable as a finer-grained
distinction from the generic Watch label; removing the registry entry
would break any user rule currently targeting it.
If avalanche_considerable remains un-emitted by v0.6, file a follow-up cleanup phase to remove it together with the rf-family hf_blackout / tropospheric_ducting legacy entries. test_alert_categories_avalanche_complete uses a SUBSET assertion (native emit ⊆ registry) so the legacy entry is allowed.
Audit table after v0.5.7-avalanche:
Registry avalanche (3):
avalanche_warning (native danger_level >= 4)
avalanche_watch [v0.5.7-avalanche NEW] (native danger_level == 3)
avalanche_considerable (legacy, no current emitter)
Native emit: {avalanche_warning, avalanche_watch} ⊆ Registry -- parity for everything emitted.
Tests
-----
PYTHONPATH=. pytest -q: 442 passed (was 431; +11 net).
- tests/test_avalanche_v057.py (new): _subjects_for("avalanche", *) returns [] for every region input; avalanche absent from _SUBJECTS_BARE and CENTRAL_ADAPTER_TO_SOURCE; flipping avalanche.feed_source=central produces zero subscriptions; avalanche_watch present under toggle="avalanche" with required fields; avalanche_warning + avalanche_considerable still registry-present; native emit set equals {avalanche_warning, avalanche_watch} and is a subset of the registry.
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
6c84baf12c |
fix(rf): v0.5.7-rf -- SWPC subject validation + protons severity=0 documentation + categories audit
Sixth family of the v0.5.7 NATS-and-categories campaign. RF family = native ducting calculator + three Central SWPC adapters (swpc_alerts, swpc_kindex, swpc_protons), all umbrella-subscribed under `central.space.>`.
Per the family-by-family pattern: cross-checked every prompt assumption against the Central v0.10.0 guide before implementing. The big surprise this phase: FIX 1 was already correct (no NATS-syntax bug to fix), and FIX 2 was a non-bug too (severity=0 already routes safely). The real work was FIX 3 -- four missing registry entries that meshai emits but the rule editor couldn't target.
FIX 1 -- SWPC subject pattern (already correct; pinned). Per Central v0.10.0 guide §swpc_alerts / §swpc_kindex / §swpc_protons, all three adapters publish under the `central.space.>` umbrella with no region in subject (space weather is planetary):
swpc_alerts: central.space.alert.<product_id> (4 tokens, product_id tail)
swpc_kindex: central.space.kindex (3 tokens, fixed)
swpc_protons: central.space.proton_flux (3 tokens, fixed)
`_subjects_for("swpc", region)` already returned `["central.space.>"]` ignoring region (v0.5.4 work got this right). Added an explanatory inline comment near the table entry calling out each adapter's concrete subject + the universal severity=0 contract (next fix), plus a test pinning the umbrella + region-ignored behavior + coverage of each per-adapter subject form. Future "let me add a region tail here" refactors will fail loudly.
FIX 2 -- swpc_protons severity=0 routing (non-bug; regression-guard pin). The prompt described a "severity=0 silently dropped" failure mode. Investigation: no such bug exists in current code.
- All three SWPC adapters publish severity=0 in the live guide samples.
- consumer.map_severity already maps 0 -> "routine" (the `if sev >= 3:`
immediate clamp doesn't hit; falls through to the default return).
- NotificationToggle.severity_channels is dict-keyed by severity STRING
(locked in by v0.5.7-seismic test_severity_channels_is_string_keyed_no_int_indexerror_risk);
"routine" is a valid key with no IndexError vector.
Three things tightened anyway: (a) inline comment near the swpc subject entry documenting "all three publish severity=0 -> routine per guide examples"; (b) end-to-end synthetic envelope test for swpc_protons injection (severity=0 in, ev.severity="routine" / ev.category="solar_radiation_storm" / ev.source="swpc" out, no exception); (c) parallel test for swpc_kindex confirming a second SWPC adapter wires identically.
FIX 3 -- ALERT_CATEGORIES rf_propagation audit. Pre-v0.5.7-rf registry had three entries under toggle="rf_propagation": hf_blackout, geomagnetic_storm, tropospheric_ducting. Audit:
Native ducting.py emits via _TIER_CATEGORY:
super_refraction -> rf_anomalous_propagation
duct -> rf_ducting_enhancement
surface_duct -> rf_ducting_enhancement
Central path via map_category:
space.alert.* -> rf_propagation_alert (swpc_alerts)
space.kindex -> geomagnetic_storm (swpc_kindex; already in registry)
space.proton_flux -> solar_radiation_storm (swpc_protons)
space.* catchall -> geomagnetic_storm
Four categories emitted but missing from the registry -- rule editor couldn't target them. Added all four under toggle="rf_propagation" with name + description + default_severity + example_message matching the guide-documented behavior:
rf_anomalous_propagation (routine, ducting super_refraction tier)
rf_ducting_enhancement (priority, ducting duct + surface_duct tiers)
rf_propagation_alert (priority, NOAA SWPC space-weather product)
solar_radiation_storm (priority, GOES proton flux S-scale)
composer.py emoji + label tables gained matching entries so live LoRa rendering shows the right glyphs (📡 for ducting forms, ⚠ for SWPC alerts, 🌐 for solar radiation, all labelled "RF").
Legacy entries kept (forward-compat / no current emitter): hf_blackout and tropospheric_ducting remain in the registry as selectable rule targets even though no current code path emits them. Reasoning:
- hf_blackout: HF-specific R-scale parsing of swpc_alerts.message could
re-introduce this emission in a future phase; removing the registry
entry would break any user rule currently configured to target it.
- tropospheric_ducting: legacy name superseded by rf_ducting_enhancement
in native ducting.py; same forward-compat concern -- a future phase
may emit a "tropospheric" specialization separate from generic ducts.
If either remains un-emitted by v0.6, file a follow-up cleanup phase to remove. Test_alert_categories_rf_complete uses a SUBSET assertion (emit set ⊆ registry) rather than equality so legacy entries are allowed.
Audit table after v0.5.7-rf:
Registry rf_propagation (7):
hf_blackout (legacy, no current emitter)
geomagnetic_storm (central swpc_kindex + catchall)
tropospheric_ducting (legacy, no current emitter)
rf_anomalous_propagation [v0.5.7-rf NEW] (native ducting super_refraction)
rf_ducting_enhancement [v0.5.7-rf NEW] (native ducting duct + surface_duct)
rf_propagation_alert [v0.5.7-rf NEW] (central swpc_alerts)
solar_radiation_storm [v0.5.7-rf NEW] (central swpc_protons)
Emit set ⊆ Registry: TRUE (no orphan emissions).
Tests
-----
PYTHONPATH=. pytest -q: 431 passed (was 413; +18 net).
- tests/test_rf_v057.py (new): umbrella subject is `central.space.>` for all regions; per-adapter published subjects all match; map_severity(0) -> "routine"; NotificationToggle.severity_channels dict-keyed (no IndexError); synthetic swpc_protons + swpc_kindex envelopes route cleanly with severity=0; four new rf_propagation entries all registry-present with required fields; geomagnetic_storm still mapped from space.kindex; map_category routing pinned for each SWPC adapter; native ducting + central SWPC emit sets are subsets of registry rf entries.
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
730b03099a |
fix(water): v0.5.7-water -- USGS NWIS hydro NATS pattern + categories audit
Fifth family of the v0.5.7 NATS-and-categories campaign. Water/hydro = USGS NWIS adapter (Central calls it `nwis`, meshai calls it `usgs` via CENTRAL_ADAPTER_TO_SOURCE remap). USGS quake stays in the seismic phase that already shipped; this phase fixes the hydro/water subscription and audits the water-side of the seismic toggle.
The Central v0.10.0 consumer integration guide was the starting point but the §nwis section text turned out to be STALE w.r.t. the regional subject suffix. Ground-truthed against the v0.10.0-itd-511 nwis.py producer subject_for() body and used the CODE as the source of truth. Documented below.
FIX 1 -- USGS NWIS hydro NATS pattern. Pre-v0.5.7-water `_subjects_for("usgs","us.id")` returned `["central.hydro.>.us.id", "central.hydro.>.unknown"]`. Both subjects are invalid NATS (`>` is only legal at the tail token).
The producer code at v0.10.0-itd-511 src/central/adapters/nwis.py:223 publishes:
central.hydro.<param>.<agency>.<site>.<region>
where <region> is either `us.<state>` (2 tokens) or `unknown` (1 token). So the live subjects on the broker are 7 tokens (per-state) or 6 tokens (unknown). The doc §nwis section text shows only the 4-token category stem `central.hydro.<parameter_code>.<agency>.<bare_site_no>` -- that text is stale; it predates the regional-routing roll-out.
Fixed by using three single-token `*` wildcards in the param/agency/site slots plus the bare region tail. Preserves the v0.5.4 INTENT (server-side regional filtering + .unknown workaround for gauges whose state Central can't resolve) while restoring NATS syntax legality:
central.hydro.*.*.*.us.id (7 tokens, per-state)
central.hydro.*.*.*.unknown (6 tokens, .unknown workaround)
Bare-form fallback (`central.hydro.>`) is unchanged for empty/None region (pre-v0.5.3 backward compat path).
FIX 2 -- ALERT_CATEGORIES water/hydro audit. Pre-v0.5.7-water registry had `stream_flood_warning` and `stream_high_water` (both toggle="seismic" from the v0.5.2 USGS-water -> Geohazards migration). Audit findings:
- Native usgs.py applies NWPS flood-stage thresholds client-side and emits
`stream_flood_warning` (reading at/above flood stage) or `stream_high_water`
(Action Stage reading). Routine gauge readings below action stage are
silently dropped on the native path (no spam).
- Central path: every NWIS reading arrives with category=`hydro.<pcode>.
<agency>.<site>` at severity=0. consumer._CATEGORY_MAP maps `hydro.*`
to `stream_flow` (added in earlier work). But `stream_flow` was MISSING
from ALERT_CATEGORIES -- routing worked via the `("stream", "seismic")`
prefix fallback, but the Advanced Rules editor couldn't target raw
central-delivered gauge readings.
Added `stream_flow` to ALERT_CATEGORIES under toggle="seismic", default_severity="routine", with an example_message that reflects the raw-reading shape. The existing `stream_flood_warning` / `stream_high_water` entries are unchanged.
NOTE on parity gap (deferred to v0.5.8+): meshai does NOT currently re-apply NWPS threshold logic to central-delivered NWIS readings. So flipping `usgs.feed_source=central` today produces a stream of routine `stream_flow` events without the flood-stage classification the native path provides. Bringing the central path to parity (apply threshold logic AFTER receiving central-delivered raw readings) is queued as future work -- intentionally out of scope here per Matt's one-fix-per-family rule.
Audit table after v0.5.7-water:
Native emit: stream_flood_warning, stream_high_water (threshold-triggered)
Central path: every hydro.* -> stream_flow (routine; no threshold)
Registry: {stream_flow, stream_flood_warning, stream_high_water} (toggle=seismic)
Quake side: earthquake_event (toggle=seismic, added v0.5.7-seismic) -- unchanged
Parity confirmed. No orphans, no missing.
Tests
-----
PYTHONPATH=. pytest -q: 413 passed (was 400; +13 net).
- tests/test_water_v057.py (new): usgs subscription is NATS-legal (no `>` anywhere, all single-token `*`); token-count matches the producer-published shape (7-token us-state + 6-token .unknown); per-region substitution (Montana sanity); bare-form backward compat; `stream_flow` present under toggle="seismic"; `stream_flood_warning` / `stream_high_water` unchanged; native + central emit set matches registry water-side subset; threshold categories still emitted by usgs.py; all realistic central pcodes fold to `stream_flow`; required-fields check; severity=0 -> "routine" sanity.
- tests/test_central_region_routing.py: updated `test_subjects_for_usgs_includes_unknown_workaround` to reflect the v0.5.7-water fix (single-token `*` wildcards instead of mid-subject `>`).
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
8359451eaa |
fix(seismic): v0.5.7-seismic -- USGS quake NATS pattern + severity=5 great-quake clamp + categories audit
Fourth family of the v0.5.7 NATS-and-categories campaign. Seismic = USGS quake adapter (the USGS water/hydro side stays under toggle="seismic" per the v0.5.2 geohazards migration but lives in the water phase that follows).
The Central v0.10.0 consumer integration guide (docs/CONSUMER-INTEGRATION.md on the v0.10.0-itd-511 branch) was treated as source of truth -- ground-truthing the prompt against the guide caught two prompt errors before they shipped (mirrors the FIRMS situation in v0.5.7-fire). Documented below.
FIX 1 -- USGS quake NATS pattern. Pre-v0.5.7-seismic `_subjects_for("usgs_quake","us.id")` returned `["central.quake.event.>.us.id"]`. That subject is BOTH invalid NATS (`>` is only legal at the tail token) AND wouldn't have matched anything Central publishes.
Per Central v0.10.0 guide §usgs_quake the actual published subject is `central.quake.event.<tier>` -- 4 tokens, no region. `<tier>` is one of {minor, light, moderate, strong, major, great} (USGS magnitude bands; bands live in the SUBJECT, not in the severity integer).
Note on prompt vs. guide discrepancy: the v0.5.7-seismic prompt described a "regional v0.9.20+ shape" `central.quake.event.<severity>.us.<state>` with 6 tokens and `us.<state>` at the tail. That's neither what Central v0.10.0 publishes nor what its guide documents. We follow the guide. Subscribing to the prompt's shape would silently match zero messages in production. State filtering for quakes happens client-side via data.latitude/longitude (same situation as FIRMS).
New subscription: `central.quake.event.>` -- tail-only `>`, NATS-legal, matches all <tier> values.
FIX 2 -- severity=5 great-quake clamp (no actual bug; regression-guard pin). The prompt described a "severity=5 IndexError or silent drop" failure mode. Investigation found NO such bug exists in the current code:
- consumer.map_severity already clamps `sev >= 3` to "immediate". A
severity=5 (or 99, or any 3+) maps safely to "immediate" with no
exception path.
- NotificationToggle.severity_channels is dict-keyed by severity STRING
({"routine","priority","immediate"}), not an int-indexed list, so
IndexError is structurally impossible from this boundary regardless
of upstream value.
- Per Central v0.10.0 guide §5b the documented severity vocabulary is
`0-4 or None`. Severity=5 is not in the published contract; the
clamp is defensive padding against future contract drift.
Three things were tightened anyway: (a) the map_severity docstring now explicitly documents the high-side clamp behavior and calls out the string-keyed dict guarantee; (b) parametrized test pins map_severity for the full 0..99 range including out-of-contract values; (c) an end-to-end synthetic-envelope test injects a severity=5 quake through _handle and asserts the resulting Event has severity="immediate" / category="earthquake_event" / source="usgs_quake" with no exception. These tests function as regression guards if a future refactor introduces the IndexError vector the prompt was guarding against.
FIX 3 -- ALERT_CATEGORIES seismic-family audit. The registry was MISSING `earthquake_event` entirely. Both native (`usgs_quake.py` emits `category="earthquake_event"`) and central (consumer._CATEGORY_MAP maps `quake.* -> earthquake_event`) paths produce that category, but get_category("earthquake_event") fell through to the mesh_health default -- so the Advanced Rules editor couldn't target quakes at all. The get_toggle() prefix fallback DID route it to "seismic" via the `("earthquake", "seismic")` rule, so events were filtered correctly; the gap was UI-selectability only.
Added the entry under toggle="seismic" with a representative example_message. composer.py already had matching emoji/label mappings (line 78-79, 107-108) from earlier work, no composer change needed.
The two hydro entries (`stream_flood_warning`, `stream_high_water`) also live under toggle="seismic" via the v0.5.2 USGS-water migration (Geohazards family in the GUI). They are OUT OF SCOPE for v0.5.7-seismic -- they belong to the water phase that follows. Verified-unchanged here so the next phase has a clean baseline.
Audit table after v0.5.7-seismic:
Native emit: usgs_quake.py -> earthquake_event
Central path: all 6 tiers (minor/light/moderate/strong/major/great) -> earthquake_event
Registry: {earthquake_event, stream_flood_warning, stream_high_water}
Quake side: parity (registry has earthquake_event; native + central emit it)
Hydro side: verified-unchanged (deferred to water phase)
Tests
-----
PYTHONPATH=. pytest -q: 400 passed (was 380; +20 net).
- tests/test_seismic_v057.py (new): quake subject tail-only `>`; no mid-subject `>`; bare-form backward compat; parametrized map_severity full range 0..99 + None / nonsense / negative; synthetic severity=5 envelope routes through _handle to severity="immediate" cleanly; NotificationToggle.severity_channels shape pinned to dict (no IndexError vector); earthquake_event present under toggle="seismic"; hydro entries still toggle="seismic" (regression guard); native + central-path quake emit set equals {earthquake_event}; required-fields check.
- tests/test_central_region_routing.py: updated `test_subjects_for_usgs_quake_us_id` -> `test_subjects_for_usgs_quake_us_id_uses_tail_only_wildcard` reflecting the guide-correct shape.
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
60e8e62e85 |
fix(fire): v0.5.7-fire -- FIRMS NATS pattern + WFIGS tombstone dedup + remove fire_proximity + categories audit
Third family of the v0.5.7 NATS-and-categories campaign. Fire is the heaviest of the campaign -- four distinct fixes plus a category audit. Two of the four were broken in production: FIRMS subscribed to a syntactically invalid pattern, and WFIGS tombstones were silently dropped.
FIX 1 -- FIRMS NATS pattern (the canonical bug). Pre-v0.5.7-fire `_subjects_for("firms","us.id")` returned `["central.fire.hotspot.>.us.id"]`, which is INVALID NATS (the `>` multi-level wildcard is only legal at the tail token). It also wouldn't have matched anything Central publishes: per the Central v0.10.0 consumer integration guide §firms, the actual published pattern is `central.fire.hotspot.<satellite>.<confidence>` (5 tokens, no us.<state> suffix). The two slots after "hotspot" are satellite name and confidence band -- NOT tile coordinates or region tokens.
Note on prompt vs. guide discrepancy: the v0.5.7-fire task spec described a tile-coord/state pattern `central.fire.hotspot.*.*.us.id` (7 tokens with us.<state> tail). That's neither what Central v0.10.0 publishes nor what its guide documents. We follow the guide. Subscribing to the prompt's 7-token pattern would silently match zero messages in production (token-count mismatch). State filtering for FIRMS happens client-side via data.latitude / data.longitude against the configured region bbox.
New subscription: `central.fire.hotspot.>` -- tail-only `>`, NATS-legal, matches all <satellite>.<confidence> combinations.
FIX 2 -- WFIGS tombstone subjects. Per guide §wfigs_incidents and §wfigs_perimeters, WFIGS publishes:
active: central.fire.incident.<state>.<county> (Convention A, depth-3 state)
active: central.fire.perimeter.<state>.<county>
tombstone: central.fire.incident.removed.<state> (5 tokens, "removed" at depth-3)
tombstone: central.fire.perimeter.removed.<state>
Pre-v0.5.7-fire `_subjects_for("fires","us.id")` subscribed only to the active subjects (`central.fire.incident.id.>` and `central.fire.perimeter.id.>`). The tombstone subjects have "removed" at depth-3 instead of the state token, so the active-subject `>` filters silently dropped EVERY tombstone. Fall-off signals never reached meshai's inhibitor, so old incidents stayed "live" in the pipeline indefinitely.
Added the two tombstone subjects to the subscription list. Both are 5-token literals with no wildcards -- trivially NATS-legal.
FIX 3 -- WFIGS tombstone dedup. Per guide §wfigs_incidents removal semantics, the tombstone env_id has the shape `<IrwinID>:removed:<iso_now>` -- the `:removed:` is sandwiched in the middle, with a timestamp tail. Pre-v0.5.7-fire the consumer.py group_key recovery was `re.sub(r":removed$", "", group_key)` -- a literal trailing `:removed` match -- which DID NOT FIRE on the WFIGS form (the regex required `:removed` at the very end of the string, but the WFIGS form has `:<iso>` after it).
Consequence: WFIGS tombstones' group_key was the full `<IrwinID>:removed:<iso>` string instead of the bare `<IrwinID>`. The pipeline grouper/inhibitor never matched tombstones to their original incidents, so the lapse signal was lost.
Fixed by switching the regex to `re.sub(r":removed(:.*)?$", "", group_key)` -- handles both the WFIGS `<IrwinID>:removed:<iso>` form AND the legacy GDACS `<id>:removed` form. The `is_tombstone` detection also gained an explicit `":removed:" in env_id` check for the WFIGS shape.
Per the guide: "the same incident can have one or more removal tombstones over its lifecycle" (it can re-enter and re-fall-off). To preserve per-tombstone distinctness for downstream lifecycle accounting, the full env_id is stashed on `Event.data["_central_tombstone_id"]` (the group_key collapses to the IrwinID by design, but the original env_id with the :<iso> tail survives on data).
FIX 4 -- ALERT_CATEGORIES fire-family audit + removed parametric entries. Per Matt's direct feedback ("fire near mesh has its own set of parameters that I don't even know what they could be. like how far is near mesh? I don't know I can't set that."), the parametric `fire_proximity` and the duplicate-named `wildfire_proximity` (both labeled "Fire Near Mesh" with parametric radius-based descriptions) were unselectable in the new Advanced Rules UI. Removed both.
Cross-referenced what FIRMS and WFIGS actually emit (per the guide and the native adapter code) and audited the registry:
Native emit:
firms.py -> new_ignition (when adapter flags new_ignition)
or wildfire_hotspot (otherwise) [v0.5.7-fire: was wildfire_proximity]
fires.py -> wildfire_incident
Central path emit (via map_category):
fire.hotspot.* -> wildfire_hotspot
fire.incident.* -> wildfire_incident
fire.perimeter.* -> wildfire_incident (perimeters merge to the incident)
fire.<other> -> wildfire_incident (catchall)
Registry after v0.5.7-fire:
{new_ignition, wildfire_hotspot, wildfire_incident}
Parity confirmed. No orphans, no missing.
Aligning firms.py to emit `wildfire_hotspot` (matching the central FIRMS map) means native + central FIRMS produce identical categories regardless of which feed path is enabled.
Composer (`_CATEGORY_EMOJI`, `_CATEGORY_LABEL`) and router (three source-attribution tables) updated to drop the removed categories and add the new ones.
Deferred to v0.5.8: distance_max_km field on rules for actual proximity filtering. Replaces the parametric fire_proximity registry entry with a parameterized rule field that the user CAN configure ("alert me about wildfire_incident within 30 km" instead of an opaque "Fire Near Mesh" toggle).
Tests
-----
PYTHONPATH=. pytest -q: 380 passed (was 366; +14 net).
- tests/test_fire_v057.py (new): FIRMS subject is tail-only `>` with no mid-subject placement; WFIGS subjects cover active + four tombstones; WFIGS tombstone strips `:removed(:.*)?$` for group_key; two same-IrwinID tombstones both propagate through _handle and share group_key, with the original env_id preserved on data["_central_tombstone_id"]; legacy GDACS `:removed` shape still strips cleanly; fire_proximity / wildfire_proximity absent from ALERT_CATEGORIES; no "Fire Near Mesh" name duplicates; fire-family parity (native + central emit == registry); required-fields check on the three fire entries.
- tests/test_central_region_routing.py: updated FIRMS test (tail-only `>`) and WFIGS test (includes tombstone subjects).
- tests/test_pipeline_toggle_filter.py, tests/test_adapter_firms.py, tests/test_v052_dispatcher.py, tests/test_pipeline_digest.py: bulk-migrated obsolete category references (wildfire_proximity -> wildfire_hotspot, fire_proximity -> wildfire_incident) so the existing test suites continue to exercise the same routing/digest/dispatch paths with the new category names.
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
14d168822b |
fix(traffic): v0.5.7-traffic -- NATS pattern fix + itd_511 sub-adapter routing + categories audit
Second family of the v0.5.7 NATS-and-categories campaign. Weather went first because its NWS pattern was already legal; traffic was carrying invalid NATS syntax in production.
FIX 1 -- Invalid `>` mid-subject in traffic. Pre-v0.5.7-traffic the subject builder shipped `central.traffic.>.{state}` for both the traffic and roads511 adapters. NATS rules say `>` is only legal at the tail token; mid-subject `>` is rejected by the broker at subscribe time (or silently delivers nothing depending on server version). Replaced with Convention B (per Central v0.10.0 meshai_integration_guide.md): single-token `*` in the event_type slot, bare state suffix -- `central.traffic.*.id` for Idaho. Shared by the wzdx, tomtom_incidents and state_511_atis adapters.
FIX 2 -- roads511 dual subscribe. The new Idaho-only itd_511 adapter in Central v0.10.0 uses Convention A (`central.traffic.<event_type>.us.<state>`, the us.<state> form). Convention B (bare state) is shared with the rest of the traffic family. roads511 now owns BOTH:
central.traffic.*.id (Convention B, shared with traffic via _subject_owned)
central.traffic.*.us.id (Convention A, itd_511-only)
Sub-adapter routing in CentralConsumer._subject_owned (v0.5.1) already keeps shared subjects scoped to the right meshai source -- no change needed.
FIX 3 -- itd_511 -> roads511 in CENTRAL_ADAPTER_TO_SOURCE. Mirrors state_511_atis (added v0.5.3). Both Idaho 511 feeds collapse to a single meshai source for UX simplicity; future v0.6 may split them if Matt needs differential rules.
FIX 4 -- Roads-family categories audit + finer event_type mapping. Pre-v0.5.7-traffic the central path flattened every traffic-domain event to `traffic_congestion` because work_zone / incident / closure had no entries in _CATEGORY_MAP and fell through to the `traffic.` catchall (then the subject-domain fallback). Added three explicit map entries before the catchall:
("work_zone", "work_zone") # catches "work_zone" and "work_zone.wzdx"
("incident", "road_incident") # catches incident.tomtom_incidents + bare
("closure", "road_closure") # catches closure + closure.itd_511
ALERT_CATEGORIES gains two new roads-family entries so the Advanced Rules editor can target them:
work_zone -- Active construction/maintenance work zone
road_incident -- Reported incident (crash, hazard, debris)
Existing entries `road_closure` and `traffic_congestion` kept. composer._CATEGORY_EMOJI gains matching glyphs (🚧 work_zone, 🚨 road_incident) so the live LoRa rendering lines up with the category example_message glyphs.
Audit cross-check (test_alert_categories_roads_complete enforces parity):
Native emit: traffic.py -> traffic_congestion; roads511.py -> road_closure
Central path emit (via map_category): {road_closure, traffic_congestion, work_zone, road_incident}
ALERT_CATEGORIES{toggle=roads}: {road_closure, traffic_congestion, work_zone, road_incident}
Parity. No orphans, no missing.
DEFERRED to v0.5.8: itd_511_cameras / traffic_cameras stream lives at a different subject domain (central.traffic_cameras.>) and needs a new meshai source (roads_cameras or similar). Out of scope for v0.5.7.
Tests
-----
PYTHONPATH=. pytest -q: 366 passed (was 345; +21 net).
- tests/test_traffic_v057.py (new): NATS-syntax checks (`>` only at tail, single-token `*`), traffic Convention B, roads511 dual-subscribe, shared bare-state subject, itd_511 + state_511_atis remap, map_category event_type preservation, ALERT_CATEGORIES roads parity (reflection-based scan of native emit + central path), required-fields check on the four roads entries.
- tests/test_central_region_routing.py: updated `test_subjects_for_traffic_and_roads511_share_state_token` -> two new tests covering Convention B (traffic) and dual-subscribe (roads511).
- tests/test_central_consumer.py: updated `test_subject_domain_fallback_for_unmapped_category` (work_zone.wzdx is now mapped, switched to a genuinely-unmapped category) + new `test_v057_traffic_work_zone_now_mapped` asserting wzdx envelopes land on ev.category=="work_zone".
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
b87696bf67 |
fix(weather): v0.5.7-weather -- NWS HTML strip + ALERT_CATEGORIES audit (NATS pattern already valid)
First family of the v0.5.7 NATS-and-categories campaign (Matt review of Central v0.10.0 meshai_integration_guide.md). Weather lands first because the NWS NATS pattern is already legal; the other five families need invalid mid-subject > rewrites that will ship per-family.
FIX 1 -- NWS NATS pattern validated. _subjects_for("nws", "us.id") -> ["central.wx.alert.us.id.>"]. The wildcard token > sits at the tail only (token index -1), so the subject is a legal NATS multi-level wildcard. No code change. Live introspection confirmed in-container.
FIX 2 -- NWS HTML strip in mesh composer. Per Central guide Surprise 3, data["description"] and data["instruction"] arrive as raw HTML (<p>, <br>, <strong>, , —, ...). Until now the composer fed event.title / event.summary straight to LoRa, so any future title/summary populated from those fields would have leaked literal markup onto the wire.
Added strip_html_tags(text) -> str in meshai/notifications/renderers/composer.py. Block-level tags (br, p, div, li, tr, h1-h6) become a single space so adjacent paragraphs do not fuse; all other tags are removed; HTML entities are decoded via html.unescape; whitespace is collapsed. Applied in _primary_identifier (title and summary paths) and _region_segment BEFORE byte-budget truncation, so the 150 B cap counts real glyphs, not markup. Universal (not NWS-gated) since strip is a no-op on plain text -- protects against future adapters that surface raw HTML too.
FIX 3 -- ALERT_CATEGORIES weather audit. Cross-referenced ALERT_CATEGORIES{toggle="weather"} against meshai/env/nws.py:_derive_category() emission set:
nws.py emits: weather_warning, weather_watch, weather_advisory, weather_statement
registry weather: weather_warning, weather_watch, weather_advisory, weather_statement
Parity. No additions, no removals. The v0.5.2 stream_* migration to the seismic family (USGS hydro under the GUI Geohazards tab) is already reflected; weather is clean at 4 entries. Added a comment block above the weather section pointing at test_alert_categories_weather_complete which now enforces this parity going forward -- if a new branch is added to _derive_category(), the test fails and forces a matching registry entry.
Tests
-----
PYTHONPATH=. pytest -q: 345 passed (was 328; +17 new in tests/test_weather_v057.py).
- strip_html_tags: simple tags, br/paragraph -> space, entity decode (& —), nested/attrs, plain-text no-op, empty input, whitespace collapse.
- compose_mesh_message integration: HTML in title scrubbed; HTML in summary fallback scrubbed; 150 B budget still holds.
- Weather parity: reflection-based scan of NWSAlertsAdapter._derive_category() vs registry; both must match.
- Required-fields check on the four weather entries.
Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Johnson
|
d49e417400 |
feat(dashboard): v0.5.6 -- Advanced Rules editor polish (grouped categories, region_scope, name tooltip, smart activity badge)
v0.5.6
Frontend-only polish to the Advanced Rules section in Notifications.tsx. Master Toggles (v0.5.0) and safe-mode are untouched.
FIX 1 -- Grouped Alert Categories. Replace the flat ~45-item checkbox list with a per-family grouped picker. Each family (mesh_health/weather/fire/rf_propagation/roads/avalanche/seismic/tracking) is a collapsible section with the lucide icon used by Master Toggles, a category count in the header, and per-family "All" / "Clear" bulk-toggle buttons. Families that already have selections expand by default. Categories whose toggle field does not match a known family fall into an "Other" group at the bottom. Uses the backend toggle field already provided by /api/notifications/categories.
FIX 2 -- region_scope multi-select. Adds a REGIONS block between WHEN and SEND VIA, wired to rule.region_scope (NotificationRuleConfig backend field has existed since v0.5.0). Fetches /api/regions alongside config/categories. Pill-style toggle buttons; empty selection means all regions (backward compat). region_scope added to createDefaultRule; addFromTemplate merges over createDefaultRule so future config fields do not need to be backfilled into every literal template.
FIX 3 -- truncated rule name hover tooltip. Adds title={rule.name} to the collapsed-header name span so long rule names stay readable on hover.
FIX 4 -- Smart activity badge. Replaces the unconditional "Never fired" badge with state-aware variants: gray "Disabled" when rule.enabled is false; green "Active" when fire_count > 0 and last_fired within 7 days; yellow "Idle (no recent activity)" when fire_count > 0 but last_fired > 7d ago; gray "No activity yet" when never fired (without implying breakage). Same badge Tailwind shape as before; last_fired surfaces via the title tooltip.
Backend untouched. npm run build (tsc strict) clean. PYTHONPATH=. pytest -q: 328 passed (unchanged from v0.5.5). Safe-mode preserved (master off, all toggles off, all adapters native, central disabled).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
