* feat(region-routing): P1 tagging + region_routes primitive + read/write API + preview launcher
- config.py: add Coverage.region_tagging (bool=False); add RegionRouteMatrix
dataclass (enabled, cells) above NotificationsConfig; add region_routes field
to NotificationsConfig; add explicit hydration branch for region_routes in
_dict_to_dataclass mirroring destinations pattern.
- coverage_area.py: add MonitoringArea.name (str|None=None, frozen); update
areas_from_config to preserve name; refactor inline geom extraction from
classify_event_areas into shared _event_geom_json helper; add
matching_area_names(geom_json, areas)->list[str] (additive, all named
matches, config-order, deduped; gate unchanged); add event_region_names
convenience wrapper.
- coverage_filter.py: add region_tagging ctor kwarg; stamp event.region/
regions before the gate when region_tagging=True and areas non-empty and
not event.regions (never clobbers satpass preset).
- pipeline/__init__.py: wire region_tagging into CoverageFilter construction.
- notification_routes.py: add GET /notifications/regions (named coverage area
names, config-order, deduped); GET /notifications/region-routing (matrix as
JSON); POST /notifications/region-routing (explicit RMW — only region_routes
changes, toggles/rules/destinations survive).
- scripts/preview_dashboard.py: mesh-free launcher — dashboard API only, no
mesh connector, no broadcast loop; vite runs separately.
All 87 coverage tests pass; 300 total pass; 6 pre-existing failures unchanged
(adapter config count mismatch + MeshCore EventType.NEW_CONTACT).
* feat(region-routing): manual region x family matrix editor page
Adds RegionRoutingMatrix.tsx — a plain editor over the region_routes
config primitive. Rows = families (via useFamilies()), cols = regions
(from GET /api/notifications/regions). Each cell exposes MT channel
(ChannelPicker single + includeDisabled), MC channel name (text input),
min_severity select (routine/priority/critical/immediate), and an enabled
checkbox. Only cells where MT or MC is set are included in the sparse
POST payload. Master enable toggle maps to top-level enabled. MT budget
guard warns when more than 7 distinct MT indices are in use. Sticky
family column; horizontal scroll for wide region sets.
Registers route /region-routing in App.tsx and adds "Region Routing"
nav entry (Map icon) under the Meshtastic section in Layout.tsx,
immediately after Routing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(region-routing): regions endpoint reads saved (disk) coverage so routing columns are dynamic without a bot restart; preview reloads config after writes
* feat(routing): unify MT/MC routing into per-family cards; region routing as an in-card expand; remove rules/destinations UI + standalone page
* refactor(routing): move Meshtastic Routing from /notifications to /meshtastic/routing (mirror /meshcore/routing); redirect legacy path
* feat(region-routing): dispatcher honors region_routes matrix (authoritative-on-match, per-region cooldown, per-channel dedup); non-matrix path unchanged
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(region-routing): matrix dedup key must match boot-restore 2-tuple form (prevents restart re-broadcast flood); regression test
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The WFIGS adapter computed changed = (old event_id set != new event_id set),
so growth of an already-known fire produced changed=False. The store only runs
_ingest_fires (and the Phase-3 fire decider) when tick() reports a change, so
growth/update broadcasts for stable fires never fired — only brand-new or
dropped fire NAMES woke the path. Include acres + containment in the change
signature. The decider stays the broadcast gate (forward-only + cooldown), so
no backlog is dumped.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MeshAI/1.0 UA intermittently trips WAFs (Idaho Power's Azure Front Door
403s it ~2/30; a browser UA gets 200 every time). Default the adapter +
preview to a browser User-Agent, retry once on 403/429, and add optional
per-source custom headers (UA/auth) editable in the GUI. Makes WAF'd and
keyed feeds pollable.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a Destinations manager to the Routing page — define each delivery target
(mesh channel / email / webhook / digest) once — and a destination picker on
each family and rule. The duplicated per-family inline email/webhook editors
move under an Advanced/legacy disclosure. De-fragments delivery config: no
more configuring the same email in two places.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add NotificationDestination + config.notifications.destinations and a
`destinations` reference list on toggles/rules. When a toggle/rule references
destinations, delivery resolves from the shared destination; when empty, the
existing inline-field delivery path runs UNCHANGED (zero regression). Lets
email/webhook/mesh-channel be defined once and reused, de-duplicating the
delivery config. UI to follow (C2).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the generic-source editor into a reusable GenericSourcesEditor and render
it as a "Custom Sources" section on the Data Feeds (Environment) page, so a
custom source sits with the built-in feeds instead of an orphan page. Remove
the standalone /data-sources route + nav. Custom sources are now managed as
first-class data feeds; enable their family in Notifications to route them.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add GET /api/notifications/families and make the Routing UI merge the static
built-in families with registered dynamic families, so a generic source's
family appears as a toggle the operator can enable and assign a delivery
destination (mesh/meshcore/email/webhook). Closes "there's nowhere to
broadcast it" — a custom source is now routable from the GUI. Data Feeds fold
+ orphan-page retirement follow in B2.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Categories/families can now be registered at runtime, not just the hardcoded
ALERT_CATEGORIES/VALID_TOGGLES. A generic data source registers its category
as a first-class family with its own (default-disabled) toggle, so its events
resolve to that family instead of being dropped as "other" or buried in
mesh_health — it becomes routable. Existing families/categories unchanged.
Phase A of making custom sources first-class feeds.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Whitelist generic_sources as a config section and add a Data Sources page:
add/edit/delete sources with the full field-mapping UI (items/id/lat/lon/
geometry/title paths + field_mappings list + summary template) and a
server-side URL Preview that shows the endpoint's JSON so operators can map
fields without knowing the structure ahead of time. Makes the generic
adapter truly no-code — point it at any public REST/GeoJSON feed from the GUI.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Universal, no-code data sources: one GenericHttpAdapter polls any public
REST/GeoJSON feed per config.generic_sources[] — dotted-path field mapping
(items/id/lat/lon/geometry/title/fields) → coverage-gated, persisted
(generic_events, v26), cold-start-silent, LLM-queryable events. Ports
Central's GenericHttpAdapter to meshai native. First real use case: Idaho
Power outages, configured (not hardcoded) — anyone can point it at their own
utility/feed. GUI editor is a follow-up.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stop waiting 25s for a PATH_RESPONSE that never arrives. Send the reply
directly, wait ~6s for the delivery ACK the lib exposes; on ACK we're done
(~1-3s, the common case). Only on no-ACK do we run path discovery + resend —
and discovery's wait drops from 25s to a config default (8s). Both timeouts
are config knobs (meshcore_ack_wait_seconds / meshcore_discovery_wait_seconds)
for live tuning. Fixes the real bug behind PR #57 (checked is_error, should
have checked ACK).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrite the satellite-pass wire to a single clean line: short ham names
(ISS/AO-27/AO-91), numeric max elevation (max 77°) instead of a bucket word,
collapsed compass sweeps (no E→E→E), and friendly observer names — dropping
the meaningless synthetic coverage_center parenthetical (and no longer seeding
that observer when explicit observers are configured). Absolute local time
kept for the 12h-advance heads-up.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Activity Log endpoint wasn't reading mesh_broadcasts_out, so it only
surfaced a partial set (MT band-propagation + satpass) and missed the
event-driven weather broadcasts and the entire MeshCore side. Query
mesh_broadcasts_out for all broadcasts across both transports and all
categories, newest-first with pagination, so the feed reflects everything
that actually went to the mesh.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Close the LLM data gaps: add build_satpass_detail (satpass_events was written
but had no reader), and give avalanche + ducting durable tables (v24/v25) with
native writers + env_reporter readers so the mesh LLM can answer avalanche,
satellite-pass, and RF-propagation questions. Persistence-only; no broadcast/
gating changes.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(firms): curated new-fire cluster broadcasts (no per-pixel, no cold-start dump)
Enable the built _maybe_emit_cluster path (was dead-coded) so FIRMS broadcasts
curated hotspot clusters as possible new fires — clustered, deduped via
cluster_broadcast_at, attributed against known WFIGS fires first (so MORA's
hotspots don't false-cluster). Give FIRMS a default Idaho bbox so it fetches
when coverage is off (coverage bbox still overrides). First-fetch silent-seed
prevents a cold-start dump of the day's existing hotspots. Raw pixels stay
store-only. Coverage geometry gate filters cluster broadcasts to the region.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(firms): first-fetch silent-seed suppresses fusion wires too (no cold-start)
Extend the FIRMS cold-start seed to suppress growth/spotting/halt fusion
broadcasts on the first fetch, not just clusters — enabling FIRMS must emit
zero broadcasts on the initial hotspot sweep. Persistence, attribution, and
dedup baselines still run during seed; only later new activity broadcasts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(fire): route native WFIGS through the Phase-3 growth decider (fix updates)
Completes the Phase-3 fire migration for the native adapter. env/fires.py now
emits canonical data (_kind/irwin_id/declared_at/acres/contained), native
fires bypass the received-delta gate and run the shared gating.fire.decide +
fire formatter (forward-only growth + containment + 8h cooldown + deferred
commit), and a native-only cold-start pre-pass silent-seeds old/known fires so
no backlog spam. Fixes growth/containment silence (MORA) and revives the
fires-table-backed reminders/digest. Reuses the existing decider — no dup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(fire): cold start seeds ALL current fires silently (no 48h dump)
Drop the fresh-ignition age window from the native cold-start seed — a fresh
deploy with an empty fires table must not broadcast fires discovered in the
last 48h. Now every fire present at boot is seeded silently; a fire only
broadcasts New if it appears on a later poll (a genuine ignition since
startup). Growth/containment updates unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The multi-box gate is authoritative, but adapters still need to FETCH the
right data — otherwise a box crossing a state line never pulls the cross-
state side. Feed each adapter's fetch scope (fires envelope, nws area=states,
hydro bBox, etc.) from the enclosing bbox of config.coverage.areas (falling
back to legacy coverage.bbox). The Shapely gate still narrows to the exact
areas; the enclosing box just ensures cross-state / multi-area data is pulled.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Upgrades the Coverage page from one bbox to a list of named areas, matching
the backend coverage.areas / Shapely set-union gate. Draw multiple boxes,
name/edit/delete each, all rendered on the map; saves config.coverage.areas
(clears legacy bbox). Coords rounded to 6dp. Enabled + per-adapter override
toggles unchanged.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The real LA leak was a zone-only advisory with no polygon and no centroid —
the fail-open gate kept it. Now NWS attaches the full GeoJSON alert geometry
(Polygon/MultiPolygon) to the event, and the coverage gate drops weather
alerts it cannot locate (fail-closed, matching Central), while staying
fail-open for other categories. Removes the old buggy adapter-level
_in_coverage heuristic (the gate supersedes it).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces the hand-rolled per-adapter region heuristics (which leaked LA/OR
broadcasts) with Central's proven mechanism: bounding-box(es) + Shapely
full-geometry intersection. Ports MonitoringArea/build_geom_json/
classify_geom_areas from the central repo; adds config.coverage.areas
(multi-box, set-union) and a CoverageFilter that gates every event on
geometry-in-any-area before broadcast. Adapter geometry enrichment (NWS
polygons, fail-closed) follows in the next phase.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
USGS rejects bBox coords with >7 decimals (raw Leaflet clicks have 14) —
round all coverage-derived coordinates to 6dp so USGS/others accept them.
TomTom flow 400 ("Point too far from nearest existing segment") on rural
grid cells is expected no-data, not an error — log debug and skip instead
of warning. Fix the fires log to not claim "in US-ID" under coverage mode.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a GENERAL > Coverage page: a Leaflet map to draw/set the universal
coverage bbox (with numeric W/S/E/N inputs), an enabled toggle, and per-
adapter override toggles (excluded_adapters). Environment's per-adapter
geographic-scope fields now show only when that adapter is overridden;
otherwise they point to the Coverage map. Saves via PUT /api/config/coverage.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds coverage.excluded_adapters: adapters listed here ignore the universal
coverage bbox and use their own config (the "advanced override", native
only). _coverage_for() returns None for excluded adapters; satpass seeding
respects it too. Backend for the GUI override toggles.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 2d: hydro discovers gauges via the USGS IV bBox query across the
coverage box (thresholds stay NWPS-cached; unrated sites never emit) instead
of a hand-listed site set; satpass seeds a single centroid observer derived
from the box. Completes wiring all native env adapters to the universal
coverage bbox. Coverage governs when set; each adapter's config is fallback.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 2c: nws derives its area= state filter from the bbox (states it
overlaps) AND drops alerts whose centroid falls outside the box for precise
scoping; traffic samples a 3x3 grid of flow points across the bbox instead
of hand-listed corridors. Coverage governs when set; each adapter's own
config is the fallback.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 2b: fires now scopes via the ArcGIS envelope from the coverage bbox
(dropping the single-state WHERE; event_id uses the fire's own POOState so
multi-state boxes work and Idaho dedup keys are unchanged); firms uses the
coverage bbox for its NASA query; ducting observes at the bbox centroid.
Coverage governs when set; each adapter's own config is the fallback.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 2a: plumb config.coverage.bbox through EnvironmentalStore to adapter
constructors (mirroring region_anchors) and wire the four straightforward
native adapters to derive their scope from it — usgs_quake/roads511 bbox,
wzdx states+bbox, avalanche center_ids — via resolve_adapter_coverage().
Coverage bbox (when set) governs; the adapter's own config field is the
fallback when unset. Central-fed adapters are unaffected (never native-
constructed). fires/firms/hydro/traffic/satpass/ducting/nws follow.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 1 of replacing per-adapter geographic scoping with one coverage
bbox. Adds a `coverage.bbox` [W,S,E,N] config and meshai/coverage.py — a
pure, tested derivation layer: geometry primitives (point_in_bbox,
intersects, centroid, grid_points, arcgis_envelope) + static US-state and
avalanche-center bbox tables + resolve_adapter_coverage() mapping one bbox
to each native adapter's effective scope. Central-fed adapters return None
(Central governs). No adapter wiring or GUI yet — foundation only.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_establish_direct_path ran a CMD-52 probe (25s timeout) on every DM reply,
even when the contact already had a valid out_path (out_path_len >= 0),
adding ~25s latency to every reply. Only probe for flood/unknown-path
contacts; for already-routed contacts send direct immediately. If a cached
route turns out stale (send fails), fall back to discovery + one retry so a
moved/broken path still self-heals. Cuts routed-contact reply latency from
~25s to ~2s.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_resolve_contact relied on ensure_contacts(), which is a no-op once the
lib's contact mirror is populated — so firmware-side auto-added senders
never synced in and every reply dropped with "could not resolve contact".
On a prefix-lookup miss, force get_contacts(lastmod=0) to pull the full
roster from the radio, then retry the lookup. Lets AIDA reply to anyone
the firmware has (auto-added or previously heard) even when the cached
mirror is stale.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enable firmware auto-add (set_autoadd_config CMD 58) at connect when
connection.meshcore_auto_add_contacts is set (default on), and refresh
the contact roster on NEW_CONTACT so replies resolve immediately. GUI
toggle on the MeshCore Connection page. So the USB AIDA companion adds
every node it hears an advert from and can send/decrypt DMs without
manual contact exchange.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The banner said 'Container restart required' and a code comment claimed 'the
container will tear down' — both wrong. /api/restart touches /tmp/meshai_restart,
which the entrypoint watches to restart the BOT PROCESS in place (~seconds); the
container is not recreated. Fix the copy + point users at the existing 'Restart
now' button. Also mention transport connections in the restart-required list.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pyserial comports() reads /sys USB metadata, absent inside a container for a
bind-mounted device node — so /api/serial-ports returned [] and the GUI "Detect"
showed nothing despite /dev/meshcore-rak (major 166) being present + openable.
Supplement comports() with a direct /dev scan: include char devices whose major
is a USB-serial major (166 ttyACM, 188 ttyUSB), catching /dev/ttyACM*/ttyUSB*
AND custom udev names like /dev/meshcore-rak that a tty* glob misses; exclude
legacy ttyS* (major 4). Merge deduped by realpath (pyserial metadata wins on
overlap). stable_path: by-id > stable custom name > by-path > raw. likely_radio
heuristic on the name (mesh|rak|lora|tbeam|heltec|nrf|companion) for bare nodes.
Resilient (unreadable /dev / stat error skipped, never raises).
8 new tests; suite at 10-failure baseline.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MeshCore can now connect over USB serial (and BLE) directly, not just TCP to
the pyMC companion. The meshcore lib already supported create_serial/create_ble;
we just wire it up. Plus a USB auto-detect scanner that resolves stable device
paths to fix ttyACM enumeration hopping across replug/reboot.
Backend:
- ConnectionConfig: meshcore_conn_type (tcp|serial|ble, default tcp),
meshcore_serial_port, meshcore_baud=115200, meshcore_ble_address (validated)
- meshcore_transport._do_connect dispatches per mode: serial ->
MeshCore.create_serial(port, baudrate, auto_reconnect, max_reconnect_attempts),
ble -> create_ble(address or None), tcp -> create_tcp (unchanged). Mode-aware
logging/reconnect. Transport otherwise unchanged (mode-agnostic once _mc exists).
- factory.meshcore_enabled(config): active when the selected mode is configured
(serial port / ble address / tcp host); back-compat — meshcore_host + default
tcp still activates exactly as before.
- serial_ports.list_serial_ports(): pyserial comports + stable_path resolution
by-id -> by-path -> raw (by-id keyed on USB serial = stable across replug),
likely_radio flag by VID (RAK/nRF/CP210x/CH340), excludes legacy ttyS*, never
raises. GET /api/serial-ports (+ container by-id passthrough hint).
Frontend:
- SerialPortPicker component: "Detect USB devices" -> lists ports (likely-radio
badge, shows stable_path) -> onChange sets the stable by-id path; manual text
fallback; empty/error/note states.
- MeshCore Connection: type selector TCP/Serial/BLE + per-mode fields (serial
picker + baud; ble address). Meshtastic serial branch now uses the picker too.
Code-ready; not activated (defaults keep TCP). 35 new tests; suite at 10-failure
baseline. Container needs /dev/serial passed through for by-id paths.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
roads511 emitted external_id=None, so it couldn't be durably pre-seeded from
the persistent tables (only wzdx/usgs_quake were) — it relied solely on the
in-memory first-poll seed. Thread a stable external_id="511_{itd_id}" through
consistently so it joins the durable layer:
- env/roads511.py: _parse_event raw event + to_event both carry
external_id="511_{id}" (== event_id). Flips _seen_key to the ext: branch and
makes the incident decider persist traffic_events(source='511', external_id)
— which ALSO restores the decider's own dedup (external_id=None was the
original roads511 leak cause).
- env/store.py _seed_from_persistent: add a "511" spec (seed from
traffic_events where source='511', by external_id) mirroring wzdx; shared
_key_ext helper so keys can't drift.
- consistency proven byte-identical (raw _seen_key == pre-seed key ==
511\x1eext:511_{id}); durable-preseed + regression tests added.
Live DB: 0 source='511' rows yet (flip recent) -> durability engages as native
rows accumulate; layer-2 in-memory seed covers the interim (atomic fetch).
Central-era itd_511 rows use a different keyspace, intentionally not covered.
Suite at 10-failure baseline (1716 passed).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The banner used a static META.hasKey flag (false for firms+roads511), so
those always showed 'API key not configured' even when keys were set —
contradicting the ManagedSecret SET badge. Now fetch /api/secrets and show
the banner only when a keyed adapter's secret is genuinely unset; copy
softened to 'API key required — set it in the field below'. wzdx kept keyless
(no banner). Keyless adapters never banner.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 100%-coverage pass added timezone to FullConfig, but it's a top-level
scalar (own PUT /api/config/timezone), not a section tab — so SectionKey =
keyof FullConfig broke SECTION_DESCRIPTIONS: Record<SectionKey,string>.
Exclude 'timezone' from SectionKey. Docker frontend build failed on this;
live container was untouched.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add bound GUI controls for every remaining user-facing + internal config
setting so the dashboard is the complete config surface (secrets stay in
.env via the ManagedSecret widgets). Per the exhaustive per-key audit.
19 GAP controls (real settings with no prior control):
- Config: global timezone (+ backend: timezone in VALID_SECTIONS + scalar
save_section branch), commands.custom_commands (kv editor), knowledge
sparse_host/port, alert_rules.high_util_hours
- Data Feeds: nws.areas, usgs.flood_thresholds (json), usgs_quake
feed_url/min_magnitude(native floor)/bbox, wzdx.registry_ttl,
satpass.broadcast_lead_seconds, central.connect_timeout,
toggles.<family>.regions (un-hidden + save-merge fix)
- Notifications: band_conditions_tz, digest.schedule, digest.include
- Danger Zones: snow.enabled + buffer_mi (unlocked)
18 INTERNAL knobs (Advanced subsections):
- connection reconnect/timing (7) split across MT/MeshCore Connection
- environmental.geocoder url/timeout/radius/limit (4)
- identity.contact_email (+ LOCAL_FIELDS mirror -> local.yaml; still feeds
NWS User-Agent); mesh_sources url + regions lat/lon already editable
- toggles.<family>.name/webhook_headers, danger_zones.webhook_headers
(new reusable KeyValueInput component)
alert_node_ids intentionally not surfaced (synthetic local default with no
dataclass home; overlaps the editable per-rule node_ids). No dead controls.
Frontend validated at Docker build. Backend py_compile OK.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
API keys/secrets now live in /data/secrets/.env (gitignored, never in config
YAML), while remaining fully editable from the dashboard. Config YAML holds
only ${VAR} references.
Backend:
- meshai/secrets_store.py: get_status (SET/NOT-SET, never values), set_secret,
delete_secret over /data/secrets/.env (resolved like load_config); authoritative
SECRET_FIELD_TO_ENV map (traffic→TOMTOM_API_KEY, firms→FIRMS_MAP_KEY,
roads511→ROADS511_API_KEY, wzdx→WZDX_API_KEY, smtp→SMTP_PASSWORD,
mesh_sources→MESHMONITOR_API_TOKEN) + backend-dependent llm_env_var
- dashboard/api/secrets_routes.py: GET /api/secrets (status only), PUT/DELETE
/api/secrets/{env_var} (validated, restart_required); registered in server.py
- config_loader: save_section preserves ${VAR} secret refs on section save
(never rejects them); EXPECTED_SECRETS += ROADS511_API_KEY, WZDX_API_KEY
- config.example.yaml + docker-entrypoint default config use ${VAR} refs;
first-run bootstraps /data/secrets/.env; .gitignore covers it
Frontend:
- components/ManagedSecret.tsx: masked, Set/Not-set badge, reveal, Save->PUT,
"restart required"; carries no config value so secrets never enter a section
save payload
- wired into Environment (tomtom/roads511/wzdx/firms), Config LLM tab
(env var by backend), Notifications (smtp)
Restart required after a secret change (env read at config-load). 11 store
tests; suite at 10-failure baseline (1714 passed).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backend half of making the dashboard the complete config surface (per an
exhaustive per-key audit).
Delete 51 vestigial/unread config fields (load-safe: _dict_to_dataclass
whitelists by field, so existing files carrying these keys still load and the
keys drop on next save):
- 36 duplicated MQTT block (host/port/username/password/topic_root/use_tls)
on memory/context/commands + env nws/swpc/ducting (grep-proven unread;
mesh_sources keeps its real MQTT fields)
- 3 no-op history cleanup keys (auto_cleanup/cleanup_interval_hours/max_age_days)
- 5 alert scaffolding (alert_cooldown_minutes, RegionAnchor.nws_zones,
battery_{warning,critical,emergency}_voltage)
- 5 danger-zone non-fire min_acres (kept fire.min_acres via a fire subclass)
- 2 deprecated adapter_config keys (nws.broadcast_severities/warning_suffix_promotes)
Bug: usgs_quake native magnitude floor was unreachable from the GUI (native
reads config.min_magnitude; the GUI "Global Floor" wrote the registry
global_mag_floor that only the Central path reads). Reconciled: min_magnitude
is the canonical native floor the frontend will bind; registry floors marked
Central-path-only. Effective filtering unchanged.
Secret-flag consistency: add environmental.roads511.api_key + wzdx.api_key to
SECRET_FIELDS (secrets move to .env in the follow-up; ${VAR} interpolation kept).
Suite at 10-failure baseline (1703 passed).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Audit confirmed the Rich TUI (`meshai --config`) is a strict subset of the
dashboard GUI (covers FEWER sections) and nothing depends on it for bootstrap.
Strip it so the dashboard is the single config surface.
- delete meshai/cli/configurator.py (~1435 lines); cli/__init__ minimal
- main.py: drop --config + run_configurator dispatch (keep --config-file);
bootstrap log now points to config.example.yaml / the dashboard
- docker-entrypoint.sh: remove the ttyd `meshai --config` block on :7682
(default-config heredoc + bot exec loop untouched)
- docker-compose.yml + Dockerfile: drop the 7682 port + ttyd install (EXPOSE 8080)
- README / config.py header / config.example.yaml: de-reference the TUI
- drop `rich` from pyproject + requirements (grep-verified unused)
Close the only GUI config gap the audit found:
- Config.tsx weather tab: add openmeteo.url + wttr.url inputs (backend already
round-trips the nested WeatherConfig dataclasses)
Bootstrap intact: config.example.yaml copy, entrypoint default-config write,
and defaulted Config() when no file exists all remain. `python3 -m meshai`
and `--config-file` unchanged. TS/import validated at Docker build.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two live backlog-broadcast leaks traced to the in-memory first-poll seed:
(1) incremental-fetch adapters (wzdx: registry tick [0 events] then feeds
tick [many]) got marked _seeded on the EMPTY first tick, so the real batch
next tick all looked "new" and broadcast; (2) in-memory seed lost on restart.
Fix — durable baseline + guard:
- _seed_from_persistent() at store init: pre-load already-received item keys
from the persistent hazard tables into self._seen, so nothing ever received
can re-broadcast (immune to fetch staging + restart). Only sources whose
native emit key PROVABLY equals a persistent key are durably seeded:
wzdx (traffic_events.external_id) + usgs_quake (quake_events.event_id).
Resilient (per-table try/except; missing table -> skip).
- _seen_key() now namespaces by evt["source"] (matches persistent tables),
via shared _key_ext/_key_eid helpers used by both seed and live emit so
they can't drift.
- non-empty-seed guard: _ingest marks only sources that carried >=1 event
this poll as _seeded -> an empty first tick can never seed-then-leak. This
is the root-cause fix; covers all adapters (roads511/traffic fetch
atomically per tick, so the guard fully protects them).
- storage untouched (self._events populated for every event); Central
path/deciders untouched.
Live-DB verified: seed pre-loads 784 wzdx + 8 quake keys -> a live wzdx poll
of 784 known zones broadcasts 0. +6 tests (incremental staging, restart,
persistent-preseed, fresh-DB fallback); suite at 10-failure baseline.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the native path's "scan accumulated state + suppress what we've
already broadcast" model with "broadcast only what newly arrived from the
API this poll." Storage is unchanged (self._events + firms_pixels etc. are
populated for EVERY received item, so the LLM/get_active backlog is intact);
only the BROADCAST decision changes.
- env/store.py: per-adapter in-memory seen-set (_seen) + _seeded. First
data-bearing poll for an adapter seeds keys and emits NOTHING (that batch
is pre-existing backlog); later polls emit only keys not seen before.
Restart => empty sets => next poll re-seeds silently. Structurally
impossible to broadcast backlog on cold start / restart / re-enable.
Key = external_id -> event_id -> content hash, namespaced per adapter.
self._events[key]=evt still runs unconditionally (storage preserved).
- Fixes the ~175 (roads511) / ~782 (wzdx) cold-start bursts AND the latent
quake/nws version (they only looked safe because Central pre-populated
their broadcast tables).
- env/satpass.py: broadcast on AOS IMMINENCE (now < aos <= now+lead,
broadcast_lead_seconds default 3600), future-only; window_hours still
governs prediction depth. Strict norad_ids post-filter + fixed
_parse_norad_ids char-iteration bug (cause of GOES/METEOR leak).
- Central path + broadcast-state tables untouched (native-only gate).
13 new tests; full suite at 10-failure baseline (1697 passed).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two bugs found running native adapters against real upstreams:
- env/wzdx.py: _select_feeds() called .strip() on the registry url field,
but Socrata "URL"-column values arrive as {"url": "..."} — crashed ALL
native WZDx discovery with AttributeError. Added _unwrap_url() (dict/str/
None-robust), applied to url/apiurl/feed_url.
- composer._context_segment: appended optional fields by key presence, so
cause: None (set by native road adapters) leaked literal "None" onto the
wire. Guard on value (cause/expires_at truthiness; containment_pct is-not-
None so 0% still renders). Legacy Mode-B path — golden tests unchanged.
+3 regression tests; golden/composer suites pass; full suite 10-failure
baseline (1685 passed).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Close the GUI gaps so a standalone (all-native) deployment needs no
hand-editing of /data/config.yaml:
- Environment.tsx: native satpass (SGP4) section — observers editor
({slug,name,lat,lon,alt_m} list, mirrors traffic corridors), tle_groups,
norad_ids, min_elevation_deg, window_hours, tle_refresh_seconds (all
environmental.satpass YAML). Central adapter_config satpass panel unchanged.
- Environment.tsx: wzdx states + registry_url added (were file-only).
- BUG FIX: the satpass enable toggle wrote adapter_config.satpass.enabled
(Central), but env/store.py gates the native adapter on
environmental.satpass.enabled — toggle now writes the native/YAML layer.
Backend round-trips automatically (SatpassConfig/WZDxConfig already declare
all fields; _dict_to_dataclass whitelists by field). Frontend-only change.
NOTE: TS build (tsc && vite build) not runnable offline (node_modules absent);
validated at Docker build / deploy time.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes the last standalone gap. Extract ingest_hotspot_pixel(pixel, *, now)
from firms_handler so the FIRMS attribution/fusion engine (firms_pixels ->
_attribute_or_cluster -> fire_pixels/fire_passes/centroid -> growth/spotting/
halt) is source-agnostic. Both the Central NATS path and native env/firms.py
drive one identical engine.
- shared _ingest_pixel_core(conn, ...) called by both ingest_hotspot_pixel
and handle_firms; Central path byte-identical (its tests pass unchanged)
- env/firms.py _fetch() feeds each fetched pixel into ingest_hotspot_pixel;
DB-level dedup makes re-fetched pixels no-ops (no double count); to_event()
returns None for raw hotspots, precomposed Event for fusion outputs
(wildfire_growth/spotting/halted via the Phase-3c formatters/gating)
- raw hotspots / new_ignition / cluster NEVER broadcast (cluster stays dead)
FLIP NOTE: keep wildfire_growth/spotting/halted OUT of cutover — native
emits precomposed with gating done inside the engine; cutover would re-run
the _kind-keyed decider on data lacking _kind and suppress.
10 new tests; Central firms/fire-tracker suites unchanged; full suite
10 failed/1682 passed (baseline 10, +10 new).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
meshai can now predict + broadcast satellite passes locally without Central.
Data plumbing (4c-1):
- env/tle_fetch.py: keyless Celestrak GP fetcher (GROUP/CATNR, FORMAT=tle) →
upserts the existing sat_tles table via a shared upsert_tle() helper
extracted into tle_handler (Central ingest refactored to call it, unchanged)
- observer_locations table (v23, SCHEMA_VERSION 22->23) + persistence helpers;
seeded from SatpassConfig.observers in main._init_components
- SatpassConfig: observers, tle_groups, norad_ids, tle_refresh_seconds,
min_elevation_deg, window_hours
Predictor + source-agnostic gate (4c-2):
- extracted gate_consolidated_pass(consolidated, *, now) from
consolidate_satpass_pending: dedup-vs-satpass_events + rate cap + format_pass
+ deferred commit. Central path byte-identical (114 tests unchanged)
- env/satpass.py: native adapter predicts passes for each sat x observer via
pass_predictor.compute_passes, consolidates IN-MEMORY per canonical hour
bucket (earliest AOS / latest LOS / max-el observer supplies peak_compass +
entry/exit observers), runs the shared gate, emits sat_pass. Commit rides
event.data so satpass_events dedups across ticks — NO satpass_pending, NO
Central-consumer timer dependency (works with Central off)
- registered in env/store.py gated on enabled and feed_source==native
32 new tests (tle_fetch 16, observer_locations 14... satpass_native 8, minus
overlaps); full suite 10-failure baseline (1672 passed).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Satpass consolidations were scheduled with in-memory asyncio timers
(_pending_satpass_timers) lost on restart, orphaning satpass_pending rows
that never consolidated/broadcast. Persist a durable due_at and rebuild
timers on startup.
- v22.sql: satpass_pending.due_at INTEGER; SCHEMA_VERSION 21->22
- due_at = received_at + CONSOLIDATION_DELAY(5); the live +N*60 stagger is
in-memory only (meaningless across restart) so not persisted — live
call_later path unchanged, due_at is a pure reboot backstop
- consumer._sweep_pending_satpass() at start(): past-due rows fire (orphans
recovered), future rows re-armed for the remaining wait; skips cids the
live drain path already owns (no double-schedule); per-row try/except
- reuses _satpass_consolidation_fire so emit logic is identical
Non-reboot behavior byte-identical. 6 new tests; suite at 10-failure baseline.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New env/wzdx.py native adapter so meshai has a LOCAL source for the
work_zone category instead of Central-only. Keyless: discovers per-state
WZDx feeds from the FHWA registry (datahub.transportation.gov), filtered to
configured states (default [ID]), fetches each WZDx v4 GeoJSON feed, and
parses road_events into canonical work_zone Event.data.
- reuses central_normalizer._parse_wzdx_federal for identical field mapping
→ renders through the existing Phase-2 formatters/gating incident path
(no new formatter/gating); verified end-to-end to a valid work-zone wire
- stable external_id = {data_source_id}:{feature_id} for incident dedup
- registry cached 6h; per-feed failures logged + skipped (never crash)
- WZDxConfig extended (registry_url, registry_ttl, states); registered in
env/store.py; enabled=false + feed_source=native default (dormant until
turned on at the coordinated flip)
19 new tests; suite at the 10-failure baseline (1640 passed). Only env/wzdx.py
(new), env/store.py, config.py, tests/test_adapter_wzdx.py touched.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
