* fix(dashboard): register gauge_sites_import router
The gauge-sites bulk-import endpoint (CSV / NWS-AHPS) was fully built and
tested but never wired into the app -- server.py never called
include_router() for it, so POST /api/gauge-sites/import 404'd in
production. Only test_tail_followups.py exercised it, via a raw
TestClient bypassing the real app.
* test(dashboard): assert gauge_sites import route is reachable via real app
Guards against the router silently going unregistered again: builds the
actual dashboard app via create_app() and drives a real CSV import
through it end-to-end (POST + GET round-trip), instead of only hitting
the router in isolation.
* feat(dashboard): add gauge-sites bulk import UI
Wires a UI onto the now-registered POST /api/gauge-sites/import endpoint,
inside the existing GaugeSites tab (no new nav entries/pages). Adds an
Import toggle next to Add site, with CSV (paste or file-load into a
textarea) and NWS-AHPS (WFO code list) modes. Required/optional CSV
columns are documented inline so an operator isn't guessing. Result
counts (inserted/updated/skipped/detail_fetched) and any partial-failure
errors from the AHPS scrape are always shown; a pending spinner covers
the AHPS path's live water.weather.gov calls so a slow response never
reads as "imported 0 sites". The list refreshes only after a successful
import that actually changed rows.
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
* fix(hydro): make native USGS gauge flood alerts renderable
env/usgs.py emits stream_flood_warning / stream_high_water Events for
elevated stream gauges, but neither category had a registered gating
decider or formatter, and event.data was left empty -- so every
detected flood/high-water reading was silently dropped before it ever
reached the mesh (get_decider/get_formatter both returned None, and
compose_mesh_message fell through with nothing to render).
- Register the existing hydro.decide()/hydro.format() (already used by
the Central-only `stream_flow` category) under stream_flood_warning
and stream_high_water too -- same shared-decider/formatter pattern
already used for avalanche_warning/_watch, weather_*, wildfire_*, and
emergency_*.
- Add both categories to cutover.NATIVE_ALWAYS_DECIDE so the decider and
formatter actually run unconditionally (mirrors the native WFIGS fire
categories): Central never emits these two category strings, so there
is no shadow-bake window to wait out.
- env/usgs.py's to_event() now populates event.data with the canonical
hydro schema (site_id, gauge_name, stage_ft, flow_cfs, unit,
threshold_state, reading_time, lat, lon, parameter_code) the shared
gate/formatter expect, mapping the adapter's flood_status strings onto
the ranked threshold_state vocabulary.
- gating/hydro.py's decide() now also OWNS an unconditional
gauge_readings INSERT for the native source (source != "nwis"): the
table had no writer since Central's nwis_handler stopped running
(2026-07-05), so every native prior-state lookup returned "normal"
forever and every elevated reading would have rebroadcast on every
15-minute tick instead of once per crossing. The Central source keeps
its own inline handler-owned INSERT unchanged (decide() stays
read-only for source="nwis").
Left as-is (not this fix): the toggle mapping for these categories
(get_toggle() -> "seismic") is unchanged. It already matches the
sibling `stream_flow` category and is enforced by
test_water_v057.py::test_existing_hydro_entries_unchanged /
test_water_categories_have_required_fields; there is no separate
water/flood toggle family in VALID_TOGGLES, and inventing one is a
config-surface change outside this fix's scope.
Known limitation documented in gating/hydro.py: since to_event() only
ever emits elevated readings (never a "back to normal" reading), a full
recede-to-normal followed by a later re-crossing into the same tier
will not re-broadcast until a higher tier is reached -- degrades toward
silence, not spam.
* test(hydro): cover native gauge flood-alert registration, gating, and wire format
Registration tests prove stream_flood_warning/stream_high_water now
resolve a decider AND a formatter (the thing that was broken), that
neither resolves to the earthquake decider despite sharing the
"seismic" toggle name, and that both are in NATIVE_ALWAYS_DECIDE.
Gate tests drive env/usgs.py's to_event() through the real
gating.hydro.decide(): a first elevated reading broadcasts (graceful
no-prior-data handling), a sustained same-band reading suppresses, an
escalation broadcasts again, and a routine reading never reaches the
gate at all (unchanged pre-fix adapter behavior). A dedicated test
confirms the new native-persistence write in decide() does not leak
into the Central source="nwis" path.
Golden formatter tests pin the wire string for a high-water and two
flood-warning tiers, plus the missing-coords drop case -- all rendered
through the same formatters.hydro.format() the Central `stream_flow`
path uses (test_hydro_refactor.py already proves that formatter is
byte-identical to the old central.nwis_handler._render()). Native
events never carry flow_cfs (to_event() only ever emits stage/height
readings), so that segment's absence is captured explicitly as current
behavior, not ported from Central.
An end-to-end test drives to_event() -> decider -> compose_mesh_message
to prove the NATIVE_ALWAYS_DECIDE gate takes effect for the actual mesh
render path, not just formatter/decider resolution in isolation.
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
* refactor(mesh_reporter): expose recommendations as list[str]
Add recommendations_list(scope, scope_value) as the canonical source of
recommendation text. build_recommendations() now just joins that list with
the historical "OPTIMIZATION RECOMMENDATIONS:" header/bullet format it
always used.
router.py:1145 injects build_recommendations()'s output into the LLM system
prompt on the live mesh-DM path — verified byte-identical before/after
across mesh/region/node/missing/empty scopes via a synthetic fixture, and
pinned with a literal-string regression test so a future refactor can't
silently change that prompt text.
This unblocks wiring recommendations into the dashboard, which previously
only reached mesh DMs.
* feat(dashboard): serve real recommendations from /api/health
mesh_reporter was never on app.state — add it alongside health_engine etc.
in server.py's existing pattern. mesh_routes.py's health endpoint now
returns mesh_reporter.recommendations_list("mesh") instead of a hardcoded
[] TODO stub.
Add recommendations_available: bool alongside recommendations: string[] so
"the engine ran and found nothing" (healthy mesh) is never indistinguishable
from "the engine couldn't run" (unwired reporter, or an exception — logged
via logger.exception and swallowed so the rest of the health response still
serves). A crashed recommendations engine must not read as "mesh is
healthy" to an operator.
Also delete main.py's phantom `getattr(mh, "recommendations", [])` on the
websocket health_update push: nothing ever set .recommendations on the
mesh_health object (always []), and no frontend consumer reads it — the
REST endpoint above is the supported path.
* feat(dashboard-frontend): render recommendations in Nodes & Health
Meshtastic → Nodes & Health → Health tab used to be a duplicate
mesh_intelligence config editor (the same MeshIntelligenceSection also
edited from Settings → Intelligence, a stale-tab-overwrite hazard). Replace
it with a real operational view: fetch /api/health and render its
recommendations list.
Three states, matching visual conventions from Dashboard.tsx's alerts list
and MeshCoreRouting.tsx's cross-link note:
- recommendations present → Lightbulb list, one card per item
- empty + recommendations_available → "No recommendations — mesh is
healthy" (CheckCircle)
- recommendations_available === false → amber warning, not the healthy
state (AlertTriangle) — a crashed backend must not look healthy
mesh_intelligence config now has exactly one home: Settings. The Health tab
points there via the existing /config?section= deep-link pattern instead of
duplicating the editor. Config.tsx itself is untouched.
MeshHealth.recommendations/.recommendations_available are optional in the
type: the websocket health_update push doesn't include them (REST-only).
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
* chore(dashboard): remove orphaned /adapter-config route
AdapterConfig is already embedded directly in Environment.tsx
(excludeKeys={CURATED_KEYS} hideLlmToggle); no in-app <Link> ever
targeted the standalone /adapter-config route. Drop the route, its
App.tsx import, the extraTitleItems entry in Layout.tsx, and the
now-unused Sliders icon import. pages/AdapterConfig.tsx itself is
kept — it's imported directly by Environment.tsx.
* chore(dashboard): redirect 5 orphaned legacy routes to their tabbed homes
/gauge-sites, /town-anchors, /mesh, /meshtastic/sources, and
/meshcore/companion each duplicate content now tabbed inside Places,
MeshtasticNodes, and MeshCoreContactsCompanion, and have no in-app
<Link> pointing at them (verified with rg). Convert them to
<Navigate replace> redirects, matching the existing /notifications
and /data-sources precedent.
/town-anchors, /meshtastic/sources, and /meshcore/companion land on
their host's first tab rather than deep-linking to the second tab
(no ?tab= support exists) — same accepted trade-off as the existing
/notifications and /data-sources redirects.
Drop the now-dead extraTitleItems entries and the redundant
'/meshcore/companion' pathTitles entry, plus the Radio, Droplets,
Layers, and Bot icon imports that were only used by those entries
(MapPin is kept — still used by the /places nav item). These 5
routes were never nav entries (extraTitleItems only), so the sidebar
stays at 17 items with 5 Meshtastic / 5 MeshCore — MT/MC symmetry
unchanged.
* docs(dashboard-api): fix misleading refresh_toggles docstring
The docstring claimed this endpoint was "kept for backwards-compat
with the dashboard's manual ping path," but rg turns up no frontend
call site for /notifications/refresh-toggles anywhere in
dashboard-frontend. The auto-refresh middleware
(_auto_refresh_toggle_filter, registered via
register_config_routes_hooks) already refreshes the ToggleFilter on
every successful notifications config PUT, covering the normal case.
Endpoint is untouched -- comment only, kept for ops/debug use.
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
* chore: remove dead mesh_sources.py module
Superseded by mesh_data_store.py, whose docstring states it "replaces
mesh_sources.py with a clean three-layer architecture." No remaining
inbound imports (absolute or relative) repo-wide; the only surviving
references to "mesh_sources" are to the unrelated config YAML section
of the same name, which is untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove dead region_tagger.py module
Only reference to "region_tagger" repo-wide was inside its own
module docstring usage example; no live imports found.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove empty meshai/cli package
Package contained only a docstring, no code. No imports of
meshai.cli found anywhere; console entry point is meshai.main:main
(pyproject.toml), and packaging (tool.setuptools.packages.find,
include = ["meshai*"]) has no explicit reference to the cli
subpackage. No MANIFEST.in/setup.py/setup.cfg exist to update.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove orphaned swpc/swpc_last golden fixtures
63 files (40 in fixtures/swpc/, 23 in fixtures/swpc_last/) never
loaded by any test. harness/goldens.py's load_fixtures(hazard) is
the only generic fixture loader and is never called with "swpc" or
"swpc_last" (only "avalanche", "nws", "nws_last", "quake"). The sole
remaining textual reference, a provenance comment in
test_swpc_refactor.py:162 ("Fixture mirrors swpc_last/0003.json..."),
is left untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The repo's blanket `dist/` .gitignore rule (line 19, inherited from the
Python template) matches directories named `dist` at any depth with no
leading slash, so it silently excluded node_modules/<pkg>/dist/ for every
package whose build output lives there. 21 packages (including
react-router) declare main/module pointing into dist/, so the committed
node_modules was non-functional -- e.g. require.resolve('react-router')
threw MODULE_NOT_FOUND against the tracked copy.
work/Dockerfile already runs `npm ci && npm run build` in its build stage,
so production was never affected by this -- the committed copy was pure
dead weight (8,619 of 9,232 tracked files, 93% of the repo).
package-lock.json is tracked, making `npm ci` fully reproducible, so this
untracks node_modules/ (added a `node_modules/` ignore rule to .gitignore)
rather than trying to fix or narrow the dist/ rule.
History is NOT rewritten -- the old blobs remain reachable via prior
commits, so clone size is unchanged. This only stops new commits from
re-adding node_modules going forward.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: excise the dead Central NATS consumer path
Central was retired and its database dropped 2026-07-15; its NATS broker no
longer exists. Verified against the live CT108 deployment: all 12 adapters
run feed_source=native, zero on central, and central.enabled is False
(default, never overridden). The consumer and its handlers were unreachable.
Removed:
- central/consumer.py and 6 dead handlers (nws, quake, swpc, nwis, avy,
incident) -- their handle_* entrypoints were reachable only from the
consumer's dispatch
- the Central wiring in main.py (init, guarded start, retry loop, stop path)
- the dead config surface: CentralConsumerConfig, EnvironmentalConfig.central,
adapter_config ("central","severity_thresholds") and its display block
- the nats-py dependency (consumer.py was its only importer)
- 19 test files that exercised only the dead path
KEPT -- these live under central/ but are imported directly by native
adapters, and deleting them would break production:
- wfigs_handler.py: firms_handler._handle_pass_boundary() calls its _render()
on the live FIRMS growth-fire path (env/firms.py -> ingest_hotspot_pixel)
- firms_handler, satpass_handler, tle_handler: split files whose handle_*
entrypoints are dead but whose engines are live. Left intact; splitting
them is separate work.
- pass_predictor, budget, idaho_gauge_sites: fully live.
The usgs_quake keys global_mag_floor / regional_mag_floor / regional_centroid
/ regional_radius_mi / broadcast_pager_alerts are NOT removed despite comments
labelling them "CENTRAL-PATH ONLY" -- notifications/gating/quake.py reads them
unconditionally in the native path. Those comments are corrected separately.
Test-count note: the suite drops ~425 tests. Most were migration PARITY tests
whose sole purpose was proving the native rewrite byte-matched the Central
handler (golden byte-parity, cross-source identity, gate-sequence replay).
With the handler deleted there is nothing left to compare against, so they
cannot exist. Native-only tests were kept and reworked where a test reached
for a central symbol incidentally. This is a real coverage loss, accepted
deliberately: the parity harness proved the refactor faithful, and git
history preserves the originals.
Suite: 1984 passed, 6 failed -- the same 6 pre-existing failures as main
(stale SCHEMA_VERSION x3, expired TLE fixtures x2, one order-dependent),
all being fixed on fix/green-test-suite. No new failures.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(nws): restore native-only golden coverage for the wire formatter
Commit ca751fb5 deleted the Central nws_handler parity harness along with
the handler itself, which took the ONLY tests that pinned formatters.nws
.format()'s literal wire output. Gate-sequence and schema-conformance
tests already survived natively; the formatter's actual rendered text did
not have any native-only regression net.
Add TestFormatterGolden to test_nws_refactor.py: 3 real-fixture cases plus
6 hand-built pathological cases mined from the deleted test_nws_handler.py
(SVR path-sampling, the "no dangling separator" regression, TOR on-ground
vs radar-indicated, FFW flood-cause detection). Every literal was verified
by temporarily restoring the pre-excision central.nws_handler._render()
from git history (ca751fb5^) in a throwaway, uncommitted script, confirming
byte-identical output against the current native format() for all 37 real
fixtures (nws/ + nws_last/) and all 9 pathological cases, then pinning the
confirmed-matching string as the literal -- not a blind snapshot of
current behavior.
quake/swpc/avalanche/hydro/incident/fire were checked and already carry
equivalent native-only golden coverage (added directly in ca751fb5), so no
changes were needed there.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(persistence): derive SCHEMA_VERSION from the migrations directory
db.py hardcoded SCHEMA_VERSION = 26 while migrations/ had already reached
v29 (v27 dispatcher floor-drop counter, v28 mesh_observations, v29 IPAWS).
The migration runner globs the directory and applies every vN.sql it finds
regardless of the constant, so a fresh DB actually landed at 29 while the
constant claimed 26 -- a three-version drift that three tests were
correctly catching.
Derive it from the highest vN.sql present instead of bumping the literal,
so it cannot drift again the next time someone adds a migration. Falls back
to 0 if the directory is missing so import never fails; the migrations dir
sits alongside db.py and ships with the package (Dockerfile COPYs meshai/).
Adds a regression guard asserting the constant matches the highest
migration file, and updates three tests that hardcoded 26 as a literal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(tle): make TLE fixtures time-relative so they cannot expire
The ISS fixtures hardcoded epochs of 2026-06-30/07-01/07-02. tle_handler
sets STALE_DAYS = 14 and get_tle_by_norad() filters on epoch >= now - 14d,
so the fixtures silently aged out on 2026-07-02 and the tests began
failing -- a time bomb, not a regression.
Compute epochs relative to wall-clock now (base = now - 2d, +/-1d for
newer/older) with correct TLE epoch-field encoding and mod-10 checksum.
STALE_DAYS is untouched -- widening it in product code would have changed
production behavior to paper over a test bug.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(fire-tracker): pin config + history db to tmp_path
load_config() defaults HistoryConfig.database to the relative path
"conversations.db", resolved against the process CWD, so every test calling
load_config() with no override shares one file for the whole session. The
conftest DB-isolation fixture only covers MESHAI_DB_PATH, not this.
Point both the config dir and the history database at the test's tmp_path.
NOTE: this does NOT resolve the order-dependent failure -- the test still
passes standalone and fails in a full run, so the polluting state lives
somewhere other than config/history. Left failing rather than weakened;
root cause still unidentified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
County-only civil CAP alerts (SAME geocode, no <polygon>) had no geometry, so
the geometry-based region tagger could not place them: no region -> no
region_routes match -> silently not broadcast. Many CEMs / 911 outages / some
AMBER alerts are county-only.
- Bundle work/meshai/county_centroids.py: Census 2023 national county gazetteer
internal points (3,222 counties + DC/territories), FIPS->(lat,lon), with
SAME PSSCCC -> 5-digit FIPS helpers.
- env/ipaws.py: when a CAP alert has SAME geocode(s) but NO polygon, set a Point
(single county) or MultiPoint (multi-county) geometry from the county
centroid(s) so the EXISTING coverage/region tagger locates it and tags ALL
matching regions. Real polygon geometry always wins (never overridden).
- dashboard/server.py: call register_config_routes_hooks(app) in create_app so
the toggle auto-refresh middleware is actually wired in prod (was test-only);
saving a family toggle now takes effect live without POST
/api/notifications/refresh-toggles.
Tests: county-only alert tags SW Idaho + matches emergency route cell;
multi-county tags all regions; polygon path unchanged; create_app wires the
refresh middleware; panhandle coverage-gap documented.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The IPAWS civil-alert adapter shipped backend-complete but frontend-partial:
it rendered only in the generic adapter-config page and the Advanced (raw)
Data Feeds tab, and the `emergency` family was absent from the curated Data
Feeds panel, the family-settings toggles, and the MeshCore routing matrix.
Adapter (ipaws), benchmarked against firms/nws:
- Environment.tsx: add `ipaws` to AdapterKey union, EnvConfig interface,
META (native-only, keyless), a new `emergency` FAMILIES group, PANEL_META_KEY
(LLM toggle), and a hand-written renderSettings panel exposing base_url,
user_agent, tick_seconds, state_fips, same_codes, exclude_weather,
status_actual_only — with coverage-scope handling like the other adapters.
- Environment.tsx: IPAWS_DEFAULT backfill so pre-ipaws GET payloads don't crash.
- Dashboard.tsx: SOURCE_ICONS entry (Siren/IPAWS) so ipaws events aren't a slug.
- ActivityLog.tsx: TABLE_LABELS + CATEGORIES + text-hint so ipaws_alerts rows
show labeled "Emergency" and honor the category filter.
- dispatcher.py: _SOURCE_TO_TABLE fallback ipaws -> ipaws_alerts so region-routed
emergency sends land labeled in the audit feed (not NULL).
Family (emergency), benchmarked against fire:
- Notifications.tsx: add `emergency` to TOGGLE_FAMILY_META (Siren icon). This
cascades to Family Settings, the Meshtastic delivery matrix, and the MeshCore
routing matrix (the last was hardcoded to the static list and previously
omitted emergency entirely). Backend VALID_TOGGLES/gating/categories were
already complete — no backend family change needed.
Tests: update the _SOURCE_TO_TABLE exact-match guard and add an ipaws audit-row
parity test. Full suite 2407 passed / 6 pre-existing unrelated failures.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Saving the "Auto-advert interval" dropdown on the MeshCore Companion page
took BOTH radios offline on 2026-07-17 06:46:52. One click, full outage.
The page PUT a single-key body to /api/config/connection:
{"meshcore_advert_interval_seconds": 10800}
_dict_to_dataclass() builds kwargs only from the keys present in the body
and lets `cls(**kwargs)` default the rest, so every OMITTED field was reset
to its dataclass default and written to disk:
type: tcp -> serial (Meshtastic offline)
tcp_host: 192.168.1.100 -> <lost> (LOCAL_FIELDS, see below)
tcp_port: 4404 -> 4403 (wrong meshmonitor vnode)
meshcore_host: 192.168.1.253 -> '' (MeshCore off; blank = off)
meshcore_conn_type: serial -> tcp (wrong transport)
meshcore_serial_port: /dev/meshcore-rak -> '' (RAK radio lost)
It was silent twice over. `connection` is restart-required, so the running
process kept the good in-memory config while the file sat gutted, waiting
for any restart to detonate. And save_section() writes the domain file
FIRST and local.yaml SECOND: meshtastic.yaml hit the disk already gutted,
then the local.yaml write (which owns connection.tcp_host via LOCAL_FIELDS)
died on `[Errno 13] Permission denied` -- so tcp_host landed in neither
file, and the 500 that would have named the cause was swallowed by the UI.
The operator saw nothing happen.
This was never one page's bug: PUT /api/config/{section} was destructive on
a partial payload for EVERY section. Other callers only survive because they
happen to spread the full object first.
Fixes, in depth:
* Route (the durable fix): merge the body over the CURRENT live section
before coercing, so omitted keys keep their live values while present
keys -- including '' / False / [] -- still apply. The base is the live
config, the same values GET serves, so a partial PUT now lands exactly
where a full-object PUT from that same GET would. Full-object callers are
unaffected. Fixed at the HTTP boundary, not in _dict_to_dataclass():
absent-key-means-default is CORRECT at config-load time, where a file
legitimately omits fields it does not override.
* Nested semantics keyed off the dataclass schema, not "is it a dict":
nested dataclass fields DEEP-MERGE (a partial region_routes must not drop
sibling cells), while bare dict/list fields REPLACE at the key (cells,
toggles, destinations, rules are dynamic maps -- deep-merging them would
resurrect deleted keys and make deletion impossible, the mirror image of
the bug being fixed).
* Page: send the full connection object like every other caller does.
* Errors are visible: the save handler no longer swallows the exception,
and updateConfig() surfaces the server's `detail` rather than a bare
"API error: 500", which is what hid Permission denied from the operator.
* Default advert interval 10800 -> 86400 (24h). 3h is far too frequent a
default for a public mesh; the UI "(default)" label moves to match.
Tests: tests/test_config_partial_save_merge.py reproduces the outage with
the exact payload, and pins merge semantics across connection AND
notifications, intentional clearing, deep-merge, and map-deletion.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
self_info() reported host/port straight from config regardless of
conn_type, so a serial companion still advertised whatever stale
meshcore_host sat in the config — the API named a device meshai was not
talking to, which is enough to send an investigation to the wrong radio.
Connection details now come from one _connection_descriptor() shared with
connect(), so the log line and the API can't drift; only the live
conn_type's fields are populated and the rest are null.
meshai's device view is otherwise built once at connect and never re-read
— contacts via ensure_contacts(), channels via _enumerate_channels(). The
lib's contact handler only ever merges (meshcore.py::_update_contacts), so
a cached roster can never shrink, and a channel provisioned on the radio
stays invisible until the process restarts. There was no refetch path at
all. Adds an explicit resync that re-reads BOTH halves: a FULL
get_contacts(lastmod=0) reconciled with replace semantics (absent contacts
are dropped) plus a channel re-enumeration, each reporting what changed.
Also adds a preventive route-health check: every region_routes cell whose
MeshCore target cannot be resolved against the live roster/channel table
is surfaced, since such a send fails silently. Room targets are matched by
pubkey prefix, exactly as the dispatcher resolves them, so a picker-stored
prefix is not misreported as dangling. Same-name/different-pubkey roster
entries are flagged too — a name alone cannot identify a contact, which is
the trap behind a room rebuilt under a new keypair.
Backend:
- meshcore_roster.py: pure reconcile_contacts / check_route_health /
find_name_collisions (no device I/O — unit-testable without a radio)
- transport: _connection_descriptor, resync, refresh_contacts,
remove_contact, import_contact, export_roster, contacts_synced_at;
auto_update_contacts enabled (configurable — it costs one incremental
fetch per advert heard, which is real chatter on a dense mesh)
- API: POST contacts/refresh, DELETE contacts/{pubkey}, GET
contacts/export, POST contacts/import, GET route-health
Frontend (existing Contacts & Companion page — no new page or nav entry):
- dangling-route + name-collision banners; resync/export/add-contact
toolbar with last-synced and the added/removed counts; staleness badges;
search, filters and sortable columns; per-contact delete behind a
confirm; Companion tab shows the real transport + target.
A full pubkey is required to delete or add: the lib resolves by prefix,
and a prefix could silently hit the wrong node.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a new native `ipaws` adapter for FEMA IPAWS-OPEN EAS civil alerts.
- Two-stage CAP fetch via base_url (direct FEMA or Conduit proxy): Atom
index -> per-entry CAP 1.2 documents.
- Non-weather civil alerts only (evacuation, Civil Emergency Message,
AMBER, 911 outage, law-enforcement, HazMat); NWS/NOAA CAP dropped so
weather is never double-broadcast.
- Idaho + neighbour statefips scope gate applied before stage-2 fetch.
- Own `ipaws_alerts` dedup table (migration v29); reuses the NWS CAP
severity + formatter pattern.
- Ships enabled=False (no transmit until explicitly enabled).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
traffic and firms no longer idle when their key is blank — they build a
keyless request (traffic: omit key= param; firms: omit the map_key path
segment), matching roads511's existing optional-key pattern. Enables
routing these feeds through a key-injecting proxy (Conduit) with the key
held only there. Key-set behavior is byte-identical.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a base-URL config field (default = the current value, backward-compatible)
to the 9 adapters that hardcoded their upstream URL — nws, swpc (4 endpoints),
ducting, fires (perimeter+points), firms, avalanche, usgs streams (3 bases),
traffic, satpass/tle_fetch — mirroring the already-compliant roads511 pattern.
Every feed URL is now overridable via config, enforcing the "everything
configurable" rule and making each adapter live-repointable via a config PUT.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
EnvironmentalStore.apply_config() rebuilds only the changed native
adapters in place on a config PUT -- dedup/seen state (store-level) is
preserved, unchanged adapters are untouched. Drops "environmental" from
RESTART_REQUIRED_SECTIONS; falls back to restart-required only for the
narrow feed_source->central case. Cascades nifc/fires -> firms. The old
restart requirement was a Central-era coupling, now moot (all-native,
central.enabled=false, CentralConsumer inert).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ReminderScheduler is the third fire-broadcast exit and the only one
that does not pass through the EventBus, so FirePacer (which paces the
Central and native fire-event exits to <=1/60s) never sees it. Its tick
is a roll-call: every eligible row is its own broadcast, dispatched in a
plain `for` loop with no gap. Unpaced, N eligible fires produce N
back-to-back mesh transmissions; the only downstream protection is
RadioSendQueue's ~2.2-2.6s per-transport inter-packet jitter, which
prevents packet collision but still lets a roll-call monopolise the mesh.
Not currently firing in production (every overdue fire is filtered by
terminate_when, so the eligible set is 0) -- this is fire-season
hardening against a latent burst, not a live incident.
Adds `spacing_seconds` (adapter_config, default 60 to match FirePacer)
enforcing a minimum gap between consecutive SUCCESSFUL reminder
deliveries. Deliberately a pure spacing change:
* WHAT gets broadcast is untouched; nothing is dropped.
* The ok-gated last_broadcast_at stamp still uses the tick's `now`.
* A failed dispatch sent no packet, so it does not arm the gap.
* Rows filtered by terminate_when/render never burn a spacing slot.
* A lone eligible fire has nothing to pace against -> zero added latency.
* The wait is interruptible by stop(): a 15-fire roll-call holds
tick_once() for ~14 min and stop() awaits the tick task, so a plain
sleep would stall shutdown.
Chose in-loop spacing over routing reminders through FirePacer itself:
reminders re-derive their targets from live DB state every tick and only
clear a row via last_broadcast_at after a confirmed send, so enqueuing
into a 60s-drain FIFO would re-enqueue the same fire on every intervening
tick -- the queue would grow faster than it drains. pacer.py, consumer.py,
store.py and main.py are untouched.
Tests fake the clock end-to-end, so 60s spacing costs the suite nothing.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gating/firms.py's decide() stamped a plain "severity" key in the
data_patch for the wildfire_spotting and wildfire_halted broadcast
paths. central/consumer.py only ever promotes data["_severity_override"]
onto Event.severity -- the plain key is a silent no-op, the same class
of bug as #118 (fixed for firms_handler.py's own inline stamps in
PR #120). Currently inert (MESHAI_CUTOVER_CATEGORIES is unset by
default), but the moment wildfire_spotting/wildfire_halted are cut
over, spotting would silently stop being "immediate".
- gating/firms.py: both data_patch sites now use _severity_override.
Checked the other gating modules (fire.py, avalanche.py, swpc.py,
quake.py, nws.py) -- all already use _severity_override correctly;
firms.py was the only one with the plain-key mistake.
- Fixed the stale module docstring claiming the unattributed-hotspot
cluster path "is DEAD" -- it has been live since d479ca53 (#73); the
stale comment directly caused a bogus bug report against production.
- test_firms_refactor.py: updated two existing tests that had codified
the buggy plain-"severity" behavior as expected, and added
TestCutoverSeverityReachesEvent, which drives the real cutover path
end-to-end through CentralConsumer._normalize and asserts the
emitted Event's severity (immediate for spotting, routine for halt).
Verified both new tests fail against the unfixed decider and pass
against the fix.
Full suite: 20 failed, 2242 passed, 72 skipped (vs. origin/main
baseline 20 failed, 2240 passed, 72 skipped -- same 20 pre-existing
failures, confirmed identical with this change stashed out; +2 passed
are the new regression tests).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
MeshCoreTransport defined _resolve_contact TWICE:
- L171 (PR #56) cache lookup -> on miss, get_contacts(lastmod=0) full
refetch -> retry. The DM / path-establishment resolver.
- L1227 (PR #92) key-prefix -> by-name lookup, no refetch. The telemetry
resolver, added later without noticing the collision.
Python silently keeps only the LAST definition in a class body, so the
line-171 implementation was dead code and PR #56 was nullified: every DM
and path-establishment caller was getting the telemetry resolver instead.
No error, no warning, invisible to the linter and the type checker.
Fix: rename the telemetry resolver to _resolve_contact_for_telemetry and
repoint its sole caller (_req_telemetry_async). The DM path (send_message)
and _establish_direct_path now get PR #56's refetch-on-miss behavior back,
which is what they need — replying to an inbound DM from a firmware
auto-added contact requires the refetch, and re-resolving after path
discovery is pointless without it.
Deliberately NOT merged into one resolver: telemetry auto-polls on a timer
against operator-selected contacts already in the roster, so a full-roster
refetch on every miss is recurring airtime for nothing; and its by-name
fallback is telemetry-specific and must not widen DM address resolution.
The two want different semantics — the bug was the name collision, not that
they should be one function.
_resolve_contact_async (the MC-event-loop twin) already carried the refetch
and was never shadowed, so the async/queue DM send path was unaffected.
Add tests/test_no_duplicate_methods.py: AST-walks every ClassDef under
work/meshai/ and fails if any class body defines the same method name twice.
This failure mode is invisible to review, the linter, and the type checker —
which is exactly why it survived. Exempts the legitimate same-name patterns
(@property/@setter/@deleter groups, @overload stacks). Verified it flags the
bug on the pre-fix source and finds no other duplicates in the tree.
Suite: 20 failed -> 17 failed (the 3 meshcore failures gone), 2240 -> 2245
passed (+3 fixed, +2 new guard tests), 72 skipped unchanged.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
wildfire_spotting is the most urgent signal in the system (a fire
throwing embers past its own containment line), but every fire event
carries a group_key (= event_id), so the Grouper held spotting for the
full grouper_window_seconds (60s live) before it could even reach the
dispatcher -- a ~60s latency FLOOR, and ~120s+ once FirePacer queuing
is added on top.
Add a module-level _NEVER_COALESCE_CATEGORIES frozenset and bypass the
coalescing window for the categories in it. Currently: wildfire_spotting
only.
This is safe because spotting is already rate-limited AT THE SOURCE: a
per-fire 1h cooldown (adapter_config.fires.spotting_cooldown_seconds,
default 3600, latched on fires.last_spotting_broadcast_at per irwin_id
in gating/firms.py) gates spotting DETECTION itself, so N active fires
yield at most N spotting alerts per hour. FirePacer (60s interval, with
head-of-line ordering for immediate severity) and the dispatcher's
per-(toggle, category, region) cooldown still apply downstream.
The bypass is scoped by CATEGORY, never by severity. Commit 85d48ce3
deliberately removed a severity == "immediate" bypass from this exact
spot because ALL fire events carry _severity_override="immediate", so a
severity bypass exempts the entire fire family from rate control. The
comment on the constant spells that out so it does not get re-added.
Tests: spotting with a group_key passes straight through; wildfire_growth
and wildfire_incident at immediate severity are STILL held (proving no
severity bypass crept back in); spotting with no group_key still passes
through. PR #129's test_immediate_severity_is_also_coalesced_no_bypass is
untouched and still passes.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Triaged all 20 known-red tests. 13 were stale tests asserting rotted
expectations against deliberate, documented behavior changes; fixed by
deriving expected values instead of hard-coding, or updating the
expectation to match a documented policy change:
- test_adapter_config_foundation.py / test_adapter_config_api.py:
REGISTRY/API key-count and key-set guards hard-coded magic numbers
(59/94/17) that rotted repeatedly. Now derive expectations from
REGISTRY itself and, for the schema version, from the migrations
directory, so they can't rot the same way again.
- test_fire_tracker_phase4.py: two tests hardcoded a nonexistent
deployment path (/opt/meshai/meshai/router.py) that matches no
Dockerfile WORKDIR in this repo; resolve the module path via
importlib.util.find_spec instead.
- test_tombstone_broadcast.py: asserted fire severity == "immediate",
which commit 2f677e85 deliberately downgraded to "priority" (to stop
fire broadcasts bypassing the Grouper/cooldown during NATS backlog
replay) without updating this test.
- test_pipeline_grouper.py: test_immediate_severity_bypasses_grouper
asserted an immediate-severity bypass that commit 85d48ce3
("fix(fire): remove immediate-severity exemption from grouper +
cooldown") DELETED on purpose -- fire events carry
_severity_override="immediate", and the exemption left fire with no
rate control at all in normal live operation. Re-adding the bypass
would re-open that fire-spam hole on a public-safety mesh, so the test
moves, not the source. Renamed + inverted to assert the real contract
(all severities coalesce; only a missing group_key passes through).
- test_tail_followups.py: dispatcher mock was missing
dispatch_scheduled_fire_broadcast (a method added alongside the
generic dispatch_scheduled_broadcast; test_reminders.py already
mocks both).
- test_tracking_v057.py: guard required an empty tracking-family
adapter list in Environment.tsx, but the frontend has long grouped
the pre-existing native satpass adapter under the "Tracking" display
section (its own "satpass" backend toggle, unrelated to the Phase-7
tracking family every other guard in this file confirms is still
unimplemented). Narrowed the guard to allow only that known entry.
- test_v052_dispatcher.py: two tests used category="wildfire_incident",
which the phase3b fire migration (#33) forced onto a dedicated
formatter via NATIVE_ALWAYS_DECIDE; swapped to wildfire_hotspot
(same emoji/label, not in NATIVE_ALWAYS_DECIDE) to keep exercising
the generic composer logic under test.
Also fixes one stale COMMENT (comment-only, no logic change) in
meshai/notifications/pipeline/__init__.py's start_pipeline(): it still
claimed "Immediate events bypass the grouper and don't need this
[periodic flush]", which has been false since 85d48ce3 and is precisely
what makes the deleted bypass look like a missing feature. The comment
now records that the removal was deliberate and must not be reverted.
The remaining 6 failures are left untouched -- 2 confirmed real bugs, to
be fixed deliberately in their own changes:
- meshcore_transport.py defines `_resolve_contact` TWICE on
MeshCoreTransport (line 171 from PR #56, line 1227 from PR #92). The
second silently shadows the first, so the DM contact-resolution
refetch-on-miss that #56 added is dead code in production. (3 tests)
- SCHEMA_VERSION (persistence/db.py:33) is stale at 26 vs. the actual
highest migration v28; v27 and v28 shipped without bumping it.
(3 tests)
Plus 1 environment gap, not a code defect:
test_natural_language_fire_question_routes_to_llm needs the `openai`
package, which is declared in requirements.txt but not installed here.
Suite: 20 failed, 2240 passed, 72 skipped -> 7 failed, 2254 passed, 72
skipped. All 7 remaining failures are ones classified above; no new
failures introduced elsewhere in the suite.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Native fire adapters (env/fires.py source="nifc", env/firms.py
source="firms") emitted straight to the EventBus from
EnvironmentalStore._emit_event with no rate limiting of their own.
FirePacer was only ever attached to CentralConsumer (main.py), which
never runs in the actual production deployment (central.enabled=False,
all adapters feed_source=native) -- so the <=1/60s throttle + immediate
head-of-line behavior fixed for Central in #120 (issue #119) was
completely inert in production. A poll that produces several distinct
fires/clusters at once (a lightning outbreak, or several tracked fires
crossing a satellite-pass boundary together) would dump all of them on
the mesh back-to-back instead of at the intended cadence.
_emit_event() now routes fire-family Events (source in
{"nifc","firms"}, severity in {"priority","immediate"}) through an
attached FirePacer, mirroring the exact gate CentralConsumer._handle
applies. main.py attaches the same FirePacer instance to env_store
right after constructing it. "routine"-severity fire events, non-fire
native adapters, and the Central path are all unaffected; a paced event
cannot re-enter either gate (native vs Central are mutually exclusive
per feed_source), so nothing can be paced twice.
Added tests/test_native_fire_pacer.py covering: native fire events
route through the pacer, an immediate event jumps an already-queued
priority queue with nothing dropped, routine-severity fire events and
non-fire native events are NOT paced, and the no-pacer-attached
fallback is unchanged.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Three independent bugs kept firms_handler's growth/spotting/halt/cluster
fusion decisions from reaching a correct mesh Event:
- #117: consumer._normalize() computed `category` from the raw Central
category BEFORE the per-adapter handler ran and never re-read
data["category"] afterward, so every firms_handler category stamp was a
silent no-op. Now re-read post-dispatch, validated against the known
category registry (unrecognized overrides are logged and ignored).
- #118: consumer.py only ever honors data["_severity_override"], but
firms_handler's halt/spotting/cluster sites stamped the plain
data["severity"] key instead (only growth used the right key). Switched
all three sites to `_severity_override` for one consistent contract.
This is severity plumbing only -- it does not change which events fire.
- #119: FirePacer's gate only matched source in ("fires","wfigs") at
severity=="priority", so FIRMS fusion broadcasts (source="firms",
growth/spotting at "immediate") never reached the pacer. Broadened the
gate to cover "firms" + {"priority","immediate"}, and gave FirePacer
head-of-line insertion so an "immediate" event is never stuck behind
already-queued "priority" events. Still unbounded/never-drops.
Cluster detection is left exactly as main ships it: live, always on, no
toggle (PR #73's curated new-fire cluster broadcasts with cold-start
silent-seeding). Only its severity-override key changes, under #118.
Updated existing tests that asserted the old (buggy) data["severity"]
contract, and added tests/test_firms_fusion_event_contract.py covering
all three fixes end-to-end through consumer._normalize()/_handle() and
FirePacer directly.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Section 1.5 (region_routes matrix branch) armed and checked the
per-region cooldown key as (toggle, category, region) with no
channel-type component. _chans always inserts mesh_broadcast before
meshcore_broadcast (insertion order in the per-cell append loop), so
for any matched cell with BOTH mt and mc populated, the mesh_broadcast
send armed the cooldown key first; the very next iteration checked
that SAME key for meshcore_broadcast and saw it as freshly cooled
down, dropping it every time. Net effect: meshcore_broadcast never
succeeded via the matrix branch whenever cooldown_seconds > 0 (true
for weather/roads/fire, all 300s), so it never armed its own
dedup/cooldown state either -- a silent, permanent MC blackout for
every region-routed family. Confirmed live: dispatcher_dedup had 555
rows, zero meshcore_broadcast; mesh_broadcasts_out was 120:4 MT:MC for
nws_alerts and 98:8 for traffic_events over 10 days (the few MC rows
that got through came from a different, non-matrix code path). fires'
98:77 near-1:1 ratio is not evidence the matrix branch worked for
fire -- those MC sends are dominated by the cooldown-exempt scheduled
reminder path (dispatch_scheduled_fire_broadcast); fire's own live
event-driven path has the identical latent bug, just masked.
Fix folds ch_type into the region string (mirrors the existing
_cd_suffix convention) so mesh_broadcast and meshcore_broadcast get
independent cooldown windows. Kept the cooldown key a 3-tuple
(instead of widening to 4) to avoid a dispatcher_cooldowns schema
migration -- _persist_cooldown() and the boot-restore SELECT are both
hard-coded to (toggle, category, region).
Verified in isolation (no live/deployed behavior change, no
transmit): a fresh Dispatcher built from the live production config
now dispatches both mesh_broadcast and meshcore_broadcast for
weather/roads/fire matched cells under a 300s cooldown. Added two
regression tests covering the gap that let this ship untested: no
existing test combined cooldown_seconds > 0 with a cell that has BOTH
mt and mc populated (test_cell_match_routes_mt_and_mc uses the
cooldown_s=0 default; test_per_region_cooldown_independence uses
cooldown_s=300 but with mc=None on every cell).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mesh-context buffer (recap/summary source) was pure in-memory
(deque(maxlen=50000)), wiped on every container restart. Adds
mesh_observations table (migration v28) + persistence/mesh_observations.py
accessors following the existing get_db()/observer_locations.py pattern.
MeshContext.observe() now write-throughs each observation to SQLite
(fail-safe -- DB errors are logged and swallowed, never block or drop the
in-memory path). __init__ loads recent rows (within max_age, up to the
hard cap) back into the deque on startup so recap works immediately after
a restart. prune() now also deletes SQLite rows older than max_age on the
same hourly cadence as the existing in-memory prune.
Verified: wrote one observation through the real observe() path, confirmed
1 row in mesh_observations; docker restart meshai; log confirms "Loaded 1
mesh observations from durable store" and a fresh MeshContext instance
recovers the observation via get_context_block().
Investigated a suspected recap-grounding bug (LLM ignoring the mesh-context
block and replying with a disclaimer) per the same task brief, but could
not reproduce it against gemini-3.1-flash-lite across 8 query phrasings
(minimal and full production-fidelity system prompts, including the
configured static prompt clause that explicitly permits the disclaimer
when no traffic is shown) -- the model correctly grounds its replies in
the observed traffic block in every case tested. No grounding-clause
change made; premise did not reproduce.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(meshcore): self-service add/remove channel provisioning from dashboard
Adds MeshCoreTransport.add_channel()/remove_channel() (meshcore_transport.py),
scanning the full 40-slot companion channel table for a free slot (no early
empty-run cutoff, unlike _enumerate_channels) and writing/clearing slots via
the only available write opcode (set_channel, 0x20) — the lib exposes no
delete-channel opcode, so removal writes the slot back to its empty state
(blank name + all-zero 16-byte secret).
POST /api/meshcore/channels and DELETE /api/meshcore/channels/{name} routes
(mesh_send_routes.py) validate name/hex-key and return the refreshed channel
list, matching the existing secrets_routes.py HTTPException idiom.
Frontend: Add-channel row (name + optional PSK hex) and a per-channel Remove
affordance inside the existing Observe MeshCore Channels block
(MeshCoreConnection.tsx), plus addMeshcoreChannel()/removeMeshcoreChannel()
API client functions (api.ts).
Built + deployed to CT 108 (docker compose build && up -d, container
healthy); self-cleaning smoke test passed — POST/DELETE of a #meshai-test
channel left the companion table exactly as found.
* MeshCore: display per-channel key (PSK) with reveal + copy
The Observe MeshCore Channels list now shows each channel's PSK hex so
operators can share it with people who want to join. Fetches the
/api/meshcore/channels/detail endpoint (name + key) instead of the
names-only list; keys are masked by default with a per-row eye reveal
toggle and a copy-to-clipboard button. Channels with no retrievable key
show a dash. Existing observe (checkbox), remove (trash), and add-row
controls are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(llm): recommend a MeshCore channel by location + flash-lite default
Add a location-aware MeshCore channel-recommendation context block to the
single-shot LLM prompt so "what channel should I join?" gets an answer with
the channel NAME and join KEY.
- router.py: new standalone helper _build_meshcore_channel_block(config,
channel_details, position) — lists ONLY observed channels
(meshcore_context.observe_channels), each with its PSK key
(channel_details) and, where a region_routes cell maps to it, the covered
region's human geography (local name, cities, centroid) built from a
channel->regions reverse map over notifications.region_routes.cells and a
best-effort fuzzy attach to mesh_intelligence regions. Injected inside the
existing should_inject_mesh gate, right after the region-geography block;
fetches channel_details() + get_node_position() off self.connector and is
fully fail-safe (any error skips the block, never breaks the LLM path).
Empty observe_channels => empty block (LLM cannot invent channels).
- composite_transport.py: add channel_details() passthrough mirroring
known_channels().
- README.md: bump the example llm model to gemini-2.5-flash-lite so a fresh
deploy is policy-compliant (flash-lite supports Google Search grounding).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(identity): transport-aware bot identity + alert-channel awareness
Fixes the LLM system prompt being transport-blind: MeshCore DMs were
being told they were on the freq51 Meshtastic mesh network and were
physical node !27780c47 (AIDA-N2), and MeshMonitor/node-health text
(Meshtastic-only) leaked into MeshCore prompts too.
- config.py: BotConfig gains mt_mesh_name/mt_node/mc_mesh_name (generic
OSS defaults empty; name/owner defaults now generic MeshAI/Unknown).
LLMConfig.system_prompt code default no longer hardcodes freq51/Twin
Falls.
- router.py: generate_llm_response() computes transport once and
branches identity: MeshCore gets an AIDA/MeshCore-radio framing with
no MT node id; Meshtastic keeps the physical-node framing from the
new config fields. MeshMonitor block and MT node-health/gateway/packet
reporting (mesh_reporter Tier1/region/node detail + _MESH_AWARENESS_PROMPT
+ region geography) are now gated to transport == meshtastic. New
_build_alert_channels_line() adds a short identity-block line on both
transports listing region_routes-derived alert channels (MT channel
index or MC channel name). MeshCore channel-recommendation block stays
transport-neutral.
- README.md: dead gemini-2.5-flash-lite example updated to
gemini-3.1-flash-lite.
Live config (meshai_data volume, not git-tracked): bot.name=AIDA,
bot.owner=K7ZVX (local.yaml identity.owner, which overrides config.yaml
per config_loader.py LOCAL_FIELDS), mt_mesh_name/mt_node set to the
freq51/AIDA-N2 values, mc_mesh_name left empty. llm.yaml system_prompt
freq51 line removed.
Verified via in-container prompt-assembly dry run through the real
generate_llm_response() path (no LLM call) for both transports, and a
live gemini-3.1-flash-lite 3-query behavior test confirming the
MeshCore channel-recommendation feature still works correctly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds PUT/DELETE /api/meshcore/room-password/{pubkey} (dedicated route — the
generic secrets allowlist rejects dynamic per-room vars) and a password_set
flag on GET /api/meshcore/rooms. The routing picker gains an inline lock +
set/clear editor on room-mode cells; state is keyed by room pubkey and shared
across cells targeting the same room. Send-time login already reads the stored
password via secrets_store — no dispatch changes needed.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
The MeshCore routing cell's channel-vs-room toggle derived its mode
solely from whether the value started with `room:`, and the Room button
merely cleared the value -- so room mode was never entered, the room
<select> (gated on that) never rendered, and a room could never be
selected (chicken-and-egg: the select was the only writer of the `room:`
value). Add explicit per-cell mode state that the toggle sets directly
(falling back to value-derived on first render, so existing room: cells
load in room mode); Home now enters room mode and renders the dropdown
even with an empty value, Hash renders the channel input.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends MeshCore routing so a region_routes `mc` cell can target a room
server, not just a `#`-channel. A room = a contact with type==3; a cell
value of `room:<pubkey>` routes to it via the existing addressed DM path
(send_msg to the room's pubkey), with an optional login for
password-protected rooms; a bare cell value stays a channel broadcast
(unchanged). Adds transport get_rooms()/login_to_room()/send_to_room_async,
a GET /api/meshcore/rooms endpoint, per-room password storage in
secrets_store (env MESHCORE_ROOM_<prefix>_PWD), and a routing-GUI
channel-vs-room picker (rooms shown by name with a path indicator).
Open rooms work end-to-end. Password-protected rooms need a follow-up:
a backend endpoint to SET the per-room password from the GUI (the generic
secrets API is allowlist-gated); the storage + login already exist.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The WZDx daily summary + DM query count from traffic_events, but work
zones weren't landing there: wzdx rode the generic _delta_emit path,
which silent-seeds the seen-set and returns before the decider's INSERT
on the cold-start first poll, so the current zone set never persisted
(summary would count ~0). Add a dedicated _ingest_wzdx (mirroring the
fires ingest) that UPSERTs every current coalesced zone into
traffic_events each poll (persist-only, last_broadcast_at=NULL, no emit,
no broadcast) and reconciles zones that drop out of the feed (never wipes
on an empty/failed fetch). Per-event work-zone broadcast stays suppressed
(the decider's work_zone gate is untouched). Retargets 4 tests in
test_store_received_delta.py that used a fake 'wzdx' source to exercise
the generic gate onto a neutral routing name.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1. WZDx work zones: replace per-event broadcasting with a once-a-day
per-region count summary. Coalesce the upstream per-direction /
per-schedule-day fan-out into one row per physical zone
(road + lat3 + lon3 + sub_type); work zones are stored in
traffic_events but no longer per-event broadcast, while 511 crash /
closure / hazard incidents still broadcast live. A WZDxSummaryScheduler
emits one count line per coverage region once a day (default 07:00
America/Boise), routed via the region_routes 'roads' cells; work-zone
details are DM-queryable (build_work_zones_detail). New config:
wzdx.summary_enabled / summary_time / summary_tz.
2. FIRMS cold-start is now restart-safe: gate the silent-seed on the
persisted firms_pixels baseline being empty (first-ever run) instead of
an in-memory per-boot flag, so a restart no longer silently absorbs a
genuinely-new hotspot cluster.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two fire-scope cleanups:
1. Remove the fire digest feature entirely -- scheduler
(notifications/scheduled/fire_digest.py), pipeline wiring, the
fires.digest_* adapter_config key registrations, and the Fire Digest
dashboard UI (ScheduledBroadcasts / Environment / Reference /
AdapterConfig / ActivityLog). The unrelated generic per-rule
notification digest is kept. Orphaned fires.digest_* config rows and the
fire_digest_broadcasts table are left as inert data (v16 migration
untouched).
2. Add a coverage-scope gate at fire ingest: _ingest_fires now skips any
fire whose coordinates fall outside all configured coverage areas (same
areas_from_config + classify_geom_areas membership the dispatch-level
CoverageFilter uses), so out-of-coverage fires are never stored, tracked,
alerted, reminded, or re-ingested. Fails open when coverage is disabled,
has no areas, or excludes the fires adapter.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fire (wfigs) reminders previously dispatched via the generic scheduled
path, which hardcoded the rf_propagation toggle (Meshtastic ch4 / MeshCore
#aida) and ignored the fire's region. They now route through a new
dispatch_scheduled_fire_broadcast() that builds a synthetic fire event from
the fire's lat/lon, derives its region the same way the live fire event
path does, and routes per region_routes.cells['fire'] (per-region MT/MC
channels), falling back to the fire toggle's own defaults when a transport
isn't matrix-owned -- never rf_propagation. rf_propagation and 511
reminders are unchanged. Reminders remain disabled (reminders_wfigs.enabled
stays false); this only fixes routing for when they are enabled.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Meshtastic connection page listed channels twice (the pre-existing
Observe-Channels selector plus a redundant read-only Index/Name/Role
table); remove the redundant table and its now-unused getChannels() /
MeshtasticChannel API helpers. On the MeshCore companion page, move the
Channels (name/hash/key) card up to sit right after the Status card,
above the Advertising settings, so it is no longer buried at the bottom.
No behavior change; layout/UX only.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the standalone Channels page and its nav entry (avoids menu bloat)
and relocate the channel listings onto the existing connection pages:
MeshCore channels (name, on-air hash, and the channel key/PSK with a copy
button) on the MeshCore companion page, and Meshtastic channels (index,
name, role) on the Meshtastic connection page. The MeshCore transport now
retains the channel_secret it already fetches at enumeration and
/api/meshcore/channels/detail returns it as `key` (hex PSK) so operators
can provision companion radios. known_channels() and /api/meshcore/channels
are unchanged.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a read-only Channels page listing both transports' channels with the
correct identifier per transport: Meshtastic by channel index (index,
name, role) from /api/channels, and MeshCore by channel name plus the
on-air hash from a new /api/meshcore/channels/detail endpoint. The
MeshCore transport now retains the channel_hash it already fetches at
enumeration (previously discarded); known_channels() and the existing
/api/meshcore/channels endpoint are unchanged. Adds a Channels nav entry
and /channels route.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Split the single region_routes.enabled master switch into per-transport
mt_enabled (Meshtastic) and mc_enabled (MeshCore) flags so the two
transports can be region-routed independently. Previously the shared
switch forced MeshCore into the region matrix; with all mc cells null it
routed MeshCore nowhere instead of falling through to the toggle-level
meshcore_channel. The dispatcher is now authoritative per-transport: a
disabled transport falls through to its toggle path, and matched-but-
inactive cells still suppress the toggle for enabled transports. The
destinations delivery branch also honors matrix-handled suppression to
prevent double-broadcast. Loader maps legacy enabled:true to
mt_enabled:true, mc_enabled:false. Each GUI routing page gains its own
master enable toggle.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends TEXT_HINTS with three missing legacy prefixes so NULL-source audit rows
render the correct family label in the Activity Log UI:
- ⏳ (U+23F3) → Weather (legacy composer weather-watch: "⏳ WX: …")
- 🌡️ → Weather (NWS heat-alert format added in PR #96)
- ⚠️ Road Incident (U+26A0+FE0F) → Traffic (legacy incident_handler rows)
Placed more-specific prefix before shorter variants to avoid false matches.
DB backfill: 0 residual NULL rows found (all rows already tagged after PR #99);
no audit-table writes performed. No mesh transmission.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend: add _SOURCE_TO_TABLE class constant in Dispatcher mapping event.source
("nws", "nifc", "wzdx", "traffic", "511") to canonical audit table names.
_post_broadcast_commit now falls back to this map when _broadcast_audit is
absent/None, so native env adapter sends (nws.py, fires.py, wzdx.py,
roads511.py, traffic.py) write a non-NULL source_event_table instead of NULL.
Existing _broadcast_audit paths (Central handlers, scheduled broadcasts) are
unchanged.
Frontend: replace naive familyLabel() string transform with explicit
TABLE_LABELS lookup (10 known tables → friendly names) plus a TEXT_HINTS
emoji-prefix heuristic for legacy NULL-source rows, so historical orphan rows
still display a meaningful label before/after backfill.
Tests: three new unit tests in test_dispatcher_persistence.py covering the
fallback path (nws→nws_alerts), the full _SOURCE_TO_TABLE map, and that
explicit _broadcast_audit is never overridden by the fallback.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Add WFIGS_Incident_Locations_Current point layer (IRWIN superset, ~6 ID fires)
alongside the existing perimeter layer (~2 ID fires). Fires are merged by IrwinID:
point layer is the authoritative superset, perimeter layer supplies polygon
geometry and validated acreage when available. Point-only fires surface with
lat/lon from the point geometry and no polygon. cold-start silent-seed path is
unchanged (first-poll batch is always silent regardless of source). Perimeter
fetch failure falls back to perimeter-only stubs; point fetch failure falls back
to perimeter-only; both failing bumps the consecutive error counter. county is
now populated from POOCounty on the point layer. Five off-air unit tests cover:
T1 merge-dedup, T2 point-only-new, T3 perimeter-geom-preferred, T4 cold-start
silent-seed 6 fires, T5 FIRMS _get_known_fires attribution.
No changes to store.py, gating/fire.py, schema, or coverage.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
CHANGE 1 (store.py): after applying gate.data_patch into event.data, promote
_severity_override and category keys onto event.severity / event.category.
Previously, decider overrides (e.g. fire: "priority" on every New/Update)
landed only in event.data, leaving event.severity at the adapter's raw value
("routine" for fires >=25 km from an anchor). This silently failed the
toggle/matrix min_severity floor. Native and Central now share identical
broadcast decisions at the shared choke point.
CHANGE 2 (dispatcher.py + v27 migration): both the toggle-path and
matrix per-cell severity-floor drop paths now emit a WARN log and
increment a new persisted counter (severity_floor_dropped) in
dispatcher_state, following the existing drop-counter pattern exactly.
v27.sql adds the column; the counter restores on restart and appears
in dispatch_stats().
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
weather_watch and weather_advisory were falling through to the legacy
Mode-B renderer (composer.py), which appended the raw expiry epoch
(`exp 1783615500.0` from _context_segment) and overflowed 140 chars,
splitting into two packets with the epoch stranded in packet 2.
Fix: register both categories with the NWS formatter and gating decider
(same path as weather_warning/weather_statement), and add them to
MESHAI_CUTOVER_CATEGORIES in docker-compose.yml so the pipeline routes
them to the new formatter on deploy.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Defect A (audit gap): _post_broadcast_commit only wrote a mesh_broadcasts_out row
when event.data["_broadcast_audit"] was a dict. Native traffic/weather/roads events
from native adapters never set that key, so every matrix-dispatched send was
invisible in the audit table even when the dispatcher logged success. Fix: write
the audit row for every mesh delivery attempt (ch_type in _MESH_CH_TYPES),
unconditionally. source_event_table/source_event_pk come from _broadcast_audit when
present, else NULL (best-effort). The early-return on empty data is preserved only
for the _on_broadcast_committed callback, not for the audit write.
Defect B (channel routing): full trace of the send path confirms the channel IS
correctly threaded from the matrix cell through _toggle_to_rule (mt_override) →
create_channel(channel_index=rule.broadcast_channel) → MeshBroadcastChannel
(self._channel) → send_message_async(channel=self._channel) → CompositeTransport
→ MeshtasticTransport send_queue job closure → _blocking_mt_send(channel) →
sendText(channelIndex=channel). No code bug: the correct channel index reaches the
radio. The missing audit rows (Defect A) prevented confirming this from the DB.
/api/channels fix: the endpoint read connector._interface which does not exist on
CompositeTransport (only on bare MeshtasticTransport), so it always returned []
when MeshCore was also configured. Fix: detect CompositeTransport and route to
meshtastic_child()._interface instead.
Tests added: matrix send without _broadcast_audit writes audit row with correct
channel+transport+success; failed delivery writes success=0 row; matrix cell
channel index reaches _blocking_mt_send end-to-end (queue path exercised);
/api/channels returns real channel list via CompositeTransport.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(transport): per-radio serialized+paced outbound send queue
Prevents simultaneous LoRa transmissions when N events arrive at once.
## Mechanism
Two `RadioSendQueue` instances (one MT, one MC), each a FIFO asyncio.Queue
with a long-running drain task. The MT queue drains on the main asyncio
loop; the MC queue drains on MeshCore's dedicated event-loop thread.
- MT sends: `run_in_executor` offloads the blocking `sendText` call;
queue started in `set_message_callback`, cancelled in `disconnect`.
- MC sends: drain loop runs pure-async MC lib coroutines directly on the
MC loop (no `_run_coro` deadlock); cross-loop callers bridge via
`concurrent.futures.Future` + `asyncio.wrap_future`.
- Pacing: `await asyncio.sleep(pacing_seconds)` between items; read live
from config per iteration; floor clamped to 0.25 s.
- Config knobs: `meshtastic_send_pacing_seconds` (default 2.0) and
`meshcore_send_pacing_seconds` (default 2.0) on `ConnectionConfig`.
## Send sites rerouted
All callers now `await connector.send_message_async(...)`:
- `notifications/channels.py` — MeshBroadcast/MeshCoreBroadcast/MeshDM/
MeshCoreDM deliver(), test_connection(), deliver_test()
- `responder.py` — DM replies in send_response()
- `transport/meshcore_transport.py` — periodic_advert_loop, telemetry
poll loop, send_advert() → send_advert_async(), req_telemetry()
→ req_telemetry_async() (all queue-routed from main loop)
- `dashboard/api/mesh_send_routes.py` — test-send, advert, telemetry poll
## Audit accuracy
`deliver()` now returns the actual bool from the radio send (not
optimistic True), so `mesh_broadcasts_out` reflects the real result.
## Tests
17 new tests in tests/test_send_queue.py covering FIFO ordering, no drops,
pacing gap, pacing floor enforcement, event-loop non-blocking, serialization,
lifecycle, MT fallback, config round-trip. Existing test stubs updated to
wire `send_message_async = AsyncMock(side_effect=send_message)` so prior
call_count / call_args assertions remain valid without changes.
Full suite: 2135 passed, 17 pre-existing failures (unchanged), 0 new regressions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(send-queue): resolve MC telemetry self-deadlock + resolve pending futures on teardown/reconnect; composite MC-channel kwarg; audit no-op false
BLOCKER 1 — req_telemetry_async self-deadlock (meshcore_transport.py):
_req_telemetry_async was calling _enqueue_mc_loop_send inside itself;
when _telem_job_outer ran inside the drain it nested another enqueue+await
on the same single-threaded drain — permanent deadlock on first telemetry poll.
Fix: _req_telemetry_async is now fully inline (no _enqueue_mc_loop_send).
_telemetry_poll_loop wraps its call in _enqueue_mc_loop_send for serialization.
req_telemetry_async's outer job calls _req_telemetry_async inline (safe).
BLOCKER 2 — pending futures abandoned on teardown/reconnect:
RadioSendQueue.stop() only cancelled the drain task; queue-sitting items had
their concurrent.futures.Futures left unresolved, causing wrap_future() callers
to hang indefinitely. Fix: stop() drains the remaining queue with get_nowait()
and cancels every pending cfut. _cancel_mc_queue() schedules the same drain-
and-cancel via call_soon_threadsafe. _start_mc_queue() cancels old drain task
and drains old queue cfuts before arming the new queue (reconnect path).
connector.disconnect() now .result(timeout=5) on stop() instead of fire-and-forget.
SHOULD-FIX 3 — composite passes MC channel as wrong kwarg (composite_transport.py):
_broadcast_async no-hint loop was calling send_message_async(channel=child_channel)
for the meshcore child; should be meshcore_channel=child_channel. Silent drop fixed.
NIT 5 — false success on zero-channel MC send (meshcore_transport.py):
send_message_async returned True when meshcore_channel is None (nothing sent).
Now returns False so audit does not record a success for a no-op.
NIT 7 — config comment contradiction (config.py):
meshtastic_send_pacing_seconds comment said "0 disables the floor" while
simultaneously stating "still floored at 0.25". Removed the contradiction.
Regression tests (tests/test_send_queue.py — 3 new, all in TestDeadlockRegression):
- test_telemetry_queue_no_deadlock: drives req_telemetry_async through a real
_mc_send_queue with fake MC commands; times out on pre-fix code (deadlock).
- test_teardown_resolves_pending_futures: enqueues slow+fast jobs, stops mid-drain,
asserts every task resolves promptly; hangs on pre-fix code.
- test_reconnect_resolves_old_futures: calls _start_mc_queue twice, asserts old
cfuts are cancelled; pre-fix leaves them unresolved.
All 17 pre-existing send-queue tests still pass (20 total now).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(meshcore): opt-in telemetry auto-poll on selected contacts
req_telemetry + a poller for selected contacts (meshcore_telemetry_contacts,
interval with a min floor, availability detection). Contacts page gains
per-node auto-poll toggles + battery/sensor readouts + Poll-now, and maps
numeric contact type codes to Chat/Repeater/Room/Sensor badges.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(meshcore-telemetry): reconcile with current transport/meshcore-lib API
- Add EventType.ACK + NEW_CONTACT to the telemetry test's fake module;
_setup_subscriptions() subscribes to both (added in main before rebase)
and the stale stub caused all three TestPollerScheduler tests to abort
with AttributeError on connect().
- Same NEW_CONTACT gap fixed in test_meshcore_conn_type.py and
test_meshcore_dm_delivery.py — these ran first (alphabetically) via
setdefault, contaminating the shared sys.modules["meshcore"] stub for
all downstream test files and causing 9 extra connect()-path failures
suite-wide (TestPeriodicAdvertScheduler, TestAdvertOnConnect, etc.).
- req_telemetry_sync(contact, min_timeout=5) matches the installed lib
(meshcore-2.3.7 binary.py) exactly — no production-code change needed.
- All 30 test_meshcore_telemetry tests pass; full-suite failures drop
16 → 7 (remaining 7 are pre-existing, unrelated to telemetry).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
