mirror of
https://github.com/zvx-echo6/meshai.git
synced 2026-08-26 17:31:34 +00:00
526 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Matt Johnson
|
bc2e487141 |
chore(dashboard): untrack committed frontend build output
work/meshai/dashboard/static/ held committed vite build artifacts
(index.html + hashed assets/*.js|css + copied public/ images) that were
last built 2026-07-07 (PR #87), 13 frontend commits ago. Docker builds
the frontend fresh and overwrites these on every image build, but the
pip install -e . path (documented in README Quick start) ships this
stale directory verbatim via [tool.setuptools.packages.find] include.
The root .gitignore already had a rule for this
("meshai/dashboard/static/") but it silently stopped matching when
|
||
|
5685ed034f |
feat(swpc): native solar_radiation_storm (S-scale) formatter + decider + pinned goldens (#145)
Co-authored-by: Matt Johnson <mj@k7zvx.com> |
|||
|
e9daabfbff |
fix(satpass): default feed_source to native; correct misleading config comments (#139)
Central is retired -- its NATS broker (nats://central.echo6.mesh:4222) no longer exists -- but SatpassConfig.feed_source still defaulted to "central", overriding the _SourcedFeed mixin default of "native". A fresh install that enabled satpass got a silently dead adapter. The dashboard reinforced it: Environment.tsx seeded feed_source 'central' and labelled the adapter "via Central". Also corrects comments that documented the opposite of the code: the adapter_config keys usgs_quake.global_mag_floor / regional_mag_floor were labelled "CENTRAL-PATH ONLY", but notifications/gating/quake.py reads them unconditionally in the native path. Following those comments would have led someone to delete live config keys. - config.py: SatpassConfig.feed_source "central" -> "native" + docstring - adapter_config/defaults.py: correct the two "CENTRAL-PATH ONLY" comments - Environment.tsx: seed 'native'; reword the satpass subtitle - tests: assert the native default; pin central explicitly where a test exercises the central path or the feed_source flip Suite: 2337 passed, 6 pre-existing failures (stale SCHEMA_VERSION x3, expired TLE fixtures x2, one order-dependent), no new failures. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
e9153943bf |
chore(wzdx): remove two dead config fields (#150)
Both were fully plumbed and read by nothing.
api_key -- self-documented as dead at config.py ("Keyless: api_key is
retained but unused"), yet wired end-to-end: a GUI ManagedSecret field, a
SECRET_FIELDS entry, an EXPECTED_SECRETS entry, a secrets_store mapping and
label, and a line in .env.example. env/wzdx.py assigned self._api_key and
never read it again. So an operator could go get an API key, paste it into
the secure secrets manager, and have it do precisely nothing -- the ritual
looked complete end to end, which is what made it worth removing rather
than leaving.
endpoints -- default ["/get/event"], exposed as an editable list in the
dashboard, never read. Copy-paste from Roads511Config.endpoints (which IS
read, at env/roads511.py; Roads511 is untouched here). WZDx discovers feeds
via the FHWA registry_url/states instead.
WZDx's actual fetch behavior is unchanged; this removes dead config only.
Note for existing installs: anyone with WZDX_API_KEY set in
/data/secrets/.env will simply have an ignored env var. Harmless -- it was
already ignored.
Suite: 2337 passed, 6 failed (the pre-existing set: stale SCHEMA_VERSION x3,
expired TLE fixtures x2, one order-dependent), 72 skipped -- exact baseline
match, no new failures.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
3c900091d7 |
docs: point onboarding at the dashboard; drop 3 phantom keys from the seeded default (#147)
* fix(entrypoint): drop three phantom history keys from the seeded default
The config written on first boot -- what EVERY fresh Docker install starts
from -- seeded three keys that do not exist on HistoryConfig and are
silently discarded on load:
auto_cleanup: true
cleanup_interval_hours: 24
max_age_days: 30
HistoryConfig has only `database`, `max_messages_per_user`, and
`conversation_timeout`.
To be precise about the impact: history cleanup DOES work -- cleanup_expired()
is wired at main.py:237 and _prune_history() honours max_messages_per_user.
What never existed is the time-based retention model these keys describe (a
30-day age cutoff on a 24h interval). An operator setting max_age_days: 90
expecting 90-day retention was silently ignored.
Not implemented here -- whether time-based retention should exist is a
product decision, not a cleanup. This only stops the default config
promising it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: point onboarding at the dashboard, not the legacy example config
The documented setup path was `cp config.example.yaml config.yaml`, but that
file is ~40% incomplete: of the 20 top-level Config fields it covers 15,
omitting coverage (the universal bbox), danger_zones, generic_sources,
meshcore_context, commands, and the current notification model
(toggles/destinations/region_routes). Its own notifications header still says
the schema "will be replaced in v0.3 by the 8-toggle model" -- which shipped
long ago. Anyone following the project's own instructions landed on a
degraded surface with no signal a richer config existed.
It is also read by NOTHING at runtime: docker-entrypoint.sh sets
MESHAI_CONFIG=/data/config.yaml and writes its own inline default on first
boot. The Dockerfile still COPYs config.example.yaml into the image, so it is
kept and now labelled reference-only rather than a starting point.
- README: Docker quick-start seeds itself; configure via the dashboard. The
pip path still uses config.example.yaml (nothing seeds one there) but now
carries an honest note that it is a minimal bootstrap, not a reference.
Adds an Advanced section for the split /data/config/ layout and the
migrate_config_v03 path into it.
- docker-compose.yml: the comment claimed config lives at /data/config.yaml
as though that were the only layout; corrected to describe both, and note
secrets live in /data/secrets/.env.
- Dockerfile: document why config.example.yaml is still shipped.
Not done deliberately: config.example.yaml is NOT expanded to cover all 20
sections. A second hand-maintained schema is what caused this drift; the
dashboard is the authoritative surface. The legacy single-file loader is
untouched and still fully supported.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(readme): move dashboard-first onboarding rewrite to root README
work/README.md is about to become a symlink to root README.md on
docs/single-readme. Retarget the Docker/dashboard-first quick start
and config.example.yaml honesty note added in
|
|||
|
12e7335add |
fix(dashboard): hard-disable the Central feed_source option (retired backend) (#154)
Central's NATS broker is gone and its database was dropped 2026-07-15, but the per-adapter Native/Central toggle on the Environment page still let an operator pick "central" for 11 of 12 adapters and have it save successfully (config.py's validator still accepts central|native; env/store.py's gate just silently skips registration when feed_source != "native"). The result: flip the switch, save cleanly, feed dies with no error anywhere. This is a minimal stopgap, not the real fix. A separate refactor will strip feed_source/hasCentral/nativeOnly and the whole toggle out once Central's removal is complete; that's out of scope here and backend behavior (config validator, store gate) is intentionally untouched so existing YAML with feed_source: central doesn't break. For now: hard-disable the Central button in FeedSourceToggle regardless of hasCentral/nativeOnly (kept referenced, not deleted, for the pending refactor), and make the accompanying copy tell the truth — "Central feed source has been retired — native only" — instead of the old adapter-scoped "not available for this adapter" text that no longer describes what's happening. Verified: GaugeSites.tsx only reads usgs.feed_source to gate its lookup button, never writes it — not a second instance of this trap. tsc --noEmit and the pytest suite show zero new failures vs. pre-edit baseline. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
2543dfa8d1 |
fix(config): warn on unknown config keys; delete 7 phantom keys from the example (#146)
* fix(config): warn on unknown config keys instead of silently dropping them
_dict_to_dataclass() silently continue'd past any key not in the target
dataclass's field set -- an operator could set a config key, restart, and
have it vanish with zero feedback (config.example.yaml's phantom
mesh_intelligence keys are exactly this bug, fixed separately).
Now logs a WARNING naming the key and the dataclass, hinting at a typo or
a renamed/removed field, via the module's existing _config_logger.
Traced every dynamic/free-form config path to rule out false positives:
notifications.toggles, notifications.destinations, generic_sources,
mesh_sources, and notifications.rules all route through explicit
dict-of-dataclass or verbatim-passthrough handling and never spuriously
warn. Two legitimate legacy shapes DO hit the strict field-check path with
keys that were never (and will never be) dataclass fields:
- notifications.channels (pre-v0.5 channel list), consumed directly from
the raw dict by _migrate_legacy_channels
- notifications.region_routes.enabled (pre mt/mc-split master switch),
read directly by the explicit region_routes handler
Both are allowlisted in _KNOWN_LEGACY_DROP_KEYS so users mid-migration
don't get spurious noise on every load.
Added tests/test_config_loader.py coverage: unknown key warns and does
not raise, both legacy shapes stay silent, and the free-form/dynamic
sections never warn for keys valid on their real target shape.
* fix(config): remove phantom config.example.yaml keys, add missing live ones
The new unknown-key warning (previous commit) caught config.example.yaml
loading with SEVEN warnings, all real drift -- none were false positives
of the warning itself:
mesh_intelligence block shipped three keys with no MeshIntelligenceConfig
field and no implementation anywhere (git log -S confirms they were never
built, not leftovers from a removal):
- region_radius_miles, infra_overrides, region_labels
These only made sense under an older auto-clustering design; what
actually exists is explicit region anchors (regions: list[RegionAnchor]).
Deleted from both the live block and the commented-out example above it,
and added the four fields that DO exist and are live but were missing
from the example: regions, critical_nodes, alert_channel, alert_rules.
Also fixed the now-misleading comment at
dashboard/api/mesh_routes.py:281, which referenced region_labels --
comment only, no code change.
notifications block shipped a whole quiet-hours subsystem that was
deliberately ripped out (commit
|
|||
|
b0d1a76e70 |
feat(dashboard): make the gauge-sites bulk import reachable (#149)
* fix(dashboard): register gauge_sites_import router The gauge-sites bulk-import endpoint (CSV / NWS-AHPS) was fully built and tested but never wired into the app -- server.py never called include_router() for it, so POST /api/gauge-sites/import 404'd in production. Only test_tail_followups.py exercised it, via a raw TestClient bypassing the real app. * test(dashboard): assert gauge_sites import route is reachable via real app Guards against the router silently going unregistered again: builds the actual dashboard app via create_app() and drives a real CSV import through it end-to-end (POST + GET round-trip), instead of only hitting the router in isolation. * feat(dashboard): add gauge-sites bulk import UI Wires a UI onto the now-registered POST /api/gauge-sites/import endpoint, inside the existing GaugeSites tab (no new nav entries/pages). Adds an Import toggle next to Add site, with CSV (paste or file-load into a textarea) and NWS-AHPS (WFO code list) modes. Required/optional CSV columns are documented inline so an operator isn't guessing. Result counts (inserted/updated/skipped/detail_fetched) and any partial-failure errors from the AHPS scrape are always shown; a pending spinner covers the AHPS path's live water.weather.gov calls so a slow response never reads as "imported 0 sites". The list refreshes only after a successful import that actually changed rows. --------- Co-authored-by: Matt Johnson <mj@k7zvx.com> |
|||
|
89a46d520a |
fix(hydro): restore USGS stream-gauge flood alerts (silently dead since the all-native flip) (#156)
* fix(hydro): make native USGS gauge flood alerts renderable env/usgs.py emits stream_flood_warning / stream_high_water Events for elevated stream gauges, but neither category had a registered gating decider or formatter, and event.data was left empty -- so every detected flood/high-water reading was silently dropped before it ever reached the mesh (get_decider/get_formatter both returned None, and compose_mesh_message fell through with nothing to render). - Register the existing hydro.decide()/hydro.format() (already used by the Central-only `stream_flow` category) under stream_flood_warning and stream_high_water too -- same shared-decider/formatter pattern already used for avalanche_warning/_watch, weather_*, wildfire_*, and emergency_*. - Add both categories to cutover.NATIVE_ALWAYS_DECIDE so the decider and formatter actually run unconditionally (mirrors the native WFIGS fire categories): Central never emits these two category strings, so there is no shadow-bake window to wait out. - env/usgs.py's to_event() now populates event.data with the canonical hydro schema (site_id, gauge_name, stage_ft, flow_cfs, unit, threshold_state, reading_time, lat, lon, parameter_code) the shared gate/formatter expect, mapping the adapter's flood_status strings onto the ranked threshold_state vocabulary. - gating/hydro.py's decide() now also OWNS an unconditional gauge_readings INSERT for the native source (source != "nwis"): the table had no writer since Central's nwis_handler stopped running (2026-07-05), so every native prior-state lookup returned "normal" forever and every elevated reading would have rebroadcast on every 15-minute tick instead of once per crossing. The Central source keeps its own inline handler-owned INSERT unchanged (decide() stays read-only for source="nwis"). Left as-is (not this fix): the toggle mapping for these categories (get_toggle() -> "seismic") is unchanged. It already matches the sibling `stream_flow` category and is enforced by test_water_v057.py::test_existing_hydro_entries_unchanged / test_water_categories_have_required_fields; there is no separate water/flood toggle family in VALID_TOGGLES, and inventing one is a config-surface change outside this fix's scope. Known limitation documented in gating/hydro.py: since to_event() only ever emits elevated readings (never a "back to normal" reading), a full recede-to-normal followed by a later re-crossing into the same tier will not re-broadcast until a higher tier is reached -- degrades toward silence, not spam. * test(hydro): cover native gauge flood-alert registration, gating, and wire format Registration tests prove stream_flood_warning/stream_high_water now resolve a decider AND a formatter (the thing that was broken), that neither resolves to the earthquake decider despite sharing the "seismic" toggle name, and that both are in NATIVE_ALWAYS_DECIDE. Gate tests drive env/usgs.py's to_event() through the real gating.hydro.decide(): a first elevated reading broadcasts (graceful no-prior-data handling), a sustained same-band reading suppresses, an escalation broadcasts again, and a routine reading never reaches the gate at all (unchanged pre-fix adapter behavior). A dedicated test confirms the new native-persistence write in decide() does not leak into the Central source="nwis" path. Golden formatter tests pin the wire string for a high-water and two flood-warning tiers, plus the missing-coords drop case -- all rendered through the same formatters.hydro.format() the Central `stream_flow` path uses (test_hydro_refactor.py already proves that formatter is byte-identical to the old central.nwis_handler._render()). Native events never carry flow_cfs (to_event() only ever emits stage/height readings), so that segment's absence is captured explicitly as current behavior, not ported from Central. An end-to-end test drives to_event() -> decider -> compose_mesh_message to prove the NATIVE_ALWAYS_DECIDE gate takes effect for the actual mesh render path, not just formatter/decider resolution in isolation. --------- Co-authored-by: Matt Johnson <mj@k7zvx.com> |
|||
|
d7913fddaa |
feat(dashboard): serve real recommendations in Nodes & Health; give mesh_intelligence one home (#148)
* refactor(mesh_reporter): expose recommendations as list[str]
Add recommendations_list(scope, scope_value) as the canonical source of
recommendation text. build_recommendations() now just joins that list with
the historical "OPTIMIZATION RECOMMENDATIONS:" header/bullet format it
always used.
router.py:1145 injects build_recommendations()'s output into the LLM system
prompt on the live mesh-DM path — verified byte-identical before/after
across mesh/region/node/missing/empty scopes via a synthetic fixture, and
pinned with a literal-string regression test so a future refactor can't
silently change that prompt text.
This unblocks wiring recommendations into the dashboard, which previously
only reached mesh DMs.
* feat(dashboard): serve real recommendations from /api/health
mesh_reporter was never on app.state — add it alongside health_engine etc.
in server.py's existing pattern. mesh_routes.py's health endpoint now
returns mesh_reporter.recommendations_list("mesh") instead of a hardcoded
[] TODO stub.
Add recommendations_available: bool alongside recommendations: string[] so
"the engine ran and found nothing" (healthy mesh) is never indistinguishable
from "the engine couldn't run" (unwired reporter, or an exception — logged
via logger.exception and swallowed so the rest of the health response still
serves). A crashed recommendations engine must not read as "mesh is
healthy" to an operator.
Also delete main.py's phantom `getattr(mh, "recommendations", [])` on the
websocket health_update push: nothing ever set .recommendations on the
mesh_health object (always []), and no frontend consumer reads it — the
REST endpoint above is the supported path.
* feat(dashboard-frontend): render recommendations in Nodes & Health
Meshtastic → Nodes & Health → Health tab used to be a duplicate
mesh_intelligence config editor (the same MeshIntelligenceSection also
edited from Settings → Intelligence, a stale-tab-overwrite hazard). Replace
it with a real operational view: fetch /api/health and render its
recommendations list.
Three states, matching visual conventions from Dashboard.tsx's alerts list
and MeshCoreRouting.tsx's cross-link note:
- recommendations present → Lightbulb list, one card per item
- empty + recommendations_available → "No recommendations — mesh is
healthy" (CheckCircle)
- recommendations_available === false → amber warning, not the healthy
state (AlertTriangle) — a crashed backend must not look healthy
mesh_intelligence config now has exactly one home: Settings. The Health tab
points there via the existing /config?section= deep-link pattern instead of
duplicating the editor. Config.tsx itself is untouched.
MeshHealth.recommendations/.recommendations_available are optional in the
type: the websocket health_update push doesn't include them (REST-only).
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
|
|||
|
ec973b606a |
chore(dashboard): remove orphaned route, redirect 5 legacy routes, fix a misleading docstring (#143)
* chore(dashboard): remove orphaned /adapter-config route
AdapterConfig is already embedded directly in Environment.tsx
(excludeKeys={CURATED_KEYS} hideLlmToggle); no in-app <Link> ever
targeted the standalone /adapter-config route. Drop the route, its
App.tsx import, the extraTitleItems entry in Layout.tsx, and the
now-unused Sliders icon import. pages/AdapterConfig.tsx itself is
kept — it's imported directly by Environment.tsx.
* chore(dashboard): redirect 5 orphaned legacy routes to their tabbed homes
/gauge-sites, /town-anchors, /mesh, /meshtastic/sources, and
/meshcore/companion each duplicate content now tabbed inside Places,
MeshtasticNodes, and MeshCoreContactsCompanion, and have no in-app
<Link> pointing at them (verified with rg). Convert them to
<Navigate replace> redirects, matching the existing /notifications
and /data-sources precedent.
/town-anchors, /meshtastic/sources, and /meshcore/companion land on
their host's first tab rather than deep-linking to the second tab
(no ?tab= support exists) — same accepted trade-off as the existing
/notifications and /data-sources redirects.
Drop the now-dead extraTitleItems entries and the redundant
'/meshcore/companion' pathTitles entry, plus the Radio, Droplets,
Layers, and Bot icon imports that were only used by those entries
(MapPin is kept — still used by the /places nav item). These 5
routes were never nav entries (extraTitleItems only), so the sidebar
stays at 17 items with 5 Meshtastic / 5 MeshCore — MT/MC symmetry
unchanged.
* docs(dashboard-api): fix misleading refresh_toggles docstring
The docstring claimed this endpoint was "kept for backwards-compat
with the dashboard's manual ping path," but rg turns up no frontend
call site for /notifications/refresh-toggles anywhere in
dashboard-frontend. The auto-refresh middleware
(_auto_refresh_toggle_filter, registered via
register_config_routes_hooks) already refreshes the ToggleFilter on
every successful notifications config PUT, covering the normal case.
Endpoint is untouched -- comment only, kept for ops/debug use.
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
|
|||
|
391a6316ad |
chore: remove dead modules and orphaned fixtures (-2,952 LOC) (#142)
* chore: remove dead mesh_sources.py module
Superseded by mesh_data_store.py, whose docstring states it "replaces
mesh_sources.py with a clean three-layer architecture." No remaining
inbound imports (absolute or relative) repo-wide; the only surviving
references to "mesh_sources" are to the unrelated config YAML section
of the same name, which is untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove dead region_tagger.py module
Only reference to "region_tagger" repo-wide was inside its own
module docstring usage example; no live imports found.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove empty meshai/cli package
Package contained only a docstring, no code. No imports of
meshai.cli found anywhere; console entry point is meshai.main:main
(pyproject.toml), and packaging (tool.setuptools.packages.find,
include = ["meshai*"]) has no explicit reference to the cli
subpackage. No MANIFEST.in/setup.py/setup.cfg exist to update.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove orphaned swpc/swpc_last golden fixtures
63 files (40 in fixtures/swpc/, 23 in fixtures/swpc_last/) never
loaded by any test. harness/goldens.py's load_fixtures(hazard) is
the only generic fixture loader and is never called with "swpc" or
"swpc_last" (only "avalanche", "nws", "nws_last", "quake"). The sole
remaining textual reference, a provenance comment in
test_swpc_refactor.py:162 ("Fixture mirrors swpc_last/0003.json..."),
is left untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
f4b717f351 |
chore: untrack dashboard-frontend/node_modules (#141)
The repo's blanket `dist/` .gitignore rule (line 19, inherited from the
Python template) matches directories named `dist` at any depth with no
leading slash, so it silently excluded node_modules/<pkg>/dist/ for every
package whose build output lives there. 21 packages (including
react-router) declare main/module pointing into dist/, so the committed
node_modules was non-functional -- e.g. require.resolve('react-router')
threw MODULE_NOT_FOUND against the tracked copy.
work/Dockerfile already runs `npm ci && npm run build` in its build stage,
so production was never affected by this -- the committed copy was pure
dead weight (8,619 of 9,232 tracked files, 93% of the repo).
package-lock.json is tracked, making `npm ci` fully reproducible, so this
untracks node_modules/ (added a `node_modules/` ignore rule to .gitignore)
rather than trying to fix or narrow the dist/ rule.
History is NOT rewritten -- the old blobs remain reachable via prior
commits, so clone size is unchanged. This only stops new commits from
re-adding node_modules going forward.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
9f06930a0d |
chore: excise the dead Central NATS consumer path (-11,328 LOC) (#144)
* chore: excise the dead Central NATS consumer path
Central was retired and its database dropped 2026-07-15; its NATS broker no
longer exists. Verified against the live CT108 deployment: all 12 adapters
run feed_source=native, zero on central, and central.enabled is False
(default, never overridden). The consumer and its handlers were unreachable.
Removed:
- central/consumer.py and 6 dead handlers (nws, quake, swpc, nwis, avy,
incident) -- their handle_* entrypoints were reachable only from the
consumer's dispatch
- the Central wiring in main.py (init, guarded start, retry loop, stop path)
- the dead config surface: CentralConsumerConfig, EnvironmentalConfig.central,
adapter_config ("central","severity_thresholds") and its display block
- the nats-py dependency (consumer.py was its only importer)
- 19 test files that exercised only the dead path
KEPT -- these live under central/ but are imported directly by native
adapters, and deleting them would break production:
- wfigs_handler.py: firms_handler._handle_pass_boundary() calls its _render()
on the live FIRMS growth-fire path (env/firms.py -> ingest_hotspot_pixel)
- firms_handler, satpass_handler, tle_handler: split files whose handle_*
entrypoints are dead but whose engines are live. Left intact; splitting
them is separate work.
- pass_predictor, budget, idaho_gauge_sites: fully live.
The usgs_quake keys global_mag_floor / regional_mag_floor / regional_centroid
/ regional_radius_mi / broadcast_pager_alerts are NOT removed despite comments
labelling them "CENTRAL-PATH ONLY" -- notifications/gating/quake.py reads them
unconditionally in the native path. Those comments are corrected separately.
Test-count note: the suite drops ~425 tests. Most were migration PARITY tests
whose sole purpose was proving the native rewrite byte-matched the Central
handler (golden byte-parity, cross-source identity, gate-sequence replay).
With the handler deleted there is nothing left to compare against, so they
cannot exist. Native-only tests were kept and reworked where a test reached
for a central symbol incidentally. This is a real coverage loss, accepted
deliberately: the parity harness proved the refactor faithful, and git
history preserves the originals.
Suite: 1984 passed, 6 failed -- the same 6 pre-existing failures as main
(stale SCHEMA_VERSION x3, expired TLE fixtures x2, one order-dependent),
all being fixed on fix/green-test-suite. No new failures.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(nws): restore native-only golden coverage for the wire formatter
Commit
|
|||
|
16f01b29d4 |
fix(persistence): derive SCHEMA_VERSION from migrations; unbreak the red suite (#140)
* fix(persistence): derive SCHEMA_VERSION from the migrations directory db.py hardcoded SCHEMA_VERSION = 26 while migrations/ had already reached v29 (v27 dispatcher floor-drop counter, v28 mesh_observations, v29 IPAWS). The migration runner globs the directory and applies every vN.sql it finds regardless of the constant, so a fresh DB actually landed at 29 while the constant claimed 26 -- a three-version drift that three tests were correctly catching. Derive it from the highest vN.sql present instead of bumping the literal, so it cannot drift again the next time someone adds a migration. Falls back to 0 if the directory is missing so import never fails; the migrations dir sits alongside db.py and ships with the package (Dockerfile COPYs meshai/). Adds a regression guard asserting the constant matches the highest migration file, and updates three tests that hardcoded 26 as a literal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(tle): make TLE fixtures time-relative so they cannot expire The ISS fixtures hardcoded epochs of 2026-06-30/07-01/07-02. tle_handler sets STALE_DAYS = 14 and get_tle_by_norad() filters on epoch >= now - 14d, so the fixtures silently aged out on 2026-07-02 and the tests began failing -- a time bomb, not a regression. Compute epochs relative to wall-clock now (base = now - 2d, +/-1d for newer/older) with correct TLE epoch-field encoding and mod-10 checksum. STALE_DAYS is untouched -- widening it in product code would have changed production behavior to paper over a test bug. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(fire-tracker): pin config + history db to tmp_path load_config() defaults HistoryConfig.database to the relative path "conversations.db", resolved against the process CWD, so every test calling load_config() with no override shares one file for the whole session. The conftest DB-isolation fixture only covers MESHAI_DB_PATH, not this. Point both the config dir and the history database at the test's tmp_path. NOTE: this does NOT resolve the order-dependent failure -- the test still passes standalone and fails in a full run, so the polluting state lives somewhere other than config/history. Left failing rather than weakened; root cause still unidentified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
8460ab50e0 |
fix(ipaws): route county-only CAP alerts + auto-refresh toggles live (#159)
County-only civil CAP alerts (SAME geocode, no <polygon>) had no geometry, so the geometry-based region tagger could not place them: no region -> no region_routes match -> silently not broadcast. Many CEMs / 911 outages / some AMBER alerts are county-only. - Bundle work/meshai/county_centroids.py: Census 2023 national county gazetteer internal points (3,222 counties + DC/territories), FIPS->(lat,lon), with SAME PSSCCC -> 5-digit FIPS helpers. - env/ipaws.py: when a CAP alert has SAME geocode(s) but NO polygon, set a Point (single county) or MultiPoint (multi-county) geometry from the county centroid(s) so the EXISTING coverage/region tagger locates it and tags ALL matching regions. Real polygon geometry always wins (never overridden). - dashboard/server.py: call register_config_routes_hooks(app) in create_app so the toggle auto-refresh middleware is actually wired in prod (was test-only); saving a family toggle now takes effect live without POST /api/notifications/refresh-toggles. Tests: county-only alert tags SW Idaho + matches emergency route cell; multi-county tags all regions; polygon path unchanged; create_app wires the refresh middleware; panhandle coverage-gap documented. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
e244405230 |
feat(dashboard): bring ipaws adapter + emergency family to full GUI parity (#158)
The IPAWS civil-alert adapter shipped backend-complete but frontend-partial: it rendered only in the generic adapter-config page and the Advanced (raw) Data Feeds tab, and the `emergency` family was absent from the curated Data Feeds panel, the family-settings toggles, and the MeshCore routing matrix. Adapter (ipaws), benchmarked against firms/nws: - Environment.tsx: add `ipaws` to AdapterKey union, EnvConfig interface, META (native-only, keyless), a new `emergency` FAMILIES group, PANEL_META_KEY (LLM toggle), and a hand-written renderSettings panel exposing base_url, user_agent, tick_seconds, state_fips, same_codes, exclude_weather, status_actual_only — with coverage-scope handling like the other adapters. - Environment.tsx: IPAWS_DEFAULT backfill so pre-ipaws GET payloads don't crash. - Dashboard.tsx: SOURCE_ICONS entry (Siren/IPAWS) so ipaws events aren't a slug. - ActivityLog.tsx: TABLE_LABELS + CATEGORIES + text-hint so ipaws_alerts rows show labeled "Emergency" and honor the category filter. - dispatcher.py: _SOURCE_TO_TABLE fallback ipaws -> ipaws_alerts so region-routed emergency sends land labeled in the audit feed (not NULL). Family (emergency), benchmarked against fire: - Notifications.tsx: add `emergency` to TOGGLE_FAMILY_META (Siren icon). This cascades to Family Settings, the Meshtastic delivery matrix, and the MeshCore routing matrix (the last was hardcoded to the static list and previously omitted emergency entirely). Backend VALID_TOGGLES/gating/categories were already complete — no backend family change needed. Tests: update the _SOURCE_TO_TABLE exact-match guard and add an ipaws audit-row parity test. Full suite 2407 passed / 6 pre-existing unrelated failures. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
c04daa6e4d |
fix(config): merge partial PUT bodies instead of resetting to defaults (#155)
Saving the "Auto-advert interval" dropdown on the MeshCore Companion page
took BOTH radios offline on 2026-07-17 06:46:52. One click, full outage.
The page PUT a single-key body to /api/config/connection:
{"meshcore_advert_interval_seconds": 10800}
_dict_to_dataclass() builds kwargs only from the keys present in the body
and lets `cls(**kwargs)` default the rest, so every OMITTED field was reset
to its dataclass default and written to disk:
type: tcp -> serial (Meshtastic offline)
tcp_host: 192.168.1.100 -> <lost> (LOCAL_FIELDS, see below)
tcp_port: 4404 -> 4403 (wrong meshmonitor vnode)
meshcore_host: 192.168.1.253 -> '' (MeshCore off; blank = off)
meshcore_conn_type: serial -> tcp (wrong transport)
meshcore_serial_port: /dev/meshcore-rak -> '' (RAK radio lost)
It was silent twice over. `connection` is restart-required, so the running
process kept the good in-memory config while the file sat gutted, waiting
for any restart to detonate. And save_section() writes the domain file
FIRST and local.yaml SECOND: meshtastic.yaml hit the disk already gutted,
then the local.yaml write (which owns connection.tcp_host via LOCAL_FIELDS)
died on `[Errno 13] Permission denied` -- so tcp_host landed in neither
file, and the 500 that would have named the cause was swallowed by the UI.
The operator saw nothing happen.
This was never one page's bug: PUT /api/config/{section} was destructive on
a partial payload for EVERY section. Other callers only survive because they
happen to spread the full object first.
Fixes, in depth:
* Route (the durable fix): merge the body over the CURRENT live section
before coercing, so omitted keys keep their live values while present
keys -- including '' / False / [] -- still apply. The base is the live
config, the same values GET serves, so a partial PUT now lands exactly
where a full-object PUT from that same GET would. Full-object callers are
unaffected. Fixed at the HTTP boundary, not in _dict_to_dataclass():
absent-key-means-default is CORRECT at config-load time, where a file
legitimately omits fields it does not override.
* Nested semantics keyed off the dataclass schema, not "is it a dict":
nested dataclass fields DEEP-MERGE (a partial region_routes must not drop
sibling cells), while bare dict/list fields REPLACE at the key (cells,
toggles, destinations, rules are dynamic maps -- deep-merging them would
resurrect deleted keys and make deletion impossible, the mirror image of
the bug being fixed).
* Page: send the full connection object like every other caller does.
* Errors are visible: the save handler no longer swallows the exception,
and updateConfig() surfaces the server's `detail` rather than a bare
"API error: 500", which is what hid Permission denied from the operator.
* Default advert interval 10800 -> 86400 (24h). 3h is far too frequent a
default for a public mesh; the UI "(default)" label moves to match.
Tests: tests/test_config_partial_save_merge.py reproduces the outage with
the exact payload, and pins merge semantics across connection AND
notifications, intentional clearing, deep-merge, and map-deletion.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
ea4c010967 |
feat(meshcore): report the true connection + add roster/channel management (#153)
self_info() reported host/port straight from config regardless of
conn_type, so a serial companion still advertised whatever stale
meshcore_host sat in the config — the API named a device meshai was not
talking to, which is enough to send an investigation to the wrong radio.
Connection details now come from one _connection_descriptor() shared with
connect(), so the log line and the API can't drift; only the live
conn_type's fields are populated and the rest are null.
meshai's device view is otherwise built once at connect and never re-read
— contacts via ensure_contacts(), channels via _enumerate_channels(). The
lib's contact handler only ever merges (meshcore.py::_update_contacts), so
a cached roster can never shrink, and a channel provisioned on the radio
stays invisible until the process restarts. There was no refetch path at
all. Adds an explicit resync that re-reads BOTH halves: a FULL
get_contacts(lastmod=0) reconciled with replace semantics (absent contacts
are dropped) plus a channel re-enumeration, each reporting what changed.
Also adds a preventive route-health check: every region_routes cell whose
MeshCore target cannot be resolved against the live roster/channel table
is surfaced, since such a send fails silently. Room targets are matched by
pubkey prefix, exactly as the dispatcher resolves them, so a picker-stored
prefix is not misreported as dangling. Same-name/different-pubkey roster
entries are flagged too — a name alone cannot identify a contact, which is
the trap behind a room rebuilt under a new keypair.
Backend:
- meshcore_roster.py: pure reconcile_contacts / check_route_health /
find_name_collisions (no device I/O — unit-testable without a radio)
- transport: _connection_descriptor, resync, refresh_contacts,
remove_contact, import_contact, export_roster, contacts_synced_at;
auto_update_contacts enabled (configurable — it costs one incremental
fetch per advert heard, which is real chatter on a dense mesh)
- API: POST contacts/refresh, DELETE contacts/{pubkey}, GET
contacts/export, POST contacts/import, GET route-health
Frontend (existing Contacts & Companion page — no new page or nav entry):
- dangling-route + name-collision banners; resync/export/add-contact
toolbar with last-synced and the added/removed counts; staleness badges;
search, filters and sortable columns; per-contact delete behind a
confirm; Companion tab shows the real transport + target.
A full pubkey is required to delete or add: the lib resolves by prefix,
and a prefix could silently hit the wrong node.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
bbe97398bc |
feat(ipaws): add FEMA IPAWS-OPEN civil-alert adapter (disabled by default) (#138)
Adds a new native `ipaws` adapter for FEMA IPAWS-OPEN EAS civil alerts. - Two-stage CAP fetch via base_url (direct FEMA or Conduit proxy): Atom index -> per-entry CAP 1.2 documents. - Non-weather civil alerts only (evacuation, Civil Emergency Message, AMBER, 911 outage, law-enforcement, HazMat); NWS/NOAA CAP dropped so weather is never double-broadcast. - Idaho + neighbour statefips scope gate applied before stage-2 fetch. - Own `ipaws_alerts` dedup table (migration v29); reuses the NWS CAP severity + formatter pattern. - Ships enabled=False (no transmit until explicitly enabled). Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
1199b3576a |
fix: make TomTom + FIRMS API keys optional (keyless-capable) (#137)
traffic and firms no longer idle when their key is blank — they build a keyless request (traffic: omit key= param; firms: omit the map_key path segment), matching roads511's existing optional-key pattern. Enables routing these feeds through a key-injecting proxy (Conduit) with the key held only there. Key-set behavior is byte-identical. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
998838bcb2 |
fix: make all native feed URLs config-driven (no hardcoded upstreams) (#136)
Adds a base-URL config field (default = the current value, backward-compatible) to the 9 adapters that hardcoded their upstream URL — nws, swpc (4 endpoints), ducting, fires (perimeter+points), firms, avalanche, usgs streams (3 bases), traffic, satpass/tle_fetch — mirroring the already-compliant roads511 pattern. Every feed URL is now overridable via config, enforcing the "everything configurable" rule and making each adapter live-repointable via a config PUT. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
8b4826f8db |
feat: hot-reload the environmental config section (no restart) (#135)
EnvironmentalStore.apply_config() rebuilds only the changed native adapters in place on a config PUT -- dedup/seen state (store-level) is preserved, unchanged adapters are untouched. Drops "environmental" from RESTART_REQUIRED_SECTIONS; falls back to restart-required only for the narrow feed_source->central case. Cascades nifc/fires -> firms. The old restart requirement was a Central-era coupling, now moot (all-native, central.enabled=false, CentralConsumer inert). Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
4fd431f907 |
fix(reminders): pace the reminder roll-call so N fires don't burst (#130)
The ReminderScheduler is the third fire-broadcast exit and the only one
that does not pass through the EventBus, so FirePacer (which paces the
Central and native fire-event exits to <=1/60s) never sees it. Its tick
is a roll-call: every eligible row is its own broadcast, dispatched in a
plain `for` loop with no gap. Unpaced, N eligible fires produce N
back-to-back mesh transmissions; the only downstream protection is
RadioSendQueue's ~2.2-2.6s per-transport inter-packet jitter, which
prevents packet collision but still lets a roll-call monopolise the mesh.
Not currently firing in production (every overdue fire is filtered by
terminate_when, so the eligible set is 0) -- this is fire-season
hardening against a latent burst, not a live incident.
Adds `spacing_seconds` (adapter_config, default 60 to match FirePacer)
enforcing a minimum gap between consecutive SUCCESSFUL reminder
deliveries. Deliberately a pure spacing change:
* WHAT gets broadcast is untouched; nothing is dropped.
* The ok-gated last_broadcast_at stamp still uses the tick's `now`.
* A failed dispatch sent no packet, so it does not arm the gap.
* Rows filtered by terminate_when/render never burn a spacing slot.
* A lone eligible fire has nothing to pace against -> zero added latency.
* The wait is interruptible by stop(): a 15-fire roll-call holds
tick_once() for ~14 min and stop() awaits the tick task, so a plain
sleep would stall shutdown.
Chose in-loop spacing over routing reminders through FirePacer itself:
reminders re-derive their targets from live DB state every tick and only
clear a row via last_broadcast_at after a confirmed send, so enqueuing
into a 60s-drain FIFO would re-enqueue the same fire on every intervening
tick -- the queue would grow faster than it drains. pacer.py, consumer.py,
store.py and main.py are untouched.
Tests fake the clock end-to-end, so 60s spacing costs the suite nothing.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
afd045aa96 |
fix(gating): firms decide() spotting/halt severity key, issue #121 (#126)
gating/firms.py's decide() stamped a plain "severity" key in the
data_patch for the wildfire_spotting and wildfire_halted broadcast
paths. central/consumer.py only ever promotes data["_severity_override"]
onto Event.severity -- the plain key is a silent no-op, the same class
of bug as #118 (fixed for firms_handler.py's own inline stamps in
PR #120). Currently inert (MESHAI_CUTOVER_CATEGORIES is unset by
default), but the moment wildfire_spotting/wildfire_halted are cut
over, spotting would silently stop being "immediate".
- gating/firms.py: both data_patch sites now use _severity_override.
Checked the other gating modules (fire.py, avalanche.py, swpc.py,
quake.py, nws.py) -- all already use _severity_override correctly;
firms.py was the only one with the plain-key mistake.
- Fixed the stale module docstring claiming the unattributed-hotspot
cluster path "is DEAD" -- it has been live since
|
|||
|
4f31abc593 |
chore: remove duplicate top-level source tree (build only from work/) (#125)
Commit |
|||
|
a85de23af7 |
fix(meshcore): un-shadow _resolve_contact; restore PR #56 refetch-on-miss (#127) (#133)
MeshCoreTransport defined _resolve_contact TWICE: - L171 (PR #56) cache lookup -> on miss, get_contacts(lastmod=0) full refetch -> retry. The DM / path-establishment resolver. - L1227 (PR #92) key-prefix -> by-name lookup, no refetch. The telemetry resolver, added later without noticing the collision. Python silently keeps only the LAST definition in a class body, so the line-171 implementation was dead code and PR #56 was nullified: every DM and path-establishment caller was getting the telemetry resolver instead. No error, no warning, invisible to the linter and the type checker. Fix: rename the telemetry resolver to _resolve_contact_for_telemetry and repoint its sole caller (_req_telemetry_async). The DM path (send_message) and _establish_direct_path now get PR #56's refetch-on-miss behavior back, which is what they need — replying to an inbound DM from a firmware auto-added contact requires the refetch, and re-resolving after path discovery is pointless without it. Deliberately NOT merged into one resolver: telemetry auto-polls on a timer against operator-selected contacts already in the roster, so a full-roster refetch on every miss is recurring airtime for nothing; and its by-name fallback is telemetry-specific and must not widen DM address resolution. The two want different semantics — the bug was the name collision, not that they should be one function. _resolve_contact_async (the MC-event-loop twin) already carried the refetch and was never shadowed, so the async/queue DM send path was unaffected. Add tests/test_no_duplicate_methods.py: AST-walks every ClassDef under work/meshai/ and fails if any class body defines the same method name twice. This failure mode is invisible to review, the linter, and the type checker — which is exactly why it survived. Exempts the legitimate same-name patterns (@property/@setter/@deleter groups, @overload stacks). Verified it flags the bug on the pre-fix source and finds no other duplicates in the tree. Suite: 20 failed -> 17 failed (the 3 meshcore failures gone), 2240 -> 2245 passed (+3 fixed, +2 new guard tests), 72 skipped unchanged. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
74a5fa44d4 |
fix(pipeline): let wildfire_spotting skip the grouper (category-scoped) (#131)
wildfire_spotting is the most urgent signal in the system (a fire
throwing embers past its own containment line), but every fire event
carries a group_key (= event_id), so the Grouper held spotting for the
full grouper_window_seconds (60s live) before it could even reach the
dispatcher -- a ~60s latency FLOOR, and ~120s+ once FirePacer queuing
is added on top.
Add a module-level _NEVER_COALESCE_CATEGORIES frozenset and bypass the
coalescing window for the categories in it. Currently: wildfire_spotting
only.
This is safe because spotting is already rate-limited AT THE SOURCE: a
per-fire 1h cooldown (adapter_config.fires.spotting_cooldown_seconds,
default 3600, latched on fires.last_spotting_broadcast_at per irwin_id
in gating/firms.py) gates spotting DETECTION itself, so N active fires
yield at most N spotting alerts per hour. FirePacer (60s interval, with
head-of-line ordering for immediate severity) and the dispatcher's
per-(toggle, category, region) cooldown still apply downstream.
The bypass is scoped by CATEGORY, never by severity. Commit
|
|||
|
c82cceffde |
test: fix 13 stale tests in the red suite, leave 6 real-bug failures (#124) (#129)
Triaged all 20 known-red tests. 13 were stale tests asserting rotted expectations against deliberate, documented behavior changes; fixed by deriving expected values instead of hard-coding, or updating the expectation to match a documented policy change: - test_adapter_config_foundation.py / test_adapter_config_api.py: REGISTRY/API key-count and key-set guards hard-coded magic numbers (59/94/17) that rotted repeatedly. Now derive expectations from REGISTRY itself and, for the schema version, from the migrations directory, so they can't rot the same way again. - test_fire_tracker_phase4.py: two tests hardcoded a nonexistent deployment path (/opt/meshai/meshai/router.py) that matches no Dockerfile WORKDIR in this repo; resolve the module path via importlib.util.find_spec instead. - test_tombstone_broadcast.py: asserted fire severity == "immediate", which commit |
|||
|
8ba8700466 |
fix(fires): wire FirePacer into the native fire broadcast path (#123)
Native fire adapters (env/fires.py source="nifc", env/firms.py source="firms") emitted straight to the EventBus from EnvironmentalStore._emit_event with no rate limiting of their own. FirePacer was only ever attached to CentralConsumer (main.py), which never runs in the actual production deployment (central.enabled=False, all adapters feed_source=native) -- so the <=1/60s throttle + immediate head-of-line behavior fixed for Central in #120 (issue #119) was completely inert in production. A poll that produces several distinct fires/clusters at once (a lightning outbreak, or several tracked fires crossing a satellite-pass boundary together) would dump all of them on the mesh back-to-back instead of at the intended cadence. _emit_event() now routes fire-family Events (source in {"nifc","firms"}, severity in {"priority","immediate"}) through an attached FirePacer, mirroring the exact gate CentralConsumer._handle applies. main.py attaches the same FirePacer instance to env_store right after constructing it. "routine"-severity fire events, non-fire native adapters, and the Central path are all unaffected; a paced event cannot re-enter either gate (native vs Central are mutually exclusive per feed_source), so nothing can be paced twice. Added tests/test_native_fire_pacer.py covering: native fire events route through the pacer, an immediate event jumps an already-queued priority queue with nothing dropped, routine-severity fire events and non-fire native events are NOT paced, and the no-pacer-attached fallback is unchanged. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
|||
|
5dd8266abe |
fix(firms): repair the FIRMS fire-fusion Event contract (issues #117-#119) (#120)
Three independent bugs kept firms_handler's growth/spotting/halt/cluster fusion decisions from reaching a correct mesh Event: - #117: consumer._normalize() computed `category` from the raw Central category BEFORE the per-adapter handler ran and never re-read data["category"] afterward, so every firms_handler category stamp was a silent no-op. Now re-read post-dispatch, validated against the known category registry (unrecognized overrides are logged and ignored). - #118: consumer.py only ever honors data["_severity_override"], but firms_handler's halt/spotting/cluster sites stamped the plain data["severity"] key instead (only growth used the right key). Switched all three sites to `_severity_override` for one consistent contract. This is severity plumbing only -- it does not change which events fire. - #119: FirePacer's gate only matched source in ("fires","wfigs") at severity=="priority", so FIRMS fusion broadcasts (source="firms", growth/spotting at "immediate") never reached the pacer. Broadened the gate to cover "firms" + {"priority","immediate"}, and gave FirePacer head-of-line insertion so an "immediate" event is never stuck behind already-queued "priority" events. Still unbounded/never-drops. Cluster detection is left exactly as main ships it: live, always on, no toggle (PR #73's curated new-fire cluster broadcasts with cold-start silent-seeding). Only its severity-override key changes, under #118. Updated existing tests that asserted the old (buggy) data["severity"] contract, and added tests/test_firms_fusion_event_contract.py covering all three fixes end-to-end through consumer._normalize()/_handle() and FirePacer directly. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
f50c2e54d8 |
fix(dispatcher): qualify region-cooldown key by channel type (#115)
Section 1.5 (region_routes matrix branch) armed and checked the per-region cooldown key as (toggle, category, region) with no channel-type component. _chans always inserts mesh_broadcast before meshcore_broadcast (insertion order in the per-cell append loop), so for any matched cell with BOTH mt and mc populated, the mesh_broadcast send armed the cooldown key first; the very next iteration checked that SAME key for meshcore_broadcast and saw it as freshly cooled down, dropping it every time. Net effect: meshcore_broadcast never succeeded via the matrix branch whenever cooldown_seconds > 0 (true for weather/roads/fire, all 300s), so it never armed its own dedup/cooldown state either -- a silent, permanent MC blackout for every region-routed family. Confirmed live: dispatcher_dedup had 555 rows, zero meshcore_broadcast; mesh_broadcasts_out was 120:4 MT:MC for nws_alerts and 98:8 for traffic_events over 10 days (the few MC rows that got through came from a different, non-matrix code path). fires' 98:77 near-1:1 ratio is not evidence the matrix branch worked for fire -- those MC sends are dominated by the cooldown-exempt scheduled reminder path (dispatch_scheduled_fire_broadcast); fire's own live event-driven path has the identical latent bug, just masked. Fix folds ch_type into the region string (mirrors the existing _cd_suffix convention) so mesh_broadcast and meshcore_broadcast get independent cooldown windows. Kept the cooldown key a 3-tuple (instead of widening to 4) to avoid a dispatcher_cooldowns schema migration -- _persist_cooldown() and the boot-restore SELECT are both hard-coded to (toggle, category, region). Verified in isolation (no live/deployed behavior change, no transmit): a fresh Dispatcher built from the live production config now dispatches both mesh_broadcast and meshcore_broadcast for weather/roads/fire matched cells under a 300s cooldown. Added two regression tests covering the gap that let this ship untested: no existing test combined cooldown_seconds > 0 with a cell that has BOTH mt and mc populated (test_cell_match_routes_mt_and_mc uses the cooldown_s=0 default; test_per_region_cooldown_independence uses cooldown_s=300 but with mc=None on every cell). Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
aa3e5943c4 |
feat(context): durable SQLite-backed retention for MeshContext observations (#114)
Mesh-context buffer (recap/summary source) was pure in-memory (deque(maxlen=50000)), wiped on every container restart. Adds mesh_observations table (migration v28) + persistence/mesh_observations.py accessors following the existing get_db()/observer_locations.py pattern. MeshContext.observe() now write-throughs each observation to SQLite (fail-safe -- DB errors are logged and swallowed, never block or drop the in-memory path). __init__ loads recent rows (within max_age, up to the hard cap) back into the deque on startup so recap works immediately after a restart. prune() now also deletes SQLite rows older than max_age on the same hourly cadence as the existing in-memory prune. Verified: wrote one observation through the real observe() path, confirmed 1 row in mesh_observations; docker restart meshai; log confirms "Loaded 1 mesh observations from durable store" and a fresh MeshContext instance recovers the observation via get_context_block(). Investigated a suspected recap-grounding bug (LLM ignoring the mesh-context block and replying with a disclaimer) per the same task brief, but could not reproduce it against gemini-3.1-flash-lite across 8 query phrasings (minimal and full production-fidelity system prompts, including the configured static prompt clause that explicitly permits the disclaimer when no traffic is shown) -- the model correctly grounds its replies in the observed traffic block in every case tested. No grounding-clause change made; premise did not reproduce. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
1a8ae710dd |
MeshCore: channel provisioning, key sharing, location-based recommendations & transport-aware identity (#113)
* feat(meshcore): self-service add/remove channel provisioning from dashboard
Adds MeshCoreTransport.add_channel()/remove_channel() (meshcore_transport.py),
scanning the full 40-slot companion channel table for a free slot (no early
empty-run cutoff, unlike _enumerate_channels) and writing/clearing slots via
the only available write opcode (set_channel, 0x20) — the lib exposes no
delete-channel opcode, so removal writes the slot back to its empty state
(blank name + all-zero 16-byte secret).
POST /api/meshcore/channels and DELETE /api/meshcore/channels/{name} routes
(mesh_send_routes.py) validate name/hex-key and return the refreshed channel
list, matching the existing secrets_routes.py HTTPException idiom.
Frontend: Add-channel row (name + optional PSK hex) and a per-channel Remove
affordance inside the existing Observe MeshCore Channels block
(MeshCoreConnection.tsx), plus addMeshcoreChannel()/removeMeshcoreChannel()
API client functions (api.ts).
Built + deployed to CT 108 (docker compose build && up -d, container
healthy); self-cleaning smoke test passed — POST/DELETE of a #meshai-test
channel left the companion table exactly as found.
* MeshCore: display per-channel key (PSK) with reveal + copy
The Observe MeshCore Channels list now shows each channel's PSK hex so
operators can share it with people who want to join. Fetches the
/api/meshcore/channels/detail endpoint (name + key) instead of the
names-only list; keys are masked by default with a per-row eye reveal
toggle and a copy-to-clipboard button. Channels with no retrievable key
show a dash. Existing observe (checkbox), remove (trash), and add-row
controls are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(llm): recommend a MeshCore channel by location + flash-lite default
Add a location-aware MeshCore channel-recommendation context block to the
single-shot LLM prompt so "what channel should I join?" gets an answer with
the channel NAME and join KEY.
- router.py: new standalone helper _build_meshcore_channel_block(config,
channel_details, position) — lists ONLY observed channels
(meshcore_context.observe_channels), each with its PSK key
(channel_details) and, where a region_routes cell maps to it, the covered
region's human geography (local name, cities, centroid) built from a
channel->regions reverse map over notifications.region_routes.cells and a
best-effort fuzzy attach to mesh_intelligence regions. Injected inside the
existing should_inject_mesh gate, right after the region-geography block;
fetches channel_details() + get_node_position() off self.connector and is
fully fail-safe (any error skips the block, never breaks the LLM path).
Empty observe_channels => empty block (LLM cannot invent channels).
- composite_transport.py: add channel_details() passthrough mirroring
known_channels().
- README.md: bump the example llm model to gemini-2.5-flash-lite so a fresh
deploy is policy-compliant (flash-lite supports Google Search grounding).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(identity): transport-aware bot identity + alert-channel awareness
Fixes the LLM system prompt being transport-blind: MeshCore DMs were
being told they were on the freq51 Meshtastic mesh network and were
physical node !27780c47 (AIDA-N2), and MeshMonitor/node-health text
(Meshtastic-only) leaked into MeshCore prompts too.
- config.py: BotConfig gains mt_mesh_name/mt_node/mc_mesh_name (generic
OSS defaults empty; name/owner defaults now generic MeshAI/Unknown).
LLMConfig.system_prompt code default no longer hardcodes freq51/Twin
Falls.
- router.py: generate_llm_response() computes transport once and
branches identity: MeshCore gets an AIDA/MeshCore-radio framing with
no MT node id; Meshtastic keeps the physical-node framing from the
new config fields. MeshMonitor block and MT node-health/gateway/packet
reporting (mesh_reporter Tier1/region/node detail + _MESH_AWARENESS_PROMPT
+ region geography) are now gated to transport == meshtastic. New
_build_alert_channels_line() adds a short identity-block line on both
transports listing region_routes-derived alert channels (MT channel
index or MC channel name). MeshCore channel-recommendation block stays
transport-neutral.
- README.md: dead gemini-2.5-flash-lite example updated to
gemini-3.1-flash-lite.
Live config (meshai_data volume, not git-tracked): bot.name=AIDA,
bot.owner=K7ZVX (local.yaml identity.owner, which overrides config.yaml
per config_loader.py LOCAL_FIELDS), mt_mesh_name/mt_node set to the
freq51/AIDA-N2 values, mc_mesh_name left empty. llm.yaml system_prompt
freq51 line removed.
Verified via in-container prompt-assembly dry run through the real
generate_llm_response() path (no LLM call) for both transports, and a
live gemini-3.1-flash-lite 3-query behavior test confirming the
MeshCore channel-recommendation feature still works correctly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
22037c9a23 |
feat(meshcore): set/clear room-server passwords from the routing picker (#112)
Adds PUT/DELETE /api/meshcore/room-password/{pubkey} (dedicated route — the
generic secrets allowlist rejects dynamic per-room vars) and a password_set
flag on GET /api/meshcore/rooms. The routing picker gains an inline lock +
set/clear editor on room-mode cells; state is keyed by room pubkey and shared
across cells targeting the same room. Send-time login already reads the stored
password via secrets_store — no dispatch changes needed.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
|
|||
|
2cc672e751 |
fix(meshcore): room-picker toggle now opens the room dropdown (#111)
The MeshCore routing cell's channel-vs-room toggle derived its mode solely from whether the value started with `room:`, and the Room button merely cleared the value -- so room mode was never entered, the room <select> (gated on that) never rendered, and a room could never be selected (chicken-and-egg: the select was the only writer of the `room:` value). Add explicit per-cell mode state that the toggle sets directly (falling back to value-derived on first render, so existing room: cells load in room mode); Home now enters room mode and renders the dropdown even with an empty value, Hash renders the channel input. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
2e7b3d6934 |
feat(meshcore): route MeshCore cells to room servers (open rooms) (#110)
Extends MeshCore routing so a region_routes `mc` cell can target a room server, not just a `#`-channel. A room = a contact with type==3; a cell value of `room:<pubkey>` routes to it via the existing addressed DM path (send_msg to the room's pubkey), with an optional login for password-protected rooms; a bare cell value stays a channel broadcast (unchanged). Adds transport get_rooms()/login_to_room()/send_to_room_async, a GET /api/meshcore/rooms endpoint, per-room password storage in secrets_store (env MESHCORE_ROOM_<prefix>_PWD), and a routing-GUI channel-vs-room picker (rooms shown by name with a path indicator). Open rooms work end-to-end. Password-protected rooms need a follow-up: a backend endpoint to SET the per-room password from the GUI (the generic secrets API is allowlist-gated); the storage + login already exist. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
77e057ae86 |
fix(wzdx): persist current work zones to traffic_events via dedicated ingest (#109)
The WZDx daily summary + DM query count from traffic_events, but work zones weren't landing there: wzdx rode the generic _delta_emit path, which silent-seeds the seen-set and returns before the decider's INSERT on the cold-start first poll, so the current zone set never persisted (summary would count ~0). Add a dedicated _ingest_wzdx (mirroring the fires ingest) that UPSERTs every current coalesced zone into traffic_events each poll (persist-only, last_broadcast_at=NULL, no emit, no broadcast) and reconciles zones that drop out of the feed (never wipes on an empty/failed fetch). Per-event work-zone broadcast stays suppressed (the decider's work_zone gate is untouched). Retargets 4 tests in test_store_received_delta.py that used a fake 'wzdx' source to exercise the generic gate onto a neutral routing name. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
15ddedf22b |
feat(adapters): WZDx daily work-zone summary + FIRMS restart-safe cold-start (#108)
1. WZDx work zones: replace per-event broadcasting with a once-a-day per-region count summary. Coalesce the upstream per-direction / per-schedule-day fan-out into one row per physical zone (road + lat3 + lon3 + sub_type); work zones are stored in traffic_events but no longer per-event broadcast, while 511 crash / closure / hazard incidents still broadcast live. A WZDxSummaryScheduler emits one count line per coverage region once a day (default 07:00 America/Boise), routed via the region_routes 'roads' cells; work-zone details are DM-queryable (build_work_zones_detail). New config: wzdx.summary_enabled / summary_time / summary_tz. 2. FIRMS cold-start is now restart-safe: gate the silent-seed on the persisted firms_pixels baseline being empty (first-ever run) instead of an in-memory per-boot flag, so a restart no longer silently absorbs a genuinely-new hotspot cluster. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
b0b0697bac |
fire: remove fire digest feature and drop out-of-coverage fires at ingest (#107)
Two fire-scope cleanups: 1. Remove the fire digest feature entirely -- scheduler (notifications/scheduled/fire_digest.py), pipeline wiring, the fires.digest_* adapter_config key registrations, and the Fire Digest dashboard UI (ScheduledBroadcasts / Environment / Reference / AdapterConfig / ActivityLog). The unrelated generic per-rule notification digest is kept. Orphaned fires.digest_* config rows and the fire_digest_broadcasts table are left as inert data (v16 migration untouched). 2. Add a coverage-scope gate at fire ingest: _ingest_fires now skips any fire whose coordinates fall outside all configured coverage areas (same areas_from_config + classify_geom_areas membership the dispatch-level CoverageFilter uses), so out-of-coverage fires are never stored, tracked, alerted, reminded, or re-ingested. Fails open when coverage is disabled, has no areas, or excludes the fires adapter. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
af826319c8 |
feat(reminders): route fire reminders through per-region fire routing (#106)
Fire (wfigs) reminders previously dispatched via the generic scheduled path, which hardcoded the rf_propagation toggle (Meshtastic ch4 / MeshCore #aida) and ignored the fire's region. They now route through a new dispatch_scheduled_fire_broadcast() that builds a synthetic fire event from the fire's lat/lon, derives its region the same way the live fire event path does, and routes per region_routes.cells['fire'] (per-region MT/MC channels), falling back to the fire toggle's own defaults when a transport isn't matrix-owned -- never rf_propagation. rf_propagation and 511 reminders are unchanged. Reminders remain disabled (reminders_wfigs.enabled stays false); this only fixes routing for when they are enabled. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
fed63b2344 |
fix(channels): dedupe Meshtastic channel list, un-bury MeshCore channels card (#105)
The Meshtastic connection page listed channels twice (the pre-existing Observe-Channels selector plus a redundant read-only Index/Name/Role table); remove the redundant table and its now-unused getChannels() / MeshtasticChannel API helpers. On the MeshCore companion page, move the Channels (name/hash/key) card up to sit right after the Status card, above the Advertising settings, so it is no longer buried at the bottom. No behavior change; layout/UX only. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
6e1831b606 |
refactor(channels): move channel info onto the connection pages + expose MeshCore channel key (#104)
Remove the standalone Channels page and its nav entry (avoids menu bloat) and relocate the channel listings onto the existing connection pages: MeshCore channels (name, on-air hash, and the channel key/PSK with a copy button) on the MeshCore companion page, and Meshtastic channels (index, name, role) on the Meshtastic connection page. The MeshCore transport now retains the channel_secret it already fetches at enumeration and /api/meshcore/channels/detail returns it as `key` (hex PSK) so operators can provision companion radios. known_channels() and /api/meshcore/channels are unchanged. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
486ad1016f |
feat(channels): read-only Channels view for Meshtastic + MeshCore (#103)
Add a read-only Channels page listing both transports' channels with the correct identifier per transport: Meshtastic by channel index (index, name, role) from /api/channels, and MeshCore by channel name plus the on-air hash from a new /api/meshcore/channels/detail endpoint. The MeshCore transport now retains the channel_hash it already fetches at enumeration (previously discarded); known_channels() and the existing /api/meshcore/channels endpoint are unchanged. Adds a Channels nav entry and /channels route. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
0feb8adaca |
fix(region-routing): independent mt/mc region-routing enable switches (#102)
Split the single region_routes.enabled master switch into per-transport mt_enabled (Meshtastic) and mc_enabled (MeshCore) flags so the two transports can be region-routed independently. Previously the shared switch forced MeshCore into the region matrix; with all mc cells null it routed MeshCore nowhere instead of falling through to the toggle-level meshcore_channel. The dispatcher is now authoritative per-transport: a disabled transport falls through to its toggle path, and matched-but- inactive cells still suppress the toggle for enabled transports. The destinations delivery branch also honors matrix-handled suppression to prevent double-broadcast. Loader maps legacy enabled:true to mt_enabled:true, mc_enabled:false. Each GUI routing page gains its own master enable toggle. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
3ee33015bd |
fix(activity-log): extend TEXT_HINTS with all observed legacy prefixes + backfill remaining 39 NULL rows (#101)
STEP 1 audit of the live DB revealed 6 distinct prefixes across the 39 residual NULL rows, not just ⏳ WX / ⚠️ Road Incident as originally anticipated: nws_alerts (30): 🌬️ Special Weather Statement (25) — NWS SPS via nws.py ⛈️ Severe Thunderstorm Warning (3) — NWS SVR via nws.py 🌩️ Update: Severe Thunderstorm (1) — NWS update via nws.py ⏳ WX: (1) — legacy composer weather-watch traffic_events (9): ⚠️ Road Incident (7) — incident_handler.py (sub_type incident) 🚫 Road Closed (2) — incident_handler.py (sub_type road_closed/closure) TEXT_HINTS additions: 🌬️ / ⛈️ / 🌩️ → Weather; 🚫 → Traffic. All 6 prefixes are unambiguous (confirmed against formatter source + live data). DB: 39 rows backfilled (audit table only). Final NULL residual = 0. No mesh transmission. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
|||
|
0857b92788 |
fix(activity-log): map legacy ⏳ WX / ⚠️ Road Incident prefixes to Weather/Traffic + backfill residual orphans (#100)
Extends TEXT_HINTS with three missing legacy prefixes so NULL-source audit rows render the correct family label in the Activity Log UI: - ⏳ (U+23F3) → Weather (legacy composer weather-watch: "⏳ WX: …") - 🌡️ → Weather (NWS heat-alert format added in PR #96) - ⚠️ Road Incident (U+26A0+FE0F) → Traffic (legacy incident_handler rows) Placed more-specific prefix before shorter variants to avoid false matches. DB backfill: 0 residual NULL rows found (all rows already tagged after PR #99); no audit-table writes performed. No mesh transmission. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
|||
|
40fcbf88e8 |
fix(activity-log): stamp source_event_table for native adapter broadcasts + UI label fallback + backfill recent orphans (#99)
Backend: add _SOURCE_TO_TABLE class constant in Dispatcher mapping event.source
("nws", "nifc", "wzdx", "traffic", "511") to canonical audit table names.
_post_broadcast_commit now falls back to this map when _broadcast_audit is
absent/None, so native env adapter sends (nws.py, fires.py, wzdx.py,
roads511.py, traffic.py) write a non-NULL source_event_table instead of NULL.
Existing _broadcast_audit paths (Central handlers, scheduled broadcasts) are
unchanged.
Frontend: replace naive familyLabel() string transform with explicit
TABLE_LABELS lookup (10 known tables → friendly names) plus a TEXT_HINTS
emoji-prefix heuristic for legacy NULL-source rows, so historical orphan rows
still display a meaningful label before/after backfill.
Tests: three new unit tests in test_dispatcher_persistence.py covering the
fallback path (nws→nws_alerts), the full _SOURCE_TO_TABLE map, and that
explicit _broadcast_audit is never overridden by the fallback.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|||
|
d988868257 |
feat(fires): fuse WFIGS incident-point layer with perimeter layer so non-perimeter fires surface (dedup by IrwinID, cold-start silent-seed) (#98)
Add WFIGS_Incident_Locations_Current point layer (IRWIN superset, ~6 ID fires) alongside the existing perimeter layer (~2 ID fires). Fires are merged by IrwinID: point layer is the authoritative superset, perimeter layer supplies polygon geometry and validated acreage when available. Point-only fires surface with lat/lon from the point geometry and no polygon. cold-start silent-seed path is unchanged (first-poll batch is always silent regardless of source). Perimeter fetch failure falls back to perimeter-only stubs; point fetch failure falls back to perimeter-only; both failing bumps the consecutive error counter. county is now populated from POOCounty on the point layer. Five off-air unit tests cover: T1 merge-dedup, T2 point-only-new, T3 perimeter-geom-preferred, T4 cold-start silent-seed 6 fires, T5 FIRMS _get_known_fires attribution. No changes to store.py, gating/fire.py, schema, or coverage. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
|||
|
099cec783c |
fix(native): promote decider severity/category onto Event + make silent severity-floor drop observable (counter+log) (#97)
CHANGE 1 (store.py): after applying gate.data_patch into event.data, promote
_severity_override and category keys onto event.severity / event.category.
Previously, decider overrides (e.g. fire: "priority" on every New/Update)
landed only in event.data, leaving event.severity at the adapter's raw value
("routine" for fires >=25 km from an anchor). This silently failed the
toggle/matrix min_severity floor. Native and Central now share identical
broadcast decisions at the shared choke point.
CHANGE 2 (dispatcher.py + v27 migration): both the toggle-path and
matrix per-cell severity-floor drop paths now emit a WARN log and
increment a new persisted counter (severity_floor_dropped) in
dispatcher_state, following the existing drop-counter pattern exactly.
v27.sql adds the column; the counter restores on restart and appears
in dispatch_stats().
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
