Satpass events carry no lat/lon/geometry, so CoverageFilter had no way
to region-tag them for region_routes matching. gate_consolidated_pass()
now attaches observer_list (comma-joined observer slugs, same shape
already computed for the audit column) into the event data, which
coverage_area.observer_region_names() reads. Defensive/fail-open: an
empty observer_list yields no region tag rather than raising.
The USGS feed is a rolling PAST-DAY feed (2.5_day.geojson), so a
genuinely first-seen quake's age-at-ingest routinely exceeds the
generic per-toggle freshness_seconds (600s) window, silently dropping
it downstream of decide()/DB-insert (last_broadcast_at stayed NULL on
every sampled first-sighting row).
- dispatcher.py: earthquake_event gets its own adapter_config-backed
freshness override (adapter_config.usgs_quake.freshness_seconds,
default 3600s), scoped to the CATEGORY (not the "seismic" toggle,
which also carries hydro stream_flood_warning/stream_high_water and
must keep the generic per-toggle freshness).
- usgs_quake.py: to_event() no longer passes the adapter's fixed
freshness in, deferring to the dispatcher override.
- adapter_config/defaults.py: adds the freshness_seconds field/default
for usgs_quake.
Adds explicit full-closure phrase matching (with partial-restriction
phrases as vetoes) for sub_type mapping in to_event(), so wording like
"All Shoulders Closed" or "Right Lane Closed" no longer gets broadcast
as a full road_closed event. Existing is_closure severity/summary logic
is untouched; the new is_full_closure_flag/_has_full_closure_language
helpers are additive and scoped to sub_type only.
Adds meshai.notifications.evac_phase.detect_phase, wired into the IPAWS
formatter/adapter, to classify READY/SET/GO evacuation phases from real
FEMA IPAWS alert headline/CMAMtext strings. Includes explicit
false-positive guards since a wrong GO detection would broadcast an
evacuation order that was never issued.
* chore(fire-ripout 2dii-a): remove dead wildfire_growth cutover path + handle_firms
wildfire_growth events are always fully precomposed (env/firms.py sets
_meshai_precomposed=True + title=<wire from _render()>) and the category is
not in cutover.NATIVE_ALWAYS_DECIDE, so the registered fire formatter could
never be reached live via compose_mesh_message for this category -- the
precomposed-title bypass always won first. Removes the dead
is_cutover("wildfire_growth") NEW-PATH branch in
fire_fusion._handle_pass_boundary (kept the live legacy leg that calls
_render unconditionally) and the now-unreachable wildfire_growth formatter
registration in notifications/formatters/__init__.py.
Also removes handle_firms, the dead Central NATS-envelope entrypoint (zero
live production callers -- Central's consumer that drove it is gone), plus
its envelope-specific filtering helpers (_confidence_passes, _in_bbox,
_coerce_severity, _log_event, FIRMS_CONFIDENCE_FLOOR/FRP_FLOOR/BBOX_OPTIONAL)
that had no live callers left. The shared, LIVE fusion core
(_ingest_pixel_core, attribution, clustering, growth/spotting/halt) and
_parse_acq_epoch are unaffected -- still the engine behind the native
ingest_hotspot_pixel entrypoint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(fire-ripout 2dii-b): remove dead handle_wfigs entrypoint
handle_wfigs (the dead Central NATS-envelope entrypoint) had zero live
production callers -- Central's consumer that drove it is gone. The LIVE
WFIGS path is env/fires.py (native adapter) -> env/store.py::_emit_event,
forced onto gating.fire.decide + the shared fire formatter via
cutover.NATIVE_ALWAYS_DECIDE, independent of this dead handler.
Removes handle_wfigs and its private-only helpers (_coerce_severity,
_log_event, _log_event_returning_id) that had no callers left. _render
remains -- it is called directly by env.fire_fusion._handle_pass_boundary on
the FIRMS wildfire_growth path, and is used as a byte-identity oracle by
tests against the shared, live fire formatter. _build_canonical,
_attach_commit_handles, _fire_too_old_to_announce, _now, _cleanup_stale_fires
and WFIGS_BROADCAST_COOLDOWN_S are kept -- each has its own direct test
coverage independent of handle_wfigs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(fire-ripout 2dii-c): rewire fire tests off the deleted dead entrypoints
handle_firms and handle_wfigs (both removed in the prior two commits) were
used throughout this test suite purely as convenient DRIVERS for live logic
(pixel attribution/clustering/growth/spotting/halt fusion; the WFIGS
New/Update/cooldown/closed decider). Per-file disposition:
Rewired to drive the LIVE native entrypoint directly (no behavior change --
same underlying _ingest_pixel_core / gating.fire.decide engines):
- test_fire_tracker_phase1/2/3.py: handle_firms -> ingest_hotspot_pixel
- test_firms_refactor.py: handle_firms driver helpers -> ingest_hotspot_pixel;
dropped the dead wildfire_growth cutover test + the fully-redundant
TestNotCutoverLegacyVerbatim class (already covered by
test_firms_native_fusion.py's TestIngestGrowth/Spotting/Halt against the
live entrypoint); wildfire_growth formatter registration test now
asserts None (matches source change).
- test_wfigs_handler.py: handle_wfigs -> _render (the live WFIGS renderer)
called directly, for the anchor-priority + missing-acres cases that
exercise shared/live code (_location_anchor).
- test_fire_refactor.py: handle_wfigs -> gating.fire.decide() driven
directly, with state written the same unconditional shape the live
native path uses; TestGateSequenceParity trimmed to focus on the
tombstone/closed lifecycle step (not covered by
test_fire_native_growth.py's native-adapter New/Update/cooldown coverage).
- test_tombstone_broadcast.py, test_fire_age_gate.py: handle_wfigs ->
gating.fire.decide() driven directly; same assertions, off the dead path.
Deleted as pure dead-entrypoint contract testing with no live equivalent:
- test_firms_native_fusion.py::TestCentralPathUnchanged (2 tests) --
existed only to guard handle_firms's own envelope parsing.
- test_firms_handler.py: confidence/FRP/bbox filtering, missing-coords,
non-firms-adapter guard, event_log accounting (all handle_firms-specific;
the native adapter filters upstream in a different code path). Kept +
rewired: the shared _ingest_pixel_core dedup behavior and
_parse_acq_epoch's int/short acq_time parsing (both genuinely live/shared).
- test_wfigs_handler.py: envelope field-extraction (acres-fallback chain,
IA-placeholder-as-name), tombstone/perimeter subject -> event_log,
New/Update/cooldown decision + audit-row wiring (all redundant with
tests/test_fire_native_growth.py's coverage of gating.fire.decide()
through the real native adapter).
- test_tombstone_broadcast.py::test_commit_callback_flips_handled --
asserted handle_wfigs's own event_log-row flip on commit, a Central-only
concept the native path never used.
- test_tail_followups.py::test_wfigs_tombstone_stamps_column -- asserted
handle_wfigs's own inline tombstoned_at UPDATE; no live producer emits
_kind=wfigs_tombstone today, so nothing in the live system stamps it.
Both live renderers stay covered: fire_format (wildfire_declared/incident,
via test_fire_refactor.py + test_fire_native_growth.py, untouched) and
_render (wildfire_growth via FIRMS, via test_fire_tracker_phase2.py +
test_firms_native_fusion.py's TestIngestGrowth, both driving the live
ingest_hotspot_pixel entrypoint).
Collection: 2010 -> 1974 tests (net -36: dead-only tests deleted, live
behavior rewired 1:1 or consolidated onto already-existing native-path
coverage). Full suite: 1974 passed, 0 failed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Final structural step. central_normalizer.py (973 LOC, misnamed — it lived
OUTSIDE central/ but its docstring described "the Central firehose") is DELETED.
No file or symbol named "central" survives (only immutable historical comments
in an already-applied migration .sql remain — those are not edited).
Split per the owner's rule ("next to what uses it"):
- Live geo/place-name utilities (nearest_town, _haversine_miles, _bearing_compass,
_compute_distance_bearing, _h3_cell, _photon_reverse_places, geocoder config)
→ the EXISTING meshai/geo.py ("Geographic utilities…"), same domain. NOT a
junk drawer. geo.py's nearest_city (hardcoded table) and central_normalizer's
nearest_town (live Photon+H3 reverse-geocode) are different implementations —
both kept, no false consolidation.
- WZDx work-zone parsers (_parse_wzdx_federal, _norm_wzdx_sub_type, _norm_direction,
_parse_mile_posts, _clean_description, _is_uninformative_road) → env/wzdx_parse.py,
next to their sole consumer env/wzdx.py.
- ~400 dead lines deleted: the Central-envelope machinery (normalize,
_parse_state_511_atis, _parse_wfigs_incidents, _parse_itd_511_work_zone,
should_skip_state_511_atis_id, is_incident_envelope_stale, normalize_road_name,
_norm_sub_type, _parse_ends_at) — only caller was the deleted Central consumer.
Consumers rewired: env/wzdx.py, env/fire_render.py, formatters/fire.py,
formatters/_anchor.py, main.py, persistence/curation.py, composer.py.
Tests: test_central_normalizer.py RENAMED → test_geo_wzdx_parse.py (live geo/wzdx
tests kept + rewired; dead-parser tests dropped). test_itd_511_work_zone.py
deleted (tested only the dead itd_511 parser — verified gone from source).
Other test files: import-path rewires. Live coverage for nearest_town /
_parse_wzdx_federal / bearing preserved across test_geo_wzdx_parse, test_adapter_wzdx,
test_wfigs_handler, test_incident_refactor.
Suite: 2010 passed, 0 failed. The 49-test drop is the dead Central-envelope
parser tests, correctly removed. Stale "Central firehose" docstrings on the
moved code updated to describe present behavior.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Moves the last two live files out of the retired-Central folder. central/ is
now EMPTY and deleted entirely (incl. __init__.py).
- fire-fusion engine (ingest_hotspot_pixel + the growth/cluster/spotting
engine) → env/fire_fusion.py, next to its sole consumer env/firms.py.
- wildfire text renderer (_render + its live helpers) → env/fire_render.py.
THE COUPLING RESOLVED: firms_handler._handle_pass_boundary had a LAZY import
inside a function body — `from meshai.central.wfigs_handler import _render` —
sitting on the live FIRMS fire-growth path. It is now a normal top-of-file
import (`from meshai.env.fire_render import _render`), visible and greppable.
PURE MOVE — no behavior change:
- The parity oracle (test_fire_refactor.py) PASSES UNCHANGED (only its import
paths updated) — proving the fire wire output is byte-for-byte identical
before and after. Fire alerts say exactly what they said.
- handle_firms / handle_wfigs (dead entrypoints, only caller was the deleted
consumer.py) were KEPT and moved rather than dropped — the wording-cleanup
PR removes them deliberately. "When unsure, keep."
- fire_render.py's geo-helper imports still point at central_normalizer — that
file's split is a SEPARATE PR; carried the imports along, did not touch it.
Consumers + test import paths rewired. Full suite: 2059 passed, 0 failed.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Moves the last non-fire live file out of central/. idaho_gauge_sites.py held
USGS stream-gauge site data + threshold ranking — pure hydro/gauge domain,
misfiled among the retired Central handlers.
Applied the owner's rule with a clean split:
- THRESHOLD_RANK (a 5-element list used ONLY by notifications/gating/hydro.py)
inlined there — single consumer, belongs next to its use.
- The gauge-site lookup (used by env/usgs.py + persistence/curation.py) moved
to env/gauge_sites.py, next to the usgs adapter that produces gauge data.
- Dropped the redundant "idaho_" prefix (the Idaho scoping is data, not code).
Consumers rewired: env/usgs.py, notifications/gating/hydro.py, and tests.
central/ is now down to firms_handler.py + wfigs_handler.py (the fire pair,
whose relocation is a separate PR because of their cross-file coupling).
Move + import-rewire only, no behavior change. Suite: 2059 passed, 0 failed.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(central-ripout 2b): create env/satellite package, move pass_predictor
pass_predictor.py was 100% live (no dead entrypoint) — SGP4 pass
computation used by both the native satpass adapter and the on-demand
!satpass command. Straight move, no code changes: meshai.central.pass_predictor
-> meshai.env.satellite.pass_predictor. Owner directive: satellite code gets
its own folder under the feed adapters, separate from env.satpass (the
adapter) to avoid colliding with env/satpass.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(central-ripout 2b): split satpass_handler.py -> env/satellite/pass_format.py
satpass_handler.py was a split file: live wire-formatting/gate logic plus
dead Central-envelope ingest machinery whose only caller was the
already-deleted central/consumer.py NATS bridge.
Moved (live, verified via rg — external callers in env/satpass.py and
commands/satpass_cmd.py, or transitively called by them):
gate_consolidated_pass, format_pass, _check_rate_cap, _upsert_satpass,
_attach_commit, _map_severity, _canonical_id, _azimuth_to_compass,
_short_sat_name, _collapse_compass, _region_paren, _is_synthetic_observer,
_format_time_12h/24h, _format_ampm, _tz_abbr, _date_label,
plus the _SHORT_SAT_NAMES/_SHORT_NAME_SUBSTR/_SYNTHETIC_OBSERVERS tables.
Dropped (dead — zero callers outside the already-deleted consumer.py and
handle_satpass/consolidate_satpass_pending themselves; verified with rg):
handle_satpass, consolidate_satpass_pending, _cleanup_pending,
load_pending_schedule, _log_event_returning_id, _coerce_float,
_coerce_int, _parse_iso_epoch, _now, CONSOLIDATION_DELAY,
_pending_consolidation_ids, drain_pending_consolidation_ids,
_elevation_bucket (already-orphaned pre-ripout: superseded by numeric
"max NN°" wire format, zero callers anywhere but its own tests),
SCHEMA_SATPASS_EVENTS/SCHEMA_SATPASS_PENDING (unused string constants —
actual schema lives in persistence/migrations/*.sql, never imported).
Also dropped now-unused `json`/`time`/`Any` imports.
Straight code move otherwise — no logic changes to any moved function.
Two docstrings updated for accuracy (module docstring, and
gate_consolidated_pass's docstring which referenced the now-deleted
Central consumer path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(central-ripout 2b): split tle_handler.py -> env/satellite/tle_store.py
tle_handler.py was a split file: live storage helpers plus a dead
Central-envelope ingest entrypoint whose only caller was the
already-deleted central/consumer.py NATS bridge.
Moved (live — used by env.tle_fetch, env.satpass, commands.satpass_cmd,
verified via rg): upsert_tle, get_fresh_tles, get_tle_by_norad,
search_tle_by_name.
Dropped (dead — handle_tle's only callers were tests and the deleted
consumer.py; verified with rg): handle_tle.
Straight code move otherwise — no logic changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(central-ripout 2b): repoint satellite consumers at env.satellite
Rewire the three production consumers (including their lazy/function-body
imports, not just module-top ones) to the new location:
- env/satpass.py: central.tle_handler -> env.satellite.tle_store,
central.pass_predictor -> env.satellite.pass_predictor,
central.satpass_handler -> env.satellite.pass_format
- env/tle_fetch.py: central.tle_handler.upsert_tle -> env.satellite.tle_store
- commands/satpass_cmd.py: all three, same mapping
Also refreshed docstrings that pointed at the old module paths or described
the now-fully-deleted Central consolidation path
(consolidate_satpass_pending / satpass_pending buffer) as a live
alternative, and updated central/__init__.py's module docstring to stop
listing the three relocated modules among central's remaining contents.
No behavior changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(central-ripout 2b): update satpass/tle test suite for the relocation
Repoints every remaining test import at the new env.satellite.* modules
and removes/adapts coverage for the Central envelope-ingest path deleted
in this pass (handle_satpass, consolidate_satpass_pending, handle_tle, and
the filtering/coercion/staleness logic that lived only inside them):
- test_satpass_native.py, test_tle_fetch.py, test_satpass_command.py:
import-path updates only (pass_predictor, tle_store). Also dropped
test_satpass_command.py's TestTLEUpsert.test_returns_none_always
(handle_tle-specific contract, no longer applicable) and rewrote its two
latest-wins tests to call upsert_tle directly — same behavior under test,
now exercised through the still-live primitive instead of the dead
wrapper.
- test_satpass_native.py: deleted test_central_consolidate_feeds_shared_gate_merged
(spied on consolidate_satpass_pending, which no longer exists). The
merge-across-observers logic it guarded is native-side (_consolidate)
and already covered by test_two_observers_consolidate_to_one_broadcast.
- test_satpass_handler.py: gutted to the one test that calls format_pass
directly (test_format_pass_worst_case_fits_140); the rest exercised
handle_satpass's observer/norad/elevation filters, which have no live
equivalent (the native adapter filters at the config level, not
per-envelope) and is redundant with test_satpass_native.py's dedup/wire
coverage via the real SatpassAdapter path.
- test_satpass_broadcast_safety.py: kept every test that calls format_pass
or gate_consolidated_pass-adjacent REGISTRY checks directly (wire format,
clean-format rules, REGISTRY defaults); deleted TestNoradIdTypeCoercion
and TestStalenessGuard (handle_satpass-only logic, no live equivalent)
and the 6 _elevation_bucket tests (_elevation_bucket itself was dead
before this pass too — zero callers anywhere but its own tests, already
superseded by the numeric "max NN°" wire format per its own docstring).
- test_satpass_persisted_timer.py: dropped test_due_at_persisted_on_normal_ingest
(handle_satpass-only); kept the two schema/migration tests, which don't
touch satpass_handler.
- Deleted outright (tested ONLY the dead Central envelope-ingest path, no
live equivalent to port to): test_satpass_event_path.py,
test_satpass_compass_fallback.py, test_satpass_wire_fields.py.
Full suite: 2059 passed, 0 failed (was 0 failed on main pre-change).
Satpass/TLE subset (99 tests across 8 files) verified green in isolation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove central.budget re-export shim
The shim's implementation lived at notifications.formatters._budget from
the start; central.budget was only a 9-line re-export kept around for
import-path compatibility. Point every importer directly at the real
module and delete the shim:
- notifications/renderers/composer.py: lazy import inside a function
- central/wfigs_handler.py, central/satpass_handler.py: import line only
- tests/test_fire_refactor.py, test_nws_refactor.py, test_firms_refactor.py:
import line only, no behavior change
test_budget_shim.py existed solely to assert identity-equality between
the shim and the real module; with the shim gone there is nothing left
for it to test, so it is deleted too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: delete dead renderers/work_zone.py
format_work_zone_mesh() had exactly one production caller, the
central/consumer.py NATS bridge deleted in the prior Central-excision
pass. Its live replacement, formatters.incident._render_work_zone()
(registered for category "work_zone" in formatters/__init__.py), is an
already-shipped byte-identical replica per that module's own docstring.
All remaining references to renderers.work_zone were prose/comments
describing the replica relationship, not imports.
Test fallout:
- tests/test_work_zone_renderer.py tested only the dead renderer in
isolation (17 cases). Deleted — the live path has its own coverage
(test_adapter_wzdx.py's formatter-integration tests, plus
TestCrossSourceIdentity::test_work_zone_category_uses_wz_renderer and
TestWorkZoneGolden in test_incident_refactor.py).
- tests/test_itd_511_work_zone.py::test_itd_511_work_zone_renderer_produces_wire
only smoke-tested the dead renderer's wire output for itd_511 data;
redundant with TestWorkZoneGolden's byte-identical fixture coverage
for the same adapter. Deleted.
- tests/test_incident_refactor.py::TestWorkZoneGolden compared the live
formatters.incident.format() output against a golden computed by
calling the dead renderer live on two real fixtures. Mirroring the
precedent already in test_nws_refactor.py for this exact situation
(golden generator deleted out from under a parity test), the two
golden strings were captured by running format_work_zone_mesh()
against these fixtures immediately before deletion and are now
pinned as literals — same coverage, no live dependency on the dead
module.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three test files (test_llm_scoping, test_fix_meshcore_save_and_llm_test,
test_config_partial_save_merge) did `sys.modules.setdefault(_mod, MagicMock())`
to stub an optional import. `setdefault` installs the MagicMock into
sys.modules for the ENTIRE pytest session even when the real package is
present — so any LATER test that does `await <that module>.<coro>(...)`
(e.g. `await aiosqlite.connect(...)`) fails with
"object MagicMock can't be used in 'await' expression" / "Event loop is closed".
This is what made test_fire_tracker_phase4::test_natural_language_fire_
question_routes_to_llm pass in isolation but fail in full-suite order — the
leak came from an earlier file, not the victim. #140 hardened the victim's
own config/history isolation but couldn't fix an external sys.modules poison.
Fix: only fall back to the MagicMock when the real module genuinely fails to
import (guarded assignment), so a present package is never replaced. Root-cause
fix in the polluters, not a skip on the victim.
Suite: 2422 passed, 0 failed, 72 skipped — fully green (was 6 failed before
#140, then 1 order-dependent failure after). Confirmed deterministic across
repeated full runs.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Central is retired -- its NATS broker (nats://central.echo6.mesh:4222) no
longer exists -- but SatpassConfig.feed_source still defaulted to "central",
overriding the _SourcedFeed mixin default of "native". A fresh install that
enabled satpass got a silently dead adapter. The dashboard reinforced it:
Environment.tsx seeded feed_source 'central' and labelled the adapter
"via Central".
Also corrects comments that documented the opposite of the code: the
adapter_config keys usgs_quake.global_mag_floor / regional_mag_floor were
labelled "CENTRAL-PATH ONLY", but notifications/gating/quake.py reads them
unconditionally in the native path. Following those comments would have led
someone to delete live config keys.
- config.py: SatpassConfig.feed_source "central" -> "native" + docstring
- adapter_config/defaults.py: correct the two "CENTRAL-PATH ONLY" comments
- Environment.tsx: seed 'native'; reword the satpass subtitle
- tests: assert the native default; pin central explicitly where a test
exercises the central path or the feed_source flip
Suite: 2337 passed, 6 pre-existing failures (stale SCHEMA_VERSION x3,
expired TLE fixtures x2, one order-dependent), no new failures.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both were fully plumbed and read by nothing.
api_key -- self-documented as dead at config.py ("Keyless: api_key is
retained but unused"), yet wired end-to-end: a GUI ManagedSecret field, a
SECRET_FIELDS entry, an EXPECTED_SECRETS entry, a secrets_store mapping and
label, and a line in .env.example. env/wzdx.py assigned self._api_key and
never read it again. So an operator could go get an API key, paste it into
the secure secrets manager, and have it do precisely nothing -- the ritual
looked complete end to end, which is what made it worth removing rather
than leaving.
endpoints -- default ["/get/event"], exposed as an editable list in the
dashboard, never read. Copy-paste from Roads511Config.endpoints (which IS
read, at env/roads511.py; Roads511 is untouched here). WZDx discovers feeds
via the FHWA registry_url/states instead.
WZDx's actual fetch behavior is unchanged; this removes dead config only.
Note for existing installs: anyone with WZDX_API_KEY set in
/data/secrets/.env will simply have an ignored env var. Harmless -- it was
already ignored.
Suite: 2337 passed, 6 failed (the pre-existing set: stale SCHEMA_VERSION x3,
expired TLE fixtures x2, one order-dependent), 72 skipped -- exact baseline
match, no new failures.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(config): warn on unknown config keys instead of silently dropping them
_dict_to_dataclass() silently continue'd past any key not in the target
dataclass's field set -- an operator could set a config key, restart, and
have it vanish with zero feedback (config.example.yaml's phantom
mesh_intelligence keys are exactly this bug, fixed separately).
Now logs a WARNING naming the key and the dataclass, hinting at a typo or
a renamed/removed field, via the module's existing _config_logger.
Traced every dynamic/free-form config path to rule out false positives:
notifications.toggles, notifications.destinations, generic_sources,
mesh_sources, and notifications.rules all route through explicit
dict-of-dataclass or verbatim-passthrough handling and never spuriously
warn. Two legitimate legacy shapes DO hit the strict field-check path with
keys that were never (and will never be) dataclass fields:
- notifications.channels (pre-v0.5 channel list), consumed directly from
the raw dict by _migrate_legacy_channels
- notifications.region_routes.enabled (pre mt/mc-split master switch),
read directly by the explicit region_routes handler
Both are allowlisted in _KNOWN_LEGACY_DROP_KEYS so users mid-migration
don't get spurious noise on every load.
Added tests/test_config_loader.py coverage: unknown key warns and does
not raise, both legacy shapes stay silent, and the free-form/dynamic
sections never warn for keys valid on their real target shape.
* fix(config): remove phantom config.example.yaml keys, add missing live ones
The new unknown-key warning (previous commit) caught config.example.yaml
loading with SEVEN warnings, all real drift -- none were false positives
of the warning itself:
mesh_intelligence block shipped three keys with no MeshIntelligenceConfig
field and no implementation anywhere (git log -S confirms they were never
built, not leftovers from a removal):
- region_radius_miles, infra_overrides, region_labels
These only made sense under an older auto-clustering design; what
actually exists is explicit region anchors (regions: list[RegionAnchor]).
Deleted from both the live block and the commented-out example above it,
and added the four fields that DO exist and are live but were missing
from the example: regions, critical_nodes, alert_channel, alert_rules.
Also fixed the now-misleading comment at
dashboard/api/mesh_routes.py:281, which referenced region_labels --
comment only, no code change.
notifications block shipped a whole quiet-hours subsystem that was
deliberately ripped out (commit b948ed77, "silent is better than ugly")
and never implemented as override_quiet -- confirmed by zero readers and
zero dataclass fields anywhere in meshai/:
- quiet_hours_enabled, quiet_hours_start, quiet_hours_end
- override_quiet (on 4 rule entries, including "Emergency Broadcast",
which falsely implied emergency alerts bypass quiet hours)
Deleted; no quiet-hours feature implemented (out of scope -- product
decision for the owner).
config.example.yaml now loads with exactly zero warnings: the loader and
the example finally agree.
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
* fix(dashboard): register gauge_sites_import router
The gauge-sites bulk-import endpoint (CSV / NWS-AHPS) was fully built and
tested but never wired into the app -- server.py never called
include_router() for it, so POST /api/gauge-sites/import 404'd in
production. Only test_tail_followups.py exercised it, via a raw
TestClient bypassing the real app.
* test(dashboard): assert gauge_sites import route is reachable via real app
Guards against the router silently going unregistered again: builds the
actual dashboard app via create_app() and drives a real CSV import
through it end-to-end (POST + GET round-trip), instead of only hitting
the router in isolation.
* feat(dashboard): add gauge-sites bulk import UI
Wires a UI onto the now-registered POST /api/gauge-sites/import endpoint,
inside the existing GaugeSites tab (no new nav entries/pages). Adds an
Import toggle next to Add site, with CSV (paste or file-load into a
textarea) and NWS-AHPS (WFO code list) modes. Required/optional CSV
columns are documented inline so an operator isn't guessing. Result
counts (inserted/updated/skipped/detail_fetched) and any partial-failure
errors from the AHPS scrape are always shown; a pending spinner covers
the AHPS path's live water.weather.gov calls so a slow response never
reads as "imported 0 sites". The list refreshes only after a successful
import that actually changed rows.
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
* fix(hydro): make native USGS gauge flood alerts renderable
env/usgs.py emits stream_flood_warning / stream_high_water Events for
elevated stream gauges, but neither category had a registered gating
decider or formatter, and event.data was left empty -- so every
detected flood/high-water reading was silently dropped before it ever
reached the mesh (get_decider/get_formatter both returned None, and
compose_mesh_message fell through with nothing to render).
- Register the existing hydro.decide()/hydro.format() (already used by
the Central-only `stream_flow` category) under stream_flood_warning
and stream_high_water too -- same shared-decider/formatter pattern
already used for avalanche_warning/_watch, weather_*, wildfire_*, and
emergency_*.
- Add both categories to cutover.NATIVE_ALWAYS_DECIDE so the decider and
formatter actually run unconditionally (mirrors the native WFIGS fire
categories): Central never emits these two category strings, so there
is no shadow-bake window to wait out.
- env/usgs.py's to_event() now populates event.data with the canonical
hydro schema (site_id, gauge_name, stage_ft, flow_cfs, unit,
threshold_state, reading_time, lat, lon, parameter_code) the shared
gate/formatter expect, mapping the adapter's flood_status strings onto
the ranked threshold_state vocabulary.
- gating/hydro.py's decide() now also OWNS an unconditional
gauge_readings INSERT for the native source (source != "nwis"): the
table had no writer since Central's nwis_handler stopped running
(2026-07-05), so every native prior-state lookup returned "normal"
forever and every elevated reading would have rebroadcast on every
15-minute tick instead of once per crossing. The Central source keeps
its own inline handler-owned INSERT unchanged (decide() stays
read-only for source="nwis").
Left as-is (not this fix): the toggle mapping for these categories
(get_toggle() -> "seismic") is unchanged. It already matches the
sibling `stream_flow` category and is enforced by
test_water_v057.py::test_existing_hydro_entries_unchanged /
test_water_categories_have_required_fields; there is no separate
water/flood toggle family in VALID_TOGGLES, and inventing one is a
config-surface change outside this fix's scope.
Known limitation documented in gating/hydro.py: since to_event() only
ever emits elevated readings (never a "back to normal" reading), a full
recede-to-normal followed by a later re-crossing into the same tier
will not re-broadcast until a higher tier is reached -- degrades toward
silence, not spam.
* test(hydro): cover native gauge flood-alert registration, gating, and wire format
Registration tests prove stream_flood_warning/stream_high_water now
resolve a decider AND a formatter (the thing that was broken), that
neither resolves to the earthquake decider despite sharing the
"seismic" toggle name, and that both are in NATIVE_ALWAYS_DECIDE.
Gate tests drive env/usgs.py's to_event() through the real
gating.hydro.decide(): a first elevated reading broadcasts (graceful
no-prior-data handling), a sustained same-band reading suppresses, an
escalation broadcasts again, and a routine reading never reaches the
gate at all (unchanged pre-fix adapter behavior). A dedicated test
confirms the new native-persistence write in decide() does not leak
into the Central source="nwis" path.
Golden formatter tests pin the wire string for a high-water and two
flood-warning tiers, plus the missing-coords drop case -- all rendered
through the same formatters.hydro.format() the Central `stream_flow`
path uses (test_hydro_refactor.py already proves that formatter is
byte-identical to the old central.nwis_handler._render()). Native
events never carry flow_cfs (to_event() only ever emits stage/height
readings), so that segment's absence is captured explicitly as current
behavior, not ported from Central.
An end-to-end test drives to_event() -> decider -> compose_mesh_message
to prove the NATIVE_ALWAYS_DECIDE gate takes effect for the actual mesh
render path, not just formatter/decider resolution in isolation.
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
* refactor(mesh_reporter): expose recommendations as list[str]
Add recommendations_list(scope, scope_value) as the canonical source of
recommendation text. build_recommendations() now just joins that list with
the historical "OPTIMIZATION RECOMMENDATIONS:" header/bullet format it
always used.
router.py:1145 injects build_recommendations()'s output into the LLM system
prompt on the live mesh-DM path — verified byte-identical before/after
across mesh/region/node/missing/empty scopes via a synthetic fixture, and
pinned with a literal-string regression test so a future refactor can't
silently change that prompt text.
This unblocks wiring recommendations into the dashboard, which previously
only reached mesh DMs.
* feat(dashboard): serve real recommendations from /api/health
mesh_reporter was never on app.state — add it alongside health_engine etc.
in server.py's existing pattern. mesh_routes.py's health endpoint now
returns mesh_reporter.recommendations_list("mesh") instead of a hardcoded
[] TODO stub.
Add recommendations_available: bool alongside recommendations: string[] so
"the engine ran and found nothing" (healthy mesh) is never indistinguishable
from "the engine couldn't run" (unwired reporter, or an exception — logged
via logger.exception and swallowed so the rest of the health response still
serves). A crashed recommendations engine must not read as "mesh is
healthy" to an operator.
Also delete main.py's phantom `getattr(mh, "recommendations", [])` on the
websocket health_update push: nothing ever set .recommendations on the
mesh_health object (always []), and no frontend consumer reads it — the
REST endpoint above is the supported path.
* feat(dashboard-frontend): render recommendations in Nodes & Health
Meshtastic → Nodes & Health → Health tab used to be a duplicate
mesh_intelligence config editor (the same MeshIntelligenceSection also
edited from Settings → Intelligence, a stale-tab-overwrite hazard). Replace
it with a real operational view: fetch /api/health and render its
recommendations list.
Three states, matching visual conventions from Dashboard.tsx's alerts list
and MeshCoreRouting.tsx's cross-link note:
- recommendations present → Lightbulb list, one card per item
- empty + recommendations_available → "No recommendations — mesh is
healthy" (CheckCircle)
- recommendations_available === false → amber warning, not the healthy
state (AlertTriangle) — a crashed backend must not look healthy
mesh_intelligence config now has exactly one home: Settings. The Health tab
points there via the existing /config?section= deep-link pattern instead of
duplicating the editor. Config.tsx itself is untouched.
MeshHealth.recommendations/.recommendations_available are optional in the
type: the websocket health_update push doesn't include them (REST-only).
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
* chore: remove dead mesh_sources.py module
Superseded by mesh_data_store.py, whose docstring states it "replaces
mesh_sources.py with a clean three-layer architecture." No remaining
inbound imports (absolute or relative) repo-wide; the only surviving
references to "mesh_sources" are to the unrelated config YAML section
of the same name, which is untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove dead region_tagger.py module
Only reference to "region_tagger" repo-wide was inside its own
module docstring usage example; no live imports found.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove empty meshai/cli package
Package contained only a docstring, no code. No imports of
meshai.cli found anywhere; console entry point is meshai.main:main
(pyproject.toml), and packaging (tool.setuptools.packages.find,
include = ["meshai*"]) has no explicit reference to the cli
subpackage. No MANIFEST.in/setup.py/setup.cfg exist to update.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove orphaned swpc/swpc_last golden fixtures
63 files (40 in fixtures/swpc/, 23 in fixtures/swpc_last/) never
loaded by any test. harness/goldens.py's load_fixtures(hazard) is
the only generic fixture loader and is never called with "swpc" or
"swpc_last" (only "avalanche", "nws", "nws_last", "quake"). The sole
remaining textual reference, a provenance comment in
test_swpc_refactor.py:162 ("Fixture mirrors swpc_last/0003.json..."),
is left untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: excise the dead Central NATS consumer path
Central was retired and its database dropped 2026-07-15; its NATS broker no
longer exists. Verified against the live CT108 deployment: all 12 adapters
run feed_source=native, zero on central, and central.enabled is False
(default, never overridden). The consumer and its handlers were unreachable.
Removed:
- central/consumer.py and 6 dead handlers (nws, quake, swpc, nwis, avy,
incident) -- their handle_* entrypoints were reachable only from the
consumer's dispatch
- the Central wiring in main.py (init, guarded start, retry loop, stop path)
- the dead config surface: CentralConsumerConfig, EnvironmentalConfig.central,
adapter_config ("central","severity_thresholds") and its display block
- the nats-py dependency (consumer.py was its only importer)
- 19 test files that exercised only the dead path
KEPT -- these live under central/ but are imported directly by native
adapters, and deleting them would break production:
- wfigs_handler.py: firms_handler._handle_pass_boundary() calls its _render()
on the live FIRMS growth-fire path (env/firms.py -> ingest_hotspot_pixel)
- firms_handler, satpass_handler, tle_handler: split files whose handle_*
entrypoints are dead but whose engines are live. Left intact; splitting
them is separate work.
- pass_predictor, budget, idaho_gauge_sites: fully live.
The usgs_quake keys global_mag_floor / regional_mag_floor / regional_centroid
/ regional_radius_mi / broadcast_pager_alerts are NOT removed despite comments
labelling them "CENTRAL-PATH ONLY" -- notifications/gating/quake.py reads them
unconditionally in the native path. Those comments are corrected separately.
Test-count note: the suite drops ~425 tests. Most were migration PARITY tests
whose sole purpose was proving the native rewrite byte-matched the Central
handler (golden byte-parity, cross-source identity, gate-sequence replay).
With the handler deleted there is nothing left to compare against, so they
cannot exist. Native-only tests were kept and reworked where a test reached
for a central symbol incidentally. This is a real coverage loss, accepted
deliberately: the parity harness proved the refactor faithful, and git
history preserves the originals.
Suite: 1984 passed, 6 failed -- the same 6 pre-existing failures as main
(stale SCHEMA_VERSION x3, expired TLE fixtures x2, one order-dependent),
all being fixed on fix/green-test-suite. No new failures.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(nws): restore native-only golden coverage for the wire formatter
Commit ca751fb5 deleted the Central nws_handler parity harness along with
the handler itself, which took the ONLY tests that pinned formatters.nws
.format()'s literal wire output. Gate-sequence and schema-conformance
tests already survived natively; the formatter's actual rendered text did
not have any native-only regression net.
Add TestFormatterGolden to test_nws_refactor.py: 3 real-fixture cases plus
6 hand-built pathological cases mined from the deleted test_nws_handler.py
(SVR path-sampling, the "no dangling separator" regression, TOR on-ground
vs radar-indicated, FFW flood-cause detection). Every literal was verified
by temporarily restoring the pre-excision central.nws_handler._render()
from git history (ca751fb5^) in a throwaway, uncommitted script, confirming
byte-identical output against the current native format() for all 37 real
fixtures (nws/ + nws_last/) and all 9 pathological cases, then pinning the
confirmed-matching string as the literal -- not a blind snapshot of
current behavior.
quake/swpc/avalanche/hydro/incident/fire were checked and already carry
equivalent native-only golden coverage (added directly in ca751fb5), so no
changes were needed there.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(persistence): derive SCHEMA_VERSION from the migrations directory
db.py hardcoded SCHEMA_VERSION = 26 while migrations/ had already reached
v29 (v27 dispatcher floor-drop counter, v28 mesh_observations, v29 IPAWS).
The migration runner globs the directory and applies every vN.sql it finds
regardless of the constant, so a fresh DB actually landed at 29 while the
constant claimed 26 -- a three-version drift that three tests were
correctly catching.
Derive it from the highest vN.sql present instead of bumping the literal,
so it cannot drift again the next time someone adds a migration. Falls back
to 0 if the directory is missing so import never fails; the migrations dir
sits alongside db.py and ships with the package (Dockerfile COPYs meshai/).
Adds a regression guard asserting the constant matches the highest
migration file, and updates three tests that hardcoded 26 as a literal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(tle): make TLE fixtures time-relative so they cannot expire
The ISS fixtures hardcoded epochs of 2026-06-30/07-01/07-02. tle_handler
sets STALE_DAYS = 14 and get_tle_by_norad() filters on epoch >= now - 14d,
so the fixtures silently aged out on 2026-07-02 and the tests began
failing -- a time bomb, not a regression.
Compute epochs relative to wall-clock now (base = now - 2d, +/-1d for
newer/older) with correct TLE epoch-field encoding and mod-10 checksum.
STALE_DAYS is untouched -- widening it in product code would have changed
production behavior to paper over a test bug.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(fire-tracker): pin config + history db to tmp_path
load_config() defaults HistoryConfig.database to the relative path
"conversations.db", resolved against the process CWD, so every test calling
load_config() with no override shares one file for the whole session. The
conftest DB-isolation fixture only covers MESHAI_DB_PATH, not this.
Point both the config dir and the history database at the test's tmp_path.
NOTE: this does NOT resolve the order-dependent failure -- the test still
passes standalone and fails in a full run, so the polluting state lives
somewhere other than config/history. Left failing rather than weakened;
root cause still unidentified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
County-only civil CAP alerts (SAME geocode, no <polygon>) had no geometry, so
the geometry-based region tagger could not place them: no region -> no
region_routes match -> silently not broadcast. Many CEMs / 911 outages / some
AMBER alerts are county-only.
- Bundle work/meshai/county_centroids.py: Census 2023 national county gazetteer
internal points (3,222 counties + DC/territories), FIPS->(lat,lon), with
SAME PSSCCC -> 5-digit FIPS helpers.
- env/ipaws.py: when a CAP alert has SAME geocode(s) but NO polygon, set a Point
(single county) or MultiPoint (multi-county) geometry from the county
centroid(s) so the EXISTING coverage/region tagger locates it and tags ALL
matching regions. Real polygon geometry always wins (never overridden).
- dashboard/server.py: call register_config_routes_hooks(app) in create_app so
the toggle auto-refresh middleware is actually wired in prod (was test-only);
saving a family toggle now takes effect live without POST
/api/notifications/refresh-toggles.
Tests: county-only alert tags SW Idaho + matches emergency route cell;
multi-county tags all regions; polygon path unchanged; create_app wires the
refresh middleware; panhandle coverage-gap documented.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The IPAWS civil-alert adapter shipped backend-complete but frontend-partial:
it rendered only in the generic adapter-config page and the Advanced (raw)
Data Feeds tab, and the `emergency` family was absent from the curated Data
Feeds panel, the family-settings toggles, and the MeshCore routing matrix.
Adapter (ipaws), benchmarked against firms/nws:
- Environment.tsx: add `ipaws` to AdapterKey union, EnvConfig interface,
META (native-only, keyless), a new `emergency` FAMILIES group, PANEL_META_KEY
(LLM toggle), and a hand-written renderSettings panel exposing base_url,
user_agent, tick_seconds, state_fips, same_codes, exclude_weather,
status_actual_only — with coverage-scope handling like the other adapters.
- Environment.tsx: IPAWS_DEFAULT backfill so pre-ipaws GET payloads don't crash.
- Dashboard.tsx: SOURCE_ICONS entry (Siren/IPAWS) so ipaws events aren't a slug.
- ActivityLog.tsx: TABLE_LABELS + CATEGORIES + text-hint so ipaws_alerts rows
show labeled "Emergency" and honor the category filter.
- dispatcher.py: _SOURCE_TO_TABLE fallback ipaws -> ipaws_alerts so region-routed
emergency sends land labeled in the audit feed (not NULL).
Family (emergency), benchmarked against fire:
- Notifications.tsx: add `emergency` to TOGGLE_FAMILY_META (Siren icon). This
cascades to Family Settings, the Meshtastic delivery matrix, and the MeshCore
routing matrix (the last was hardcoded to the static list and previously
omitted emergency entirely). Backend VALID_TOGGLES/gating/categories were
already complete — no backend family change needed.
Tests: update the _SOURCE_TO_TABLE exact-match guard and add an ipaws audit-row
parity test. Full suite 2407 passed / 6 pre-existing unrelated failures.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Saving the "Auto-advert interval" dropdown on the MeshCore Companion page
took BOTH radios offline on 2026-07-17 06:46:52. One click, full outage.
The page PUT a single-key body to /api/config/connection:
{"meshcore_advert_interval_seconds": 10800}
_dict_to_dataclass() builds kwargs only from the keys present in the body
and lets `cls(**kwargs)` default the rest, so every OMITTED field was reset
to its dataclass default and written to disk:
type: tcp -> serial (Meshtastic offline)
tcp_host: 192.168.1.100 -> <lost> (LOCAL_FIELDS, see below)
tcp_port: 4404 -> 4403 (wrong meshmonitor vnode)
meshcore_host: 192.168.1.253 -> '' (MeshCore off; blank = off)
meshcore_conn_type: serial -> tcp (wrong transport)
meshcore_serial_port: /dev/meshcore-rak -> '' (RAK radio lost)
It was silent twice over. `connection` is restart-required, so the running
process kept the good in-memory config while the file sat gutted, waiting
for any restart to detonate. And save_section() writes the domain file
FIRST and local.yaml SECOND: meshtastic.yaml hit the disk already gutted,
then the local.yaml write (which owns connection.tcp_host via LOCAL_FIELDS)
died on `[Errno 13] Permission denied` -- so tcp_host landed in neither
file, and the 500 that would have named the cause was swallowed by the UI.
The operator saw nothing happen.
This was never one page's bug: PUT /api/config/{section} was destructive on
a partial payload for EVERY section. Other callers only survive because they
happen to spread the full object first.
Fixes, in depth:
* Route (the durable fix): merge the body over the CURRENT live section
before coercing, so omitted keys keep their live values while present
keys -- including '' / False / [] -- still apply. The base is the live
config, the same values GET serves, so a partial PUT now lands exactly
where a full-object PUT from that same GET would. Full-object callers are
unaffected. Fixed at the HTTP boundary, not in _dict_to_dataclass():
absent-key-means-default is CORRECT at config-load time, where a file
legitimately omits fields it does not override.
* Nested semantics keyed off the dataclass schema, not "is it a dict":
nested dataclass fields DEEP-MERGE (a partial region_routes must not drop
sibling cells), while bare dict/list fields REPLACE at the key (cells,
toggles, destinations, rules are dynamic maps -- deep-merging them would
resurrect deleted keys and make deletion impossible, the mirror image of
the bug being fixed).
* Page: send the full connection object like every other caller does.
* Errors are visible: the save handler no longer swallows the exception,
and updateConfig() surfaces the server's `detail` rather than a bare
"API error: 500", which is what hid Permission denied from the operator.
* Default advert interval 10800 -> 86400 (24h). 3h is far too frequent a
default for a public mesh; the UI "(default)" label moves to match.
Tests: tests/test_config_partial_save_merge.py reproduces the outage with
the exact payload, and pins merge semantics across connection AND
notifications, intentional clearing, deep-merge, and map-deletion.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
self_info() reported host/port straight from config regardless of
conn_type, so a serial companion still advertised whatever stale
meshcore_host sat in the config — the API named a device meshai was not
talking to, which is enough to send an investigation to the wrong radio.
Connection details now come from one _connection_descriptor() shared with
connect(), so the log line and the API can't drift; only the live
conn_type's fields are populated and the rest are null.
meshai's device view is otherwise built once at connect and never re-read
— contacts via ensure_contacts(), channels via _enumerate_channels(). The
lib's contact handler only ever merges (meshcore.py::_update_contacts), so
a cached roster can never shrink, and a channel provisioned on the radio
stays invisible until the process restarts. There was no refetch path at
all. Adds an explicit resync that re-reads BOTH halves: a FULL
get_contacts(lastmod=0) reconciled with replace semantics (absent contacts
are dropped) plus a channel re-enumeration, each reporting what changed.
Also adds a preventive route-health check: every region_routes cell whose
MeshCore target cannot be resolved against the live roster/channel table
is surfaced, since such a send fails silently. Room targets are matched by
pubkey prefix, exactly as the dispatcher resolves them, so a picker-stored
prefix is not misreported as dangling. Same-name/different-pubkey roster
entries are flagged too — a name alone cannot identify a contact, which is
the trap behind a room rebuilt under a new keypair.
Backend:
- meshcore_roster.py: pure reconcile_contacts / check_route_health /
find_name_collisions (no device I/O — unit-testable without a radio)
- transport: _connection_descriptor, resync, refresh_contacts,
remove_contact, import_contact, export_roster, contacts_synced_at;
auto_update_contacts enabled (configurable — it costs one incremental
fetch per advert heard, which is real chatter on a dense mesh)
- API: POST contacts/refresh, DELETE contacts/{pubkey}, GET
contacts/export, POST contacts/import, GET route-health
Frontend (existing Contacts & Companion page — no new page or nav entry):
- dangling-route + name-collision banners; resync/export/add-contact
toolbar with last-synced and the added/removed counts; staleness badges;
search, filters and sortable columns; per-contact delete behind a
confirm; Companion tab shows the real transport + target.
A full pubkey is required to delete or add: the lib resolves by prefix,
and a prefix could silently hit the wrong node.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a new native `ipaws` adapter for FEMA IPAWS-OPEN EAS civil alerts.
- Two-stage CAP fetch via base_url (direct FEMA or Conduit proxy): Atom
index -> per-entry CAP 1.2 documents.
- Non-weather civil alerts only (evacuation, Civil Emergency Message,
AMBER, 911 outage, law-enforcement, HazMat); NWS/NOAA CAP dropped so
weather is never double-broadcast.
- Idaho + neighbour statefips scope gate applied before stage-2 fetch.
- Own `ipaws_alerts` dedup table (migration v29); reuses the NWS CAP
severity + formatter pattern.
- Ships enabled=False (no transmit until explicitly enabled).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
traffic and firms no longer idle when their key is blank — they build a
keyless request (traffic: omit key= param; firms: omit the map_key path
segment), matching roads511's existing optional-key pattern. Enables
routing these feeds through a key-injecting proxy (Conduit) with the key
held only there. Key-set behavior is byte-identical.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a base-URL config field (default = the current value, backward-compatible)
to the 9 adapters that hardcoded their upstream URL — nws, swpc (4 endpoints),
ducting, fires (perimeter+points), firms, avalanche, usgs streams (3 bases),
traffic, satpass/tle_fetch — mirroring the already-compliant roads511 pattern.
Every feed URL is now overridable via config, enforcing the "everything
configurable" rule and making each adapter live-repointable via a config PUT.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
EnvironmentalStore.apply_config() rebuilds only the changed native
adapters in place on a config PUT -- dedup/seen state (store-level) is
preserved, unchanged adapters are untouched. Drops "environmental" from
RESTART_REQUIRED_SECTIONS; falls back to restart-required only for the
narrow feed_source->central case. Cascades nifc/fires -> firms. The old
restart requirement was a Central-era coupling, now moot (all-native,
central.enabled=false, CentralConsumer inert).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ReminderScheduler is the third fire-broadcast exit and the only one
that does not pass through the EventBus, so FirePacer (which paces the
Central and native fire-event exits to <=1/60s) never sees it. Its tick
is a roll-call: every eligible row is its own broadcast, dispatched in a
plain `for` loop with no gap. Unpaced, N eligible fires produce N
back-to-back mesh transmissions; the only downstream protection is
RadioSendQueue's ~2.2-2.6s per-transport inter-packet jitter, which
prevents packet collision but still lets a roll-call monopolise the mesh.
Not currently firing in production (every overdue fire is filtered by
terminate_when, so the eligible set is 0) -- this is fire-season
hardening against a latent burst, not a live incident.
Adds `spacing_seconds` (adapter_config, default 60 to match FirePacer)
enforcing a minimum gap between consecutive SUCCESSFUL reminder
deliveries. Deliberately a pure spacing change:
* WHAT gets broadcast is untouched; nothing is dropped.
* The ok-gated last_broadcast_at stamp still uses the tick's `now`.
* A failed dispatch sent no packet, so it does not arm the gap.
* Rows filtered by terminate_when/render never burn a spacing slot.
* A lone eligible fire has nothing to pace against -> zero added latency.
* The wait is interruptible by stop(): a 15-fire roll-call holds
tick_once() for ~14 min and stop() awaits the tick task, so a plain
sleep would stall shutdown.
Chose in-loop spacing over routing reminders through FirePacer itself:
reminders re-derive their targets from live DB state every tick and only
clear a row via last_broadcast_at after a confirmed send, so enqueuing
into a 60s-drain FIFO would re-enqueue the same fire on every intervening
tick -- the queue would grow faster than it drains. pacer.py, consumer.py,
store.py and main.py are untouched.
Tests fake the clock end-to-end, so 60s spacing costs the suite nothing.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gating/firms.py's decide() stamped a plain "severity" key in the
data_patch for the wildfire_spotting and wildfire_halted broadcast
paths. central/consumer.py only ever promotes data["_severity_override"]
onto Event.severity -- the plain key is a silent no-op, the same class
of bug as #118 (fixed for firms_handler.py's own inline stamps in
PR #120). Currently inert (MESHAI_CUTOVER_CATEGORIES is unset by
default), but the moment wildfire_spotting/wildfire_halted are cut
over, spotting would silently stop being "immediate".
- gating/firms.py: both data_patch sites now use _severity_override.
Checked the other gating modules (fire.py, avalanche.py, swpc.py,
quake.py, nws.py) -- all already use _severity_override correctly;
firms.py was the only one with the plain-key mistake.
- Fixed the stale module docstring claiming the unattributed-hotspot
cluster path "is DEAD" -- it has been live since d479ca53 (#73); the
stale comment directly caused a bogus bug report against production.
- test_firms_refactor.py: updated two existing tests that had codified
the buggy plain-"severity" behavior as expected, and added
TestCutoverSeverityReachesEvent, which drives the real cutover path
end-to-end through CentralConsumer._normalize and asserts the
emitted Event's severity (immediate for spotting, routine for halt).
Verified both new tests fail against the unfixed decider and pass
against the fix.
Full suite: 20 failed, 2242 passed, 72 skipped (vs. origin/main
baseline 20 failed, 2240 passed, 72 skipped -- same 20 pre-existing
failures, confirmed identical with this change stashed out; +2 passed
are the new regression tests).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
MeshCoreTransport defined _resolve_contact TWICE:
- L171 (PR #56) cache lookup -> on miss, get_contacts(lastmod=0) full
refetch -> retry. The DM / path-establishment resolver.
- L1227 (PR #92) key-prefix -> by-name lookup, no refetch. The telemetry
resolver, added later without noticing the collision.
Python silently keeps only the LAST definition in a class body, so the
line-171 implementation was dead code and PR #56 was nullified: every DM
and path-establishment caller was getting the telemetry resolver instead.
No error, no warning, invisible to the linter and the type checker.
Fix: rename the telemetry resolver to _resolve_contact_for_telemetry and
repoint its sole caller (_req_telemetry_async). The DM path (send_message)
and _establish_direct_path now get PR #56's refetch-on-miss behavior back,
which is what they need — replying to an inbound DM from a firmware
auto-added contact requires the refetch, and re-resolving after path
discovery is pointless without it.
Deliberately NOT merged into one resolver: telemetry auto-polls on a timer
against operator-selected contacts already in the roster, so a full-roster
refetch on every miss is recurring airtime for nothing; and its by-name
fallback is telemetry-specific and must not widen DM address resolution.
The two want different semantics — the bug was the name collision, not that
they should be one function.
_resolve_contact_async (the MC-event-loop twin) already carried the refetch
and was never shadowed, so the async/queue DM send path was unaffected.
Add tests/test_no_duplicate_methods.py: AST-walks every ClassDef under
work/meshai/ and fails if any class body defines the same method name twice.
This failure mode is invisible to review, the linter, and the type checker —
which is exactly why it survived. Exempts the legitimate same-name patterns
(@property/@setter/@deleter groups, @overload stacks). Verified it flags the
bug on the pre-fix source and finds no other duplicates in the tree.
Suite: 20 failed -> 17 failed (the 3 meshcore failures gone), 2240 -> 2245
passed (+3 fixed, +2 new guard tests), 72 skipped unchanged.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
wildfire_spotting is the most urgent signal in the system (a fire
throwing embers past its own containment line), but every fire event
carries a group_key (= event_id), so the Grouper held spotting for the
full grouper_window_seconds (60s live) before it could even reach the
dispatcher -- a ~60s latency FLOOR, and ~120s+ once FirePacer queuing
is added on top.
Add a module-level _NEVER_COALESCE_CATEGORIES frozenset and bypass the
coalescing window for the categories in it. Currently: wildfire_spotting
only.
This is safe because spotting is already rate-limited AT THE SOURCE: a
per-fire 1h cooldown (adapter_config.fires.spotting_cooldown_seconds,
default 3600, latched on fires.last_spotting_broadcast_at per irwin_id
in gating/firms.py) gates spotting DETECTION itself, so N active fires
yield at most N spotting alerts per hour. FirePacer (60s interval, with
head-of-line ordering for immediate severity) and the dispatcher's
per-(toggle, category, region) cooldown still apply downstream.
The bypass is scoped by CATEGORY, never by severity. Commit 85d48ce3
deliberately removed a severity == "immediate" bypass from this exact
spot because ALL fire events carry _severity_override="immediate", so a
severity bypass exempts the entire fire family from rate control. The
comment on the constant spells that out so it does not get re-added.
Tests: spotting with a group_key passes straight through; wildfire_growth
and wildfire_incident at immediate severity are STILL held (proving no
severity bypass crept back in); spotting with no group_key still passes
through. PR #129's test_immediate_severity_is_also_coalesced_no_bypass is
untouched and still passes.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Triaged all 20 known-red tests. 13 were stale tests asserting rotted
expectations against deliberate, documented behavior changes; fixed by
deriving expected values instead of hard-coding, or updating the
expectation to match a documented policy change:
- test_adapter_config_foundation.py / test_adapter_config_api.py:
REGISTRY/API key-count and key-set guards hard-coded magic numbers
(59/94/17) that rotted repeatedly. Now derive expectations from
REGISTRY itself and, for the schema version, from the migrations
directory, so they can't rot the same way again.
- test_fire_tracker_phase4.py: two tests hardcoded a nonexistent
deployment path (/opt/meshai/meshai/router.py) that matches no
Dockerfile WORKDIR in this repo; resolve the module path via
importlib.util.find_spec instead.
- test_tombstone_broadcast.py: asserted fire severity == "immediate",
which commit 2f677e85 deliberately downgraded to "priority" (to stop
fire broadcasts bypassing the Grouper/cooldown during NATS backlog
replay) without updating this test.
- test_pipeline_grouper.py: test_immediate_severity_bypasses_grouper
asserted an immediate-severity bypass that commit 85d48ce3
("fix(fire): remove immediate-severity exemption from grouper +
cooldown") DELETED on purpose -- fire events carry
_severity_override="immediate", and the exemption left fire with no
rate control at all in normal live operation. Re-adding the bypass
would re-open that fire-spam hole on a public-safety mesh, so the test
moves, not the source. Renamed + inverted to assert the real contract
(all severities coalesce; only a missing group_key passes through).
- test_tail_followups.py: dispatcher mock was missing
dispatch_scheduled_fire_broadcast (a method added alongside the
generic dispatch_scheduled_broadcast; test_reminders.py already
mocks both).
- test_tracking_v057.py: guard required an empty tracking-family
adapter list in Environment.tsx, but the frontend has long grouped
the pre-existing native satpass adapter under the "Tracking" display
section (its own "satpass" backend toggle, unrelated to the Phase-7
tracking family every other guard in this file confirms is still
unimplemented). Narrowed the guard to allow only that known entry.
- test_v052_dispatcher.py: two tests used category="wildfire_incident",
which the phase3b fire migration (#33) forced onto a dedicated
formatter via NATIVE_ALWAYS_DECIDE; swapped to wildfire_hotspot
(same emoji/label, not in NATIVE_ALWAYS_DECIDE) to keep exercising
the generic composer logic under test.
Also fixes one stale COMMENT (comment-only, no logic change) in
meshai/notifications/pipeline/__init__.py's start_pipeline(): it still
claimed "Immediate events bypass the grouper and don't need this
[periodic flush]", which has been false since 85d48ce3 and is precisely
what makes the deleted bypass look like a missing feature. The comment
now records that the removal was deliberate and must not be reverted.
The remaining 6 failures are left untouched -- 2 confirmed real bugs, to
be fixed deliberately in their own changes:
- meshcore_transport.py defines `_resolve_contact` TWICE on
MeshCoreTransport (line 171 from PR #56, line 1227 from PR #92). The
second silently shadows the first, so the DM contact-resolution
refetch-on-miss that #56 added is dead code in production. (3 tests)
- SCHEMA_VERSION (persistence/db.py:33) is stale at 26 vs. the actual
highest migration v28; v27 and v28 shipped without bumping it.
(3 tests)
Plus 1 environment gap, not a code defect:
test_natural_language_fire_question_routes_to_llm needs the `openai`
package, which is declared in requirements.txt but not installed here.
Suite: 20 failed, 2240 passed, 72 skipped -> 7 failed, 2254 passed, 72
skipped. All 7 remaining failures are ones classified above; no new
failures introduced elsewhere in the suite.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Native fire adapters (env/fires.py source="nifc", env/firms.py
source="firms") emitted straight to the EventBus from
EnvironmentalStore._emit_event with no rate limiting of their own.
FirePacer was only ever attached to CentralConsumer (main.py), which
never runs in the actual production deployment (central.enabled=False,
all adapters feed_source=native) -- so the <=1/60s throttle + immediate
head-of-line behavior fixed for Central in #120 (issue #119) was
completely inert in production. A poll that produces several distinct
fires/clusters at once (a lightning outbreak, or several tracked fires
crossing a satellite-pass boundary together) would dump all of them on
the mesh back-to-back instead of at the intended cadence.
_emit_event() now routes fire-family Events (source in
{"nifc","firms"}, severity in {"priority","immediate"}) through an
attached FirePacer, mirroring the exact gate CentralConsumer._handle
applies. main.py attaches the same FirePacer instance to env_store
right after constructing it. "routine"-severity fire events, non-fire
native adapters, and the Central path are all unaffected; a paced event
cannot re-enter either gate (native vs Central are mutually exclusive
per feed_source), so nothing can be paced twice.
Added tests/test_native_fire_pacer.py covering: native fire events
route through the pacer, an immediate event jumps an already-queued
priority queue with nothing dropped, routine-severity fire events and
non-fire native events are NOT paced, and the no-pacer-attached
fallback is unchanged.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Three independent bugs kept firms_handler's growth/spotting/halt/cluster
fusion decisions from reaching a correct mesh Event:
- #117: consumer._normalize() computed `category` from the raw Central
category BEFORE the per-adapter handler ran and never re-read
data["category"] afterward, so every firms_handler category stamp was a
silent no-op. Now re-read post-dispatch, validated against the known
category registry (unrecognized overrides are logged and ignored).
- #118: consumer.py only ever honors data["_severity_override"], but
firms_handler's halt/spotting/cluster sites stamped the plain
data["severity"] key instead (only growth used the right key). Switched
all three sites to `_severity_override` for one consistent contract.
This is severity plumbing only -- it does not change which events fire.
- #119: FirePacer's gate only matched source in ("fires","wfigs") at
severity=="priority", so FIRMS fusion broadcasts (source="firms",
growth/spotting at "immediate") never reached the pacer. Broadened the
gate to cover "firms" + {"priority","immediate"}, and gave FirePacer
head-of-line insertion so an "immediate" event is never stuck behind
already-queued "priority" events. Still unbounded/never-drops.
Cluster detection is left exactly as main ships it: live, always on, no
toggle (PR #73's curated new-fire cluster broadcasts with cold-start
silent-seeding). Only its severity-override key changes, under #118.
Updated existing tests that asserted the old (buggy) data["severity"]
contract, and added tests/test_firms_fusion_event_contract.py covering
all three fixes end-to-end through consumer._normalize()/_handle() and
FirePacer directly.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Section 1.5 (region_routes matrix branch) armed and checked the
per-region cooldown key as (toggle, category, region) with no
channel-type component. _chans always inserts mesh_broadcast before
meshcore_broadcast (insertion order in the per-cell append loop), so
for any matched cell with BOTH mt and mc populated, the mesh_broadcast
send armed the cooldown key first; the very next iteration checked
that SAME key for meshcore_broadcast and saw it as freshly cooled
down, dropping it every time. Net effect: meshcore_broadcast never
succeeded via the matrix branch whenever cooldown_seconds > 0 (true
for weather/roads/fire, all 300s), so it never armed its own
dedup/cooldown state either -- a silent, permanent MC blackout for
every region-routed family. Confirmed live: dispatcher_dedup had 555
rows, zero meshcore_broadcast; mesh_broadcasts_out was 120:4 MT:MC for
nws_alerts and 98:8 for traffic_events over 10 days (the few MC rows
that got through came from a different, non-matrix code path). fires'
98:77 near-1:1 ratio is not evidence the matrix branch worked for
fire -- those MC sends are dominated by the cooldown-exempt scheduled
reminder path (dispatch_scheduled_fire_broadcast); fire's own live
event-driven path has the identical latent bug, just masked.
Fix folds ch_type into the region string (mirrors the existing
_cd_suffix convention) so mesh_broadcast and meshcore_broadcast get
independent cooldown windows. Kept the cooldown key a 3-tuple
(instead of widening to 4) to avoid a dispatcher_cooldowns schema
migration -- _persist_cooldown() and the boot-restore SELECT are both
hard-coded to (toggle, category, region).
Verified in isolation (no live/deployed behavior change, no
transmit): a fresh Dispatcher built from the live production config
now dispatches both mesh_broadcast and meshcore_broadcast for
weather/roads/fire matched cells under a 300s cooldown. Added two
regression tests covering the gap that let this ship untested: no
existing test combined cooldown_seconds > 0 with a cell that has BOTH
mt and mc populated (test_cell_match_routes_mt_and_mc uses the
cooldown_s=0 default; test_per_region_cooldown_independence uses
cooldown_s=300 but with mc=None on every cell).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends MeshCore routing so a region_routes `mc` cell can target a room
server, not just a `#`-channel. A room = a contact with type==3; a cell
value of `room:<pubkey>` routes to it via the existing addressed DM path
(send_msg to the room's pubkey), with an optional login for
password-protected rooms; a bare cell value stays a channel broadcast
(unchanged). Adds transport get_rooms()/login_to_room()/send_to_room_async,
a GET /api/meshcore/rooms endpoint, per-room password storage in
secrets_store (env MESHCORE_ROOM_<prefix>_PWD), and a routing-GUI
channel-vs-room picker (rooms shown by name with a path indicator).
Open rooms work end-to-end. Password-protected rooms need a follow-up:
a backend endpoint to SET the per-room password from the GUI (the generic
secrets API is allowlist-gated); the storage + login already exist.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The WZDx daily summary + DM query count from traffic_events, but work
zones weren't landing there: wzdx rode the generic _delta_emit path,
which silent-seeds the seen-set and returns before the decider's INSERT
on the cold-start first poll, so the current zone set never persisted
(summary would count ~0). Add a dedicated _ingest_wzdx (mirroring the
fires ingest) that UPSERTs every current coalesced zone into
traffic_events each poll (persist-only, last_broadcast_at=NULL, no emit,
no broadcast) and reconciles zones that drop out of the feed (never wipes
on an empty/failed fetch). Per-event work-zone broadcast stays suppressed
(the decider's work_zone gate is untouched). Retargets 4 tests in
test_store_received_delta.py that used a fake 'wzdx' source to exercise
the generic gate onto a neutral routing name.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1. WZDx work zones: replace per-event broadcasting with a once-a-day
per-region count summary. Coalesce the upstream per-direction /
per-schedule-day fan-out into one row per physical zone
(road + lat3 + lon3 + sub_type); work zones are stored in
traffic_events but no longer per-event broadcast, while 511 crash /
closure / hazard incidents still broadcast live. A WZDxSummaryScheduler
emits one count line per coverage region once a day (default 07:00
America/Boise), routed via the region_routes 'roads' cells; work-zone
details are DM-queryable (build_work_zones_detail). New config:
wzdx.summary_enabled / summary_time / summary_tz.
2. FIRMS cold-start is now restart-safe: gate the silent-seed on the
persisted firms_pixels baseline being empty (first-ever run) instead of
an in-memory per-boot flag, so a restart no longer silently absorbs a
genuinely-new hotspot cluster.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two fire-scope cleanups:
1. Remove the fire digest feature entirely -- scheduler
(notifications/scheduled/fire_digest.py), pipeline wiring, the
fires.digest_* adapter_config key registrations, and the Fire Digest
dashboard UI (ScheduledBroadcasts / Environment / Reference /
AdapterConfig / ActivityLog). The unrelated generic per-rule
notification digest is kept. Orphaned fires.digest_* config rows and the
fire_digest_broadcasts table are left as inert data (v16 migration
untouched).
2. Add a coverage-scope gate at fire ingest: _ingest_fires now skips any
fire whose coordinates fall outside all configured coverage areas (same
areas_from_config + classify_geom_areas membership the dispatch-level
CoverageFilter uses), so out-of-coverage fires are never stored, tracked,
alerted, reminded, or re-ingested. Fails open when coverage is disabled,
has no areas, or excludes the fires adapter.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fire (wfigs) reminders previously dispatched via the generic scheduled
path, which hardcoded the rf_propagation toggle (Meshtastic ch4 / MeshCore
#aida) and ignored the fire's region. They now route through a new
dispatch_scheduled_fire_broadcast() that builds a synthetic fire event from
the fire's lat/lon, derives its region the same way the live fire event
path does, and routes per region_routes.cells['fire'] (per-region MT/MC
channels), falling back to the fire toggle's own defaults when a transport
isn't matrix-owned -- never rf_propagation. rf_propagation and 511
reminders are unchanged. Reminders remain disabled (reminders_wfigs.enabled
stays false); this only fixes routing for when they are enabled.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Split the single region_routes.enabled master switch into per-transport
mt_enabled (Meshtastic) and mc_enabled (MeshCore) flags so the two
transports can be region-routed independently. Previously the shared
switch forced MeshCore into the region matrix; with all mc cells null it
routed MeshCore nowhere instead of falling through to the toggle-level
meshcore_channel. The dispatcher is now authoritative per-transport: a
disabled transport falls through to its toggle path, and matched-but-
inactive cells still suppress the toggle for enabled transports. The
destinations delivery branch also honors matrix-handled suppression to
prevent double-broadcast. Loader maps legacy enabled:true to
mt_enabled:true, mc_enabled:false. Each GUI routing page gains its own
master enable toggle.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backend: add _SOURCE_TO_TABLE class constant in Dispatcher mapping event.source
("nws", "nifc", "wzdx", "traffic", "511") to canonical audit table names.
_post_broadcast_commit now falls back to this map when _broadcast_audit is
absent/None, so native env adapter sends (nws.py, fires.py, wzdx.py,
roads511.py, traffic.py) write a non-NULL source_event_table instead of NULL.
Existing _broadcast_audit paths (Central handlers, scheduled broadcasts) are
unchanged.
Frontend: replace naive familyLabel() string transform with explicit
TABLE_LABELS lookup (10 known tables → friendly names) plus a TEXT_HINTS
emoji-prefix heuristic for legacy NULL-source rows, so historical orphan rows
still display a meaningful label before/after backfill.
Tests: three new unit tests in test_dispatcher_persistence.py covering the
fallback path (nws→nws_alerts), the full _SOURCE_TO_TABLE map, and that
explicit _broadcast_audit is never overridden by the fallback.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Add WFIGS_Incident_Locations_Current point layer (IRWIN superset, ~6 ID fires)
alongside the existing perimeter layer (~2 ID fires). Fires are merged by IrwinID:
point layer is the authoritative superset, perimeter layer supplies polygon
geometry and validated acreage when available. Point-only fires surface with
lat/lon from the point geometry and no polygon. cold-start silent-seed path is
unchanged (first-poll batch is always silent regardless of source). Perimeter
fetch failure falls back to perimeter-only stubs; point fetch failure falls back
to perimeter-only; both failing bumps the consecutive error counter. county is
now populated from POOCounty on the point layer. Five off-air unit tests cover:
T1 merge-dedup, T2 point-only-new, T3 perimeter-geom-preferred, T4 cold-start
silent-seed 6 fires, T5 FIRMS _get_known_fires attribution.
No changes to store.py, gating/fire.py, schema, or coverage.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
CHANGE 1 (store.py): after applying gate.data_patch into event.data, promote
_severity_override and category keys onto event.severity / event.category.
Previously, decider overrides (e.g. fire: "priority" on every New/Update)
landed only in event.data, leaving event.severity at the adapter's raw value
("routine" for fires >=25 km from an anchor). This silently failed the
toggle/matrix min_severity floor. Native and Central now share identical
broadcast decisions at the shared choke point.
CHANGE 2 (dispatcher.py + v27 migration): both the toggle-path and
matrix per-cell severity-floor drop paths now emit a WARN log and
increment a new persisted counter (severity_floor_dropped) in
dispatcher_state, following the existing drop-counter pattern exactly.
v27.sql adds the column; the counter restores on restart and appears
in dispatch_stats().
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Defect A (audit gap): _post_broadcast_commit only wrote a mesh_broadcasts_out row
when event.data["_broadcast_audit"] was a dict. Native traffic/weather/roads events
from native adapters never set that key, so every matrix-dispatched send was
invisible in the audit table even when the dispatcher logged success. Fix: write
the audit row for every mesh delivery attempt (ch_type in _MESH_CH_TYPES),
unconditionally. source_event_table/source_event_pk come from _broadcast_audit when
present, else NULL (best-effort). The early-return on empty data is preserved only
for the _on_broadcast_committed callback, not for the audit write.
Defect B (channel routing): full trace of the send path confirms the channel IS
correctly threaded from the matrix cell through _toggle_to_rule (mt_override) →
create_channel(channel_index=rule.broadcast_channel) → MeshBroadcastChannel
(self._channel) → send_message_async(channel=self._channel) → CompositeTransport
→ MeshtasticTransport send_queue job closure → _blocking_mt_send(channel) →
sendText(channelIndex=channel). No code bug: the correct channel index reaches the
radio. The missing audit rows (Defect A) prevented confirming this from the DB.
/api/channels fix: the endpoint read connector._interface which does not exist on
CompositeTransport (only on bare MeshtasticTransport), so it always returned []
when MeshCore was also configured. Fix: detect CompositeTransport and route to
meshtastic_child()._interface instead.
Tests added: matrix send without _broadcast_audit writes audit row with correct
channel+transport+success; failed delivery writes success=0 row; matrix cell
channel index reaches _blocking_mt_send end-to-end (queue path exercised);
/api/channels returns real channel list via CompositeTransport.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(transport): per-radio serialized+paced outbound send queue
Prevents simultaneous LoRa transmissions when N events arrive at once.
## Mechanism
Two `RadioSendQueue` instances (one MT, one MC), each a FIFO asyncio.Queue
with a long-running drain task. The MT queue drains on the main asyncio
loop; the MC queue drains on MeshCore's dedicated event-loop thread.
- MT sends: `run_in_executor` offloads the blocking `sendText` call;
queue started in `set_message_callback`, cancelled in `disconnect`.
- MC sends: drain loop runs pure-async MC lib coroutines directly on the
MC loop (no `_run_coro` deadlock); cross-loop callers bridge via
`concurrent.futures.Future` + `asyncio.wrap_future`.
- Pacing: `await asyncio.sleep(pacing_seconds)` between items; read live
from config per iteration; floor clamped to 0.25 s.
- Config knobs: `meshtastic_send_pacing_seconds` (default 2.0) and
`meshcore_send_pacing_seconds` (default 2.0) on `ConnectionConfig`.
## Send sites rerouted
All callers now `await connector.send_message_async(...)`:
- `notifications/channels.py` — MeshBroadcast/MeshCoreBroadcast/MeshDM/
MeshCoreDM deliver(), test_connection(), deliver_test()
- `responder.py` — DM replies in send_response()
- `transport/meshcore_transport.py` — periodic_advert_loop, telemetry
poll loop, send_advert() → send_advert_async(), req_telemetry()
→ req_telemetry_async() (all queue-routed from main loop)
- `dashboard/api/mesh_send_routes.py` — test-send, advert, telemetry poll
## Audit accuracy
`deliver()` now returns the actual bool from the radio send (not
optimistic True), so `mesh_broadcasts_out` reflects the real result.
## Tests
17 new tests in tests/test_send_queue.py covering FIFO ordering, no drops,
pacing gap, pacing floor enforcement, event-loop non-blocking, serialization,
lifecycle, MT fallback, config round-trip. Existing test stubs updated to
wire `send_message_async = AsyncMock(side_effect=send_message)` so prior
call_count / call_args assertions remain valid without changes.
Full suite: 2135 passed, 17 pre-existing failures (unchanged), 0 new regressions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(send-queue): resolve MC telemetry self-deadlock + resolve pending futures on teardown/reconnect; composite MC-channel kwarg; audit no-op false
BLOCKER 1 — req_telemetry_async self-deadlock (meshcore_transport.py):
_req_telemetry_async was calling _enqueue_mc_loop_send inside itself;
when _telem_job_outer ran inside the drain it nested another enqueue+await
on the same single-threaded drain — permanent deadlock on first telemetry poll.
Fix: _req_telemetry_async is now fully inline (no _enqueue_mc_loop_send).
_telemetry_poll_loop wraps its call in _enqueue_mc_loop_send for serialization.
req_telemetry_async's outer job calls _req_telemetry_async inline (safe).
BLOCKER 2 — pending futures abandoned on teardown/reconnect:
RadioSendQueue.stop() only cancelled the drain task; queue-sitting items had
their concurrent.futures.Futures left unresolved, causing wrap_future() callers
to hang indefinitely. Fix: stop() drains the remaining queue with get_nowait()
and cancels every pending cfut. _cancel_mc_queue() schedules the same drain-
and-cancel via call_soon_threadsafe. _start_mc_queue() cancels old drain task
and drains old queue cfuts before arming the new queue (reconnect path).
connector.disconnect() now .result(timeout=5) on stop() instead of fire-and-forget.
SHOULD-FIX 3 — composite passes MC channel as wrong kwarg (composite_transport.py):
_broadcast_async no-hint loop was calling send_message_async(channel=child_channel)
for the meshcore child; should be meshcore_channel=child_channel. Silent drop fixed.
NIT 5 — false success on zero-channel MC send (meshcore_transport.py):
send_message_async returned True when meshcore_channel is None (nothing sent).
Now returns False so audit does not record a success for a no-op.
NIT 7 — config comment contradiction (config.py):
meshtastic_send_pacing_seconds comment said "0 disables the floor" while
simultaneously stating "still floored at 0.25". Removed the contradiction.
Regression tests (tests/test_send_queue.py — 3 new, all in TestDeadlockRegression):
- test_telemetry_queue_no_deadlock: drives req_telemetry_async through a real
_mc_send_queue with fake MC commands; times out on pre-fix code (deadlock).
- test_teardown_resolves_pending_futures: enqueues slow+fast jobs, stops mid-drain,
asserts every task resolves promptly; hangs on pre-fix code.
- test_reconnect_resolves_old_futures: calls _start_mc_queue twice, asserts old
cfuts are cancelled; pre-fix leaves them unresolved.
All 17 pre-existing send-queue tests still pass (20 total now).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(meshcore): opt-in telemetry auto-poll on selected contacts
req_telemetry + a poller for selected contacts (meshcore_telemetry_contacts,
interval with a min floor, availability detection). Contacts page gains
per-node auto-poll toggles + battery/sensor readouts + Poll-now, and maps
numeric contact type codes to Chat/Repeater/Room/Sensor badges.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(meshcore-telemetry): reconcile with current transport/meshcore-lib API
- Add EventType.ACK + NEW_CONTACT to the telemetry test's fake module;
_setup_subscriptions() subscribes to both (added in main before rebase)
and the stale stub caused all three TestPollerScheduler tests to abort
with AttributeError on connect().
- Same NEW_CONTACT gap fixed in test_meshcore_conn_type.py and
test_meshcore_dm_delivery.py — these ran first (alphabetically) via
setdefault, contaminating the shared sys.modules["meshcore"] stub for
all downstream test files and causing 9 extra connect()-path failures
suite-wide (TestPeriodicAdvertScheduler, TestAdvertOnConnect, etc.).
- req_telemetry_sync(contact, min_timeout=5) matches the installed lib
(meshcore-2.3.7 binary.py) exactly — no production-code change needed.
- All 30 test_meshcore_telemetry tests pass; full-suite failures drop
16 → 7 (remaining 7 are pre-existing, unrelated to telemetry).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
