mirror of
https://github.com/zvx-echo6/meshai.git
synced 2026-08-26 17:31:34 +00:00
170 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Matt Johnson
|
c2fd4d7131 |
feat(dashboard): custom-announcements frontend (Announcements panel)
Extends the existing ScheduledBroadcasts page with an Announcements
section (list + create/edit form) over the announcement_routes.py API
from
|
||
|
Matt Johnson
|
c0572b04a1 |
feat: user-crafted scheduled announcements (custom_announcements)
Free-text broadcasts the owner types straight into the GUI on a clock-slot schedule (daily / interval_days / weekly / monthly) -- no placeholders, no data sources, no templating, no SQL from the user. - v30 migration: custom_announcements table (own explicit channel list per row, new rows start disabled). - CustomAnnouncementScheduler (notifications/scheduled/custom_announcements.py): 60s tick modelled on ReminderScheduler; monthly day-of-month clamped via calendar.monthrange; restart-safe dedup keyed on the local calendar date of last_sent_at; spacing_seconds roll-call pacing between announcements firing in the same tick. - Dispatcher.dispatch_scheduled_custom_broadcast(): delivers to the announcement's own channel list (no toggle/region_routes matrix), one mesh_broadcasts_out audit row per target, cold-start grace. - New announcement_routes.py router: GET/POST/PUT/DELETE /api/announcements + POST .../preview (wire text + char/byte count, never sends). No send-now endpoint anywhere. - Wired into notifications/pipeline/__init__.py (alongside ReminderScheduler) and dashboard/server.py; single_packet_max_chars runtime override added in main.py's budget list. 57 new tests (recurrence kinds, Feb-29/28 clamp, restart-safe dedup, pacing, budget truncation, full input validation, multi-target audit rows, no send-anywhere guarantee). Full suite: 2090 passed, 0 failed (up from 2033 baseline), 9 pre-existing warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Matt Johnson
|
2eddd9b572 |
fix(reminders): resolve Active: fire location via _fire_anchor, not county/state
ReminderScheduler._render built the wfigs "Active:" reminder location as a
bare " / ".join(county, state), skipping the anchor resolution (geocoder_city
-> curated town_anchors -> Photon -> landclass -> county -> state) that the
New/Update fire formatter (formatters/fire.py::_fire_anchor) already uses.
Reminders now call the same _fire_anchor helper on dict(row) (sqlite3.Row has
no .get()) and fit the result to fit_to_budget(..., budget_for("wfigs")),
matching how the other renderers guard the mesh packet budget.
|
||
|
Matt Johnson
|
522458f194 |
fix: sync _TOWN_ANCHORS_SEED with the production town_anchors table
_TOWN_ANCHORS_SEED covered only 29 hand-picked towns, while the live CT108 town_anchors table (GUI-curated since) has grown to 186. Any fresh deploy (fresh volume, or the pre-v19 sqlite path) would seed only the original 29 and silently lose anchor-resolution coverage for the other 157 real, already-in-production towns. Regenerated the dict from a live dump of CT108's town_anchors (name, lat, lon, state; all 186 rows confirmed enabled=1), alphabetized, same dict-of-dicts shape and column-aligned brace formatting as before (alignment column widened to fit the longest name, "mountain home afb"). seed_town_anchors() is unchanged and still INSERT OR IGNORE, so it stays idempotent against the existing production rows. Expanding the seed changes which town resolves as "nearest" for a few existing test fixtures whose incident/work-zone coordinates are genuinely closer to a newly-added real town (e.g. Oakley, Wilder) than to the old 29-town subset's nearest match -- those goldens and assertions are updated to the new (correct) nearest-town result. test_api_post_add_town's probe town is renamed from the now-real "Bellevue" to a fictitious "Testopolis" to avoid a duplicate-name 400. |
||
|
Matt Johnson
|
3961f7ec04 |
fix: honour town_anchors.enabled in alert anchor-resolution queries
resolve_anchor() (notifications/formatters/_anchor.py) and _location_anchor() (env/fire_render.py) selected all town_anchors rows regardless of the enabled flag, so a disabled anchor could still be used in outbound alert text -- only the dashboard/curation routes are meant to see disabled rows. Add AND enabled = 1 to both queries. No-op today: all 186 live town_anchors rows are enabled=1. Matters the next time someone disables an anchor from the curation UI. Also updates the traffic_last/0003.json wzdx golden literal in test_incident_refactor.py and adds TestAnchorResolve:: test_disabled_anchor_excluded, which forces the Photon fallback to miss and asserts a disabled-only DB row is not selected. |
||
|
Matt Johnson
|
bdbc2afe7f |
MeshCore reconnect persistence: implement 0=unlimited max-reconnect-attempts sentinel
connect() now translates a configured meshcore_max_reconnect_attempts of 0 (or <=0) into an effectively-unbounded count before handing it to the meshcore library's ConnectionManager, so its retry loop never permanently exhausts. config.py's comment already documented \"0 = unlimited\" but that sentinel was never actually implemented -- literal 0 meant zero attempts, and the shipped default of 5 (at ~1s/attempt) gave up after ~5s with no external supervisor to retry again, leaving MeshCore dead until a manual container restart. Also flips the repo default from 5 to 0 so fresh deploys get unlimited retries without extra config. Proven via a 60s forced-outage auto-recovery test: the link recovers from any-length vnode/radio outage instead of giving up after ~5s. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Matt Johnson
|
c5aa0e1f42 |
Fix event-loop starvation, MeshCore stability, config-page hardening
- mesh_data_store.py / env/store.py: make refresh() async, offload blocking polls via asyncio.to_thread/gather so 7 lockstep sources no longer starve the shared event loop. - main.py: gather pollers concurrently + set_default_executor thread pool. - Dockerfile / docker-compose.yml: healthcheck now curls the dashboard for a real liveness signal instead of a process-exists check. - transport/meshcore_transport.py: MeshCore keepalive loop (get_time() every 120s), reconnect re-arm (_post_reconnect_setup_async from _on_connect_event), and MC channel-name normalization (_resolve_mc_channel_idx strips a leading #). - dashboard-frontend: MeshCoreConnection.tsx config-page hardening, new ErrorBoundary component, wired into App.tsx. - tests: fix ~40 call sites broken by refresh() becoming async ( test_generic_http.py, test_store_received_delta.py, test_store_wzdx_persist.py) by wrapping with asyncio.run(), matching this suite's existing convention for calling async code from sync test functions. Verified: all 40 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Matt Johnson
|
3c281c96e2 |
fix(fire): honor tombstoned_at as a durable mute in the gate decider
fires.tombstoned_at (the same column the dashboard's active views already filter on: /api/env/active WHERE tombstoned_at IS NULL) is now checked in decide() before the row-missing/growth/cooldown branches, so a tombstoned incident can never re-broadcast through any path. Defensive on both the read (falls back to the pre-v12 column set if tombstoned_at doesn't exist) and the lookup (missing key reads as "not tombstoned" rather than raising) -- an un-migrated schema must not crash the decider. |
||
|
Matt Johnson
|
81ade1e323 |
fix(satpass): region-tag satpass events via observer_list
Satpass events carry no lat/lon/geometry, so CoverageFilter had no way to region-tag them for region_routes matching. gate_consolidated_pass() now attaches observer_list (comma-joined observer slugs, same shape already computed for the audit column) into the event data, which coverage_area.observer_region_names() reads. Defensive/fail-open: an empty observer_list yields no region tag rather than raising. |
||
|
Matt Johnson
|
738ffc71c3 |
fix(usgs_quake): stop dropping first-sighted quakes at the freshness gate
The USGS feed is a rolling PAST-DAY feed (2.5_day.geojson), so a genuinely first-seen quake's age-at-ingest routinely exceeds the generic per-toggle freshness_seconds (600s) window, silently dropping it downstream of decide()/DB-insert (last_broadcast_at stayed NULL on every sampled first-sighting row). - dispatcher.py: earthquake_event gets its own adapter_config-backed freshness override (adapter_config.usgs_quake.freshness_seconds, default 3600s), scoped to the CATEGORY (not the "seismic" toggle, which also carries hydro stream_flood_warning/stream_high_water and must keep the generic per-toggle freshness). - usgs_quake.py: to_event() no longer passes the adapter's fixed freshness in, deferring to the dispatcher override. - adapter_config/defaults.py: adds the freshness_seconds field/default for usgs_quake. |
||
|
Matt Johnson
|
50553b26f5 |
fix(roads511): distinguish full closures from partial restrictions
Adds explicit full-closure phrase matching (with partial-restriction phrases as vetoes) for sub_type mapping in to_event(), so wording like "All Shoulders Closed" or "Right Lane Closed" no longer gets broadcast as a full road_closed event. Existing is_closure severity/summary logic is untouched; the new is_full_closure_flag/_has_full_closure_language helpers are additive and scoped to sub_type only. |
||
|
Matt Johnson
|
c4706d3c92 |
fix(ipaws): route county-only alerts through evac-phase detection
Adds meshai.notifications.evac_phase.detect_phase, wired into the IPAWS formatter/adapter, to classify READY/SET/GO evacuation phases from real FEMA IPAWS alert headline/CMAMtext strings. Includes explicit false-positive guards since a wrong GO detection would broadcast an evacuation order that was never issued. |
||
|
67176d66d5 |
chore(central-ripout 2d-ii): remove dead fire scraps (NOT the 'rewrite' — there wasn't one) (#166)
* chore(fire-ripout 2dii-a): remove dead wildfire_growth cutover path + handle_firms
wildfire_growth events are always fully precomposed (env/firms.py sets
_meshai_precomposed=True + title=<wire from _render()>) and the category is
not in cutover.NATIVE_ALWAYS_DECIDE, so the registered fire formatter could
never be reached live via compose_mesh_message for this category -- the
precomposed-title bypass always won first. Removes the dead
is_cutover("wildfire_growth") NEW-PATH branch in
fire_fusion._handle_pass_boundary (kept the live legacy leg that calls
_render unconditionally) and the now-unreachable wildfire_growth formatter
registration in notifications/formatters/__init__.py.
Also removes handle_firms, the dead Central NATS-envelope entrypoint (zero
live production callers -- Central's consumer that drove it is gone), plus
its envelope-specific filtering helpers (_confidence_passes, _in_bbox,
_coerce_severity, _log_event, FIRMS_CONFIDENCE_FLOOR/FRP_FLOOR/BBOX_OPTIONAL)
that had no live callers left. The shared, LIVE fusion core
(_ingest_pixel_core, attribution, clustering, growth/spotting/halt) and
_parse_acq_epoch are unaffected -- still the engine behind the native
ingest_hotspot_pixel entrypoint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(fire-ripout 2dii-b): remove dead handle_wfigs entrypoint
handle_wfigs (the dead Central NATS-envelope entrypoint) had zero live
production callers -- Central's consumer that drove it is gone. The LIVE
WFIGS path is env/fires.py (native adapter) -> env/store.py::_emit_event,
forced onto gating.fire.decide + the shared fire formatter via
cutover.NATIVE_ALWAYS_DECIDE, independent of this dead handler.
Removes handle_wfigs and its private-only helpers (_coerce_severity,
_log_event, _log_event_returning_id) that had no callers left. _render
remains -- it is called directly by env.fire_fusion._handle_pass_boundary on
the FIRMS wildfire_growth path, and is used as a byte-identity oracle by
tests against the shared, live fire formatter. _build_canonical,
_attach_commit_handles, _fire_too_old_to_announce, _now, _cleanup_stale_fires
and WFIGS_BROADCAST_COOLDOWN_S are kept -- each has its own direct test
coverage independent of handle_wfigs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(fire-ripout 2dii-c): rewire fire tests off the deleted dead entrypoints
handle_firms and handle_wfigs (both removed in the prior two commits) were
used throughout this test suite purely as convenient DRIVERS for live logic
(pixel attribution/clustering/growth/spotting/halt fusion; the WFIGS
New/Update/cooldown/closed decider). Per-file disposition:
Rewired to drive the LIVE native entrypoint directly (no behavior change --
same underlying _ingest_pixel_core / gating.fire.decide engines):
- test_fire_tracker_phase1/2/3.py: handle_firms -> ingest_hotspot_pixel
- test_firms_refactor.py: handle_firms driver helpers -> ingest_hotspot_pixel;
dropped the dead wildfire_growth cutover test + the fully-redundant
TestNotCutoverLegacyVerbatim class (already covered by
test_firms_native_fusion.py's TestIngestGrowth/Spotting/Halt against the
live entrypoint); wildfire_growth formatter registration test now
asserts None (matches source change).
- test_wfigs_handler.py: handle_wfigs -> _render (the live WFIGS renderer)
called directly, for the anchor-priority + missing-acres cases that
exercise shared/live code (_location_anchor).
- test_fire_refactor.py: handle_wfigs -> gating.fire.decide() driven
directly, with state written the same unconditional shape the live
native path uses; TestGateSequenceParity trimmed to focus on the
tombstone/closed lifecycle step (not covered by
test_fire_native_growth.py's native-adapter New/Update/cooldown coverage).
- test_tombstone_broadcast.py, test_fire_age_gate.py: handle_wfigs ->
gating.fire.decide() driven directly; same assertions, off the dead path.
Deleted as pure dead-entrypoint contract testing with no live equivalent:
- test_firms_native_fusion.py::TestCentralPathUnchanged (2 tests) --
existed only to guard handle_firms's own envelope parsing.
- test_firms_handler.py: confidence/FRP/bbox filtering, missing-coords,
non-firms-adapter guard, event_log accounting (all handle_firms-specific;
the native adapter filters upstream in a different code path). Kept +
rewired: the shared _ingest_pixel_core dedup behavior and
_parse_acq_epoch's int/short acq_time parsing (both genuinely live/shared).
- test_wfigs_handler.py: envelope field-extraction (acres-fallback chain,
IA-placeholder-as-name), tombstone/perimeter subject -> event_log,
New/Update/cooldown decision + audit-row wiring (all redundant with
tests/test_fire_native_growth.py's coverage of gating.fire.decide()
through the real native adapter).
- test_tombstone_broadcast.py::test_commit_callback_flips_handled --
asserted handle_wfigs's own event_log-row flip on commit, a Central-only
concept the native path never used.
- test_tail_followups.py::test_wfigs_tombstone_stamps_column -- asserted
handle_wfigs's own inline tombstoned_at UPDATE; no live producer emits
_kind=wfigs_tombstone today, so nothing in the live system stamps it.
Both live renderers stay covered: fire_format (wildfire_declared/incident,
via test_fire_refactor.py + test_fire_native_growth.py, untouched) and
_render (wildfire_growth via FIRMS, via test_fire_tracker_phase2.py +
test_firms_native_fusion.py's TestIngestGrowth, both driving the live
ingest_hotspot_pixel entrypoint).
Collection: 2010 -> 1974 tests (net -36: dead-only tests deleted, live
behavior rewired 1:1 or consolidated onto already-existing native-path
coverage). Full suite: 1974 passed, 0 failed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
6050392d09 |
chore(central-ripout 2e): split central_normalizer.py; eliminate the "central" name (#165)
Final structural step. central_normalizer.py (973 LOC, misnamed — it lived
OUTSIDE central/ but its docstring described "the Central firehose") is DELETED.
No file or symbol named "central" survives (only immutable historical comments
in an already-applied migration .sql remain — those are not edited).
Split per the owner's rule ("next to what uses it"):
- Live geo/place-name utilities (nearest_town, _haversine_miles, _bearing_compass,
_compute_distance_bearing, _h3_cell, _photon_reverse_places, geocoder config)
→ the EXISTING meshai/geo.py ("Geographic utilities…"), same domain. NOT a
junk drawer. geo.py's nearest_city (hardcoded table) and central_normalizer's
nearest_town (live Photon+H3 reverse-geocode) are different implementations —
both kept, no false consolidation.
- WZDx work-zone parsers (_parse_wzdx_federal, _norm_wzdx_sub_type, _norm_direction,
_parse_mile_posts, _clean_description, _is_uninformative_road) → env/wzdx_parse.py,
next to their sole consumer env/wzdx.py.
- ~400 dead lines deleted: the Central-envelope machinery (normalize,
_parse_state_511_atis, _parse_wfigs_incidents, _parse_itd_511_work_zone,
should_skip_state_511_atis_id, is_incident_envelope_stale, normalize_road_name,
_norm_sub_type, _parse_ends_at) — only caller was the deleted Central consumer.
Consumers rewired: env/wzdx.py, env/fire_render.py, formatters/fire.py,
formatters/_anchor.py, main.py, persistence/curation.py, composer.py.
Tests: test_central_normalizer.py RENAMED → test_geo_wzdx_parse.py (live geo/wzdx
tests kept + rewired; dead-parser tests dropped). test_itd_511_work_zone.py
deleted (tested only the dead itd_511 parser — verified gone from source).
Other test files: import-path rewires. Live coverage for nearest_town /
_parse_wzdx_federal / bearing preserved across test_geo_wzdx_parse, test_adapter_wzdx,
test_wfigs_handler, test_incident_refactor.
Suite: 2010 passed, 0 failed. The 49-test drop is the dead Central-envelope
parser tests, correctly removed. Stale "Central firehose" docstrings on the
moved code updated to describe present behavior.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
de91751bbc |
chore(central-ripout 2d-i): relocate the fire engine + renderer out of central/ (#164)
Moves the last two live files out of the retired-Central folder. central/ is now EMPTY and deleted entirely (incl. __init__.py). - fire-fusion engine (ingest_hotspot_pixel + the growth/cluster/spotting engine) → env/fire_fusion.py, next to its sole consumer env/firms.py. - wildfire text renderer (_render + its live helpers) → env/fire_render.py. THE COUPLING RESOLVED: firms_handler._handle_pass_boundary had a LAZY import inside a function body — `from meshai.central.wfigs_handler import _render` — sitting on the live FIRMS fire-growth path. It is now a normal top-of-file import (`from meshai.env.fire_render import _render`), visible and greppable. PURE MOVE — no behavior change: - The parity oracle (test_fire_refactor.py) PASSES UNCHANGED (only its import paths updated) — proving the fire wire output is byte-for-byte identical before and after. Fire alerts say exactly what they said. - handle_firms / handle_wfigs (dead entrypoints, only caller was the deleted consumer.py) were KEPT and moved rather than dropped — the wording-cleanup PR removes them deliberately. "When unsure, keep." - fire_render.py's geo-helper imports still point at central_normalizer — that file's split is a SEPARATE PR; carried the imports along, did not touch it. Consumers + test import paths rewired. Full suite: 2059 passed, 0 failed. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
dd932b4ee9 |
chore(central-ripout 2c): relocate gauge sites to env/gauge_sites.py (#163)
Moves the last non-fire live file out of central/. idaho_gauge_sites.py held USGS stream-gauge site data + threshold ranking — pure hydro/gauge domain, misfiled among the retired Central handlers. Applied the owner's rule with a clean split: - THRESHOLD_RANK (a 5-element list used ONLY by notifications/gating/hydro.py) inlined there — single consumer, belongs next to its use. - The gauge-site lookup (used by env/usgs.py + persistence/curation.py) moved to env/gauge_sites.py, next to the usgs adapter that produces gauge data. - Dropped the redundant "idaho_" prefix (the Idaho scoping is data, not code). Consumers rewired: env/usgs.py, notifications/gating/hydro.py, and tests. central/ is now down to firms_handler.py + wfigs_handler.py (the fire pair, whose relocation is a separate PR because of their cross-file coupling). Move + import-rewire only, no behavior change. Suite: 2059 passed, 0 failed. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
41178831e4 |
chore(central-ripout 2b): relocate satellite code to env/satellite/ (#162)
* chore(central-ripout 2b): create env/satellite package, move pass_predictor
pass_predictor.py was 100% live (no dead entrypoint) — SGP4 pass
computation used by both the native satpass adapter and the on-demand
!satpass command. Straight move, no code changes: meshai.central.pass_predictor
-> meshai.env.satellite.pass_predictor. Owner directive: satellite code gets
its own folder under the feed adapters, separate from env.satpass (the
adapter) to avoid colliding with env/satpass.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(central-ripout 2b): split satpass_handler.py -> env/satellite/pass_format.py
satpass_handler.py was a split file: live wire-formatting/gate logic plus
dead Central-envelope ingest machinery whose only caller was the
already-deleted central/consumer.py NATS bridge.
Moved (live, verified via rg — external callers in env/satpass.py and
commands/satpass_cmd.py, or transitively called by them):
gate_consolidated_pass, format_pass, _check_rate_cap, _upsert_satpass,
_attach_commit, _map_severity, _canonical_id, _azimuth_to_compass,
_short_sat_name, _collapse_compass, _region_paren, _is_synthetic_observer,
_format_time_12h/24h, _format_ampm, _tz_abbr, _date_label,
plus the _SHORT_SAT_NAMES/_SHORT_NAME_SUBSTR/_SYNTHETIC_OBSERVERS tables.
Dropped (dead — zero callers outside the already-deleted consumer.py and
handle_satpass/consolidate_satpass_pending themselves; verified with rg):
handle_satpass, consolidate_satpass_pending, _cleanup_pending,
load_pending_schedule, _log_event_returning_id, _coerce_float,
_coerce_int, _parse_iso_epoch, _now, CONSOLIDATION_DELAY,
_pending_consolidation_ids, drain_pending_consolidation_ids,
_elevation_bucket (already-orphaned pre-ripout: superseded by numeric
"max NN°" wire format, zero callers anywhere but its own tests),
SCHEMA_SATPASS_EVENTS/SCHEMA_SATPASS_PENDING (unused string constants —
actual schema lives in persistence/migrations/*.sql, never imported).
Also dropped now-unused `json`/`time`/`Any` imports.
Straight code move otherwise — no logic changes to any moved function.
Two docstrings updated for accuracy (module docstring, and
gate_consolidated_pass's docstring which referenced the now-deleted
Central consumer path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(central-ripout 2b): split tle_handler.py -> env/satellite/tle_store.py
tle_handler.py was a split file: live storage helpers plus a dead
Central-envelope ingest entrypoint whose only caller was the
already-deleted central/consumer.py NATS bridge.
Moved (live — used by env.tle_fetch, env.satpass, commands.satpass_cmd,
verified via rg): upsert_tle, get_fresh_tles, get_tle_by_norad,
search_tle_by_name.
Dropped (dead — handle_tle's only callers were tests and the deleted
consumer.py; verified with rg): handle_tle.
Straight code move otherwise — no logic changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(central-ripout 2b): repoint satellite consumers at env.satellite
Rewire the three production consumers (including their lazy/function-body
imports, not just module-top ones) to the new location:
- env/satpass.py: central.tle_handler -> env.satellite.tle_store,
central.pass_predictor -> env.satellite.pass_predictor,
central.satpass_handler -> env.satellite.pass_format
- env/tle_fetch.py: central.tle_handler.upsert_tle -> env.satellite.tle_store
- commands/satpass_cmd.py: all three, same mapping
Also refreshed docstrings that pointed at the old module paths or described
the now-fully-deleted Central consolidation path
(consolidate_satpass_pending / satpass_pending buffer) as a live
alternative, and updated central/__init__.py's module docstring to stop
listing the three relocated modules among central's remaining contents.
No behavior changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(central-ripout 2b): update satpass/tle test suite for the relocation
Repoints every remaining test import at the new env.satellite.* modules
and removes/adapts coverage for the Central envelope-ingest path deleted
in this pass (handle_satpass, consolidate_satpass_pending, handle_tle, and
the filtering/coercion/staleness logic that lived only inside them):
- test_satpass_native.py, test_tle_fetch.py, test_satpass_command.py:
import-path updates only (pass_predictor, tle_store). Also dropped
test_satpass_command.py's TestTLEUpsert.test_returns_none_always
(handle_tle-specific contract, no longer applicable) and rewrote its two
latest-wins tests to call upsert_tle directly — same behavior under test,
now exercised through the still-live primitive instead of the dead
wrapper.
- test_satpass_native.py: deleted test_central_consolidate_feeds_shared_gate_merged
(spied on consolidate_satpass_pending, which no longer exists). The
merge-across-observers logic it guarded is native-side (_consolidate)
and already covered by test_two_observers_consolidate_to_one_broadcast.
- test_satpass_handler.py: gutted to the one test that calls format_pass
directly (test_format_pass_worst_case_fits_140); the rest exercised
handle_satpass's observer/norad/elevation filters, which have no live
equivalent (the native adapter filters at the config level, not
per-envelope) and is redundant with test_satpass_native.py's dedup/wire
coverage via the real SatpassAdapter path.
- test_satpass_broadcast_safety.py: kept every test that calls format_pass
or gate_consolidated_pass-adjacent REGISTRY checks directly (wire format,
clean-format rules, REGISTRY defaults); deleted TestNoradIdTypeCoercion
and TestStalenessGuard (handle_satpass-only logic, no live equivalent)
and the 6 _elevation_bucket tests (_elevation_bucket itself was dead
before this pass too — zero callers anywhere but its own tests, already
superseded by the numeric "max NN°" wire format per its own docstring).
- test_satpass_persisted_timer.py: dropped test_due_at_persisted_on_normal_ingest
(handle_satpass-only); kept the two schema/migration tests, which don't
touch satpass_handler.
- Deleted outright (tested ONLY the dead Central envelope-ingest path, no
live equivalent to port to): test_satpass_event_path.py,
test_satpass_compass_fallback.py, test_satpass_wire_fields.py.
Full suite: 2059 passed, 0 failed (was 0 failed on main pre-change).
Satpass/TLE subset (99 tests across 8 files) verified green in isolation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
ff127bee18 |
chore(central-ripout 2a): remove the budget shim + dead work_zone renderer (#161)
* chore: remove central.budget re-export shim The shim's implementation lived at notifications.formatters._budget from the start; central.budget was only a 9-line re-export kept around for import-path compatibility. Point every importer directly at the real module and delete the shim: - notifications/renderers/composer.py: lazy import inside a function - central/wfigs_handler.py, central/satpass_handler.py: import line only - tests/test_fire_refactor.py, test_nws_refactor.py, test_firms_refactor.py: import line only, no behavior change test_budget_shim.py existed solely to assert identity-equality between the shim and the real module; with the shim gone there is nothing left for it to test, so it is deleted too. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: delete dead renderers/work_zone.py format_work_zone_mesh() had exactly one production caller, the central/consumer.py NATS bridge deleted in the prior Central-excision pass. Its live replacement, formatters.incident._render_work_zone() (registered for category "work_zone" in formatters/__init__.py), is an already-shipped byte-identical replica per that module's own docstring. All remaining references to renderers.work_zone were prose/comments describing the replica relationship, not imports. Test fallout: - tests/test_work_zone_renderer.py tested only the dead renderer in isolation (17 cases). Deleted — the live path has its own coverage (test_adapter_wzdx.py's formatter-integration tests, plus TestCrossSourceIdentity::test_work_zone_category_uses_wz_renderer and TestWorkZoneGolden in test_incident_refactor.py). - tests/test_itd_511_work_zone.py::test_itd_511_work_zone_renderer_produces_wire only smoke-tested the dead renderer's wire output for itd_511 data; redundant with TestWorkZoneGolden's byte-identical fixture coverage for the same adapter. Deleted. - tests/test_incident_refactor.py::TestWorkZoneGolden compared the live formatters.incident.format() output against a golden computed by calling the dead renderer live on two real fixtures. Mirroring the precedent already in test_nws_refactor.py for this exact situation (golden generator deleted out from under a parity test), the two golden strings were captured by running format_work_zone_mesh() against these fixtures immediately before deletion and are now pinned as literals — same coverage, no live dependency on the dead module. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
b1535744a1 |
fix(tests): stop poisoning sys.modules session-wide with a bare MagicMock (#160)
Three test files (test_llm_scoping, test_fix_meshcore_save_and_llm_test, test_config_partial_save_merge) did `sys.modules.setdefault(_mod, MagicMock())` to stub an optional import. `setdefault` installs the MagicMock into sys.modules for the ENTIRE pytest session even when the real package is present — so any LATER test that does `await <that module>.<coro>(...)` (e.g. `await aiosqlite.connect(...)`) fails with "object MagicMock can't be used in 'await' expression" / "Event loop is closed". This is what made test_fire_tracker_phase4::test_natural_language_fire_ question_routes_to_llm pass in isolation but fail in full-suite order — the leak came from an earlier file, not the victim. #140 hardened the victim's own config/history isolation but couldn't fix an external sys.modules poison. Fix: only fall back to the MagicMock when the real module genuinely fails to import (guarded assignment), so a present package is never replaced. Root-cause fix in the polluters, not a skip on the victim. Suite: 2422 passed, 0 failed, 72 skipped — fully green (was 6 failed before #140, then 1 order-dependent failure after). Confirmed deterministic across repeated full runs. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
cd226689da |
docs: one README (root canonical) — the two had cross-drifted (#152)
* docs(readme): reconcile root README as the single source of truth /README.md (renders on GitHub) and work/README.md (packaged by work/pyproject.toml's readme = "README.md") had cross-drifted: each had two of four correct lines. Root had the correct `cd meshai/work` path and work/-prefixed curl URLs; work/README.md had the correct gemini-3.1-flash-lite model (google retired the gemini-2.x lite tier on this project's API key). Pull the one missing fix (model name) into root/README.md. Verified via diff that these were the only 4 lines (8 diff lines) that ever differed between the two files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build(work): eliminate work/README.md as a second source of truth The hand-maintained duplicate is what caused the cross-drift fixed in the previous commit. Replace work/README.md with a symlink to ../README.md so there is exactly one file to edit. This required unhooking work/'s packaging from a physical README.md: - setuptools' pyproject reader hard-rejects readme paths outside the project dir (`_assert_local` in setuptools/config/expand.py) — so `readme = "../README.md"` is not an option, tested and confirmed. - The symlink resolves fine for local packaging (pip install -e ., python -m build --sdist/--wheel all tested passing, PKG-INFO correctly carries the root content through the symlink). - It does NOT resolve for the Docker image build: work/Dockerfile's `COPY README.md .` and both work/docker-compose.yml (context: .) and .github/workflows/docker-publish.yml (context: work) pin the build context to work/, which does not contain the symlink's target. Confirmed with an isolated repro: Docker COPY on a symlink whose target is outside the build context fails with "too many links". Repointing the build context at the repo root would touch every COPY path in the Dockerfile plus CI — out of scope here and not worth it for a README. Chose the minimal fix instead: drop `readme = "README.md"` from work/pyproject.toml (meshai isn't published to PyPI — no publish workflow exists, only GHCR image publishing — so there's no long_description to lose in practice) and drop the now-unnecessary `COPY README.md .` from work/Dockerfile. Confirmed a dangling same-named symlink left in the build context, never COPYed, does not break context transfer. Tested end-to-end: a full `docker build -f work/Dockerfile work` against the real Dockerfile succeeded (frontend build, apt deps, pip install -e ., fastembed model fetch), and the resulting image imports meshai and reports correct `pip show` metadata with no README involved. Note for docs/onboarding-via-gui (PR #147) and chore/untrack-dashboard-static: both edited work/README.md, the file GitHub never rendered. That target is gone; their Quick-start content needs to be re-applied to root README.md when those branches rebase. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
18e9ae9127 |
chore: untrack dashboard build output (stale by 13 commits on the pip path) (#151)
* chore(dashboard): untrack committed frontend build output work/meshai/dashboard/static/ held committed vite build artifacts (index.html + hashed assets/*.js|css + copied public/ images) that were last built 2026-07-07 (PR #87), 13 frontend commits ago. Docker builds the frontend fresh and overwrites these on every image build, but the pip install -e . path (documented in README Quick start) ships this stale directory verbatim via [tool.setuptools.packages.find] include. The root .gitignore already had a rule for this ("meshai/dashboard/static/") but it silently stopped matching when |
|||
|
5685ed034f |
feat(swpc): native solar_radiation_storm (S-scale) formatter + decider + pinned goldens (#145)
Co-authored-by: Matt Johnson <mj@k7zvx.com> |
|||
|
e9daabfbff |
fix(satpass): default feed_source to native; correct misleading config comments (#139)
Central is retired -- its NATS broker (nats://central.echo6.mesh:4222) no longer exists -- but SatpassConfig.feed_source still defaulted to "central", overriding the _SourcedFeed mixin default of "native". A fresh install that enabled satpass got a silently dead adapter. The dashboard reinforced it: Environment.tsx seeded feed_source 'central' and labelled the adapter "via Central". Also corrects comments that documented the opposite of the code: the adapter_config keys usgs_quake.global_mag_floor / regional_mag_floor were labelled "CENTRAL-PATH ONLY", but notifications/gating/quake.py reads them unconditionally in the native path. Following those comments would have led someone to delete live config keys. - config.py: SatpassConfig.feed_source "central" -> "native" + docstring - adapter_config/defaults.py: correct the two "CENTRAL-PATH ONLY" comments - Environment.tsx: seed 'native'; reword the satpass subtitle - tests: assert the native default; pin central explicitly where a test exercises the central path or the feed_source flip Suite: 2337 passed, 6 pre-existing failures (stale SCHEMA_VERSION x3, expired TLE fixtures x2, one order-dependent), no new failures. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
e9153943bf |
chore(wzdx): remove two dead config fields (#150)
Both were fully plumbed and read by nothing.
api_key -- self-documented as dead at config.py ("Keyless: api_key is
retained but unused"), yet wired end-to-end: a GUI ManagedSecret field, a
SECRET_FIELDS entry, an EXPECTED_SECRETS entry, a secrets_store mapping and
label, and a line in .env.example. env/wzdx.py assigned self._api_key and
never read it again. So an operator could go get an API key, paste it into
the secure secrets manager, and have it do precisely nothing -- the ritual
looked complete end to end, which is what made it worth removing rather
than leaving.
endpoints -- default ["/get/event"], exposed as an editable list in the
dashboard, never read. Copy-paste from Roads511Config.endpoints (which IS
read, at env/roads511.py; Roads511 is untouched here). WZDx discovers feeds
via the FHWA registry_url/states instead.
WZDx's actual fetch behavior is unchanged; this removes dead config only.
Note for existing installs: anyone with WZDX_API_KEY set in
/data/secrets/.env will simply have an ignored env var. Harmless -- it was
already ignored.
Suite: 2337 passed, 6 failed (the pre-existing set: stale SCHEMA_VERSION x3,
expired TLE fixtures x2, one order-dependent), 72 skipped -- exact baseline
match, no new failures.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
3c900091d7 |
docs: point onboarding at the dashboard; drop 3 phantom keys from the seeded default (#147)
* fix(entrypoint): drop three phantom history keys from the seeded default
The config written on first boot -- what EVERY fresh Docker install starts
from -- seeded three keys that do not exist on HistoryConfig and are
silently discarded on load:
auto_cleanup: true
cleanup_interval_hours: 24
max_age_days: 30
HistoryConfig has only `database`, `max_messages_per_user`, and
`conversation_timeout`.
To be precise about the impact: history cleanup DOES work -- cleanup_expired()
is wired at main.py:237 and _prune_history() honours max_messages_per_user.
What never existed is the time-based retention model these keys describe (a
30-day age cutoff on a 24h interval). An operator setting max_age_days: 90
expecting 90-day retention was silently ignored.
Not implemented here -- whether time-based retention should exist is a
product decision, not a cleanup. This only stops the default config
promising it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: point onboarding at the dashboard, not the legacy example config
The documented setup path was `cp config.example.yaml config.yaml`, but that
file is ~40% incomplete: of the 20 top-level Config fields it covers 15,
omitting coverage (the universal bbox), danger_zones, generic_sources,
meshcore_context, commands, and the current notification model
(toggles/destinations/region_routes). Its own notifications header still says
the schema "will be replaced in v0.3 by the 8-toggle model" -- which shipped
long ago. Anyone following the project's own instructions landed on a
degraded surface with no signal a richer config existed.
It is also read by NOTHING at runtime: docker-entrypoint.sh sets
MESHAI_CONFIG=/data/config.yaml and writes its own inline default on first
boot. The Dockerfile still COPYs config.example.yaml into the image, so it is
kept and now labelled reference-only rather than a starting point.
- README: Docker quick-start seeds itself; configure via the dashboard. The
pip path still uses config.example.yaml (nothing seeds one there) but now
carries an honest note that it is a minimal bootstrap, not a reference.
Adds an Advanced section for the split /data/config/ layout and the
migrate_config_v03 path into it.
- docker-compose.yml: the comment claimed config lives at /data/config.yaml
as though that were the only layout; corrected to describe both, and note
secrets live in /data/secrets/.env.
- Dockerfile: document why config.example.yaml is still shipped.
Not done deliberately: config.example.yaml is NOT expanded to cover all 20
sections. A second hand-maintained schema is what caused this drift; the
dashboard is the authoritative surface. The legacy single-file loader is
untouched and still fully supported.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(readme): move dashboard-first onboarding rewrite to root README
work/README.md is about to become a symlink to root README.md on
docs/single-readme. Retarget the Docker/dashboard-first quick start
and config.example.yaml honesty note added in
|
|||
|
12e7335add |
fix(dashboard): hard-disable the Central feed_source option (retired backend) (#154)
Central's NATS broker is gone and its database was dropped 2026-07-15, but the per-adapter Native/Central toggle on the Environment page still let an operator pick "central" for 11 of 12 adapters and have it save successfully (config.py's validator still accepts central|native; env/store.py's gate just silently skips registration when feed_source != "native"). The result: flip the switch, save cleanly, feed dies with no error anywhere. This is a minimal stopgap, not the real fix. A separate refactor will strip feed_source/hasCentral/nativeOnly and the whole toggle out once Central's removal is complete; that's out of scope here and backend behavior (config validator, store gate) is intentionally untouched so existing YAML with feed_source: central doesn't break. For now: hard-disable the Central button in FeedSourceToggle regardless of hasCentral/nativeOnly (kept referenced, not deleted, for the pending refactor), and make the accompanying copy tell the truth — "Central feed source has been retired — native only" — instead of the old adapter-scoped "not available for this adapter" text that no longer describes what's happening. Verified: GaugeSites.tsx only reads usgs.feed_source to gate its lookup button, never writes it — not a second instance of this trap. tsc --noEmit and the pytest suite show zero new failures vs. pre-edit baseline. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
2543dfa8d1 |
fix(config): warn on unknown config keys; delete 7 phantom keys from the example (#146)
* fix(config): warn on unknown config keys instead of silently dropping them
_dict_to_dataclass() silently continue'd past any key not in the target
dataclass's field set -- an operator could set a config key, restart, and
have it vanish with zero feedback (config.example.yaml's phantom
mesh_intelligence keys are exactly this bug, fixed separately).
Now logs a WARNING naming the key and the dataclass, hinting at a typo or
a renamed/removed field, via the module's existing _config_logger.
Traced every dynamic/free-form config path to rule out false positives:
notifications.toggles, notifications.destinations, generic_sources,
mesh_sources, and notifications.rules all route through explicit
dict-of-dataclass or verbatim-passthrough handling and never spuriously
warn. Two legitimate legacy shapes DO hit the strict field-check path with
keys that were never (and will never be) dataclass fields:
- notifications.channels (pre-v0.5 channel list), consumed directly from
the raw dict by _migrate_legacy_channels
- notifications.region_routes.enabled (pre mt/mc-split master switch),
read directly by the explicit region_routes handler
Both are allowlisted in _KNOWN_LEGACY_DROP_KEYS so users mid-migration
don't get spurious noise on every load.
Added tests/test_config_loader.py coverage: unknown key warns and does
not raise, both legacy shapes stay silent, and the free-form/dynamic
sections never warn for keys valid on their real target shape.
* fix(config): remove phantom config.example.yaml keys, add missing live ones
The new unknown-key warning (previous commit) caught config.example.yaml
loading with SEVEN warnings, all real drift -- none were false positives
of the warning itself:
mesh_intelligence block shipped three keys with no MeshIntelligenceConfig
field and no implementation anywhere (git log -S confirms they were never
built, not leftovers from a removal):
- region_radius_miles, infra_overrides, region_labels
These only made sense under an older auto-clustering design; what
actually exists is explicit region anchors (regions: list[RegionAnchor]).
Deleted from both the live block and the commented-out example above it,
and added the four fields that DO exist and are live but were missing
from the example: regions, critical_nodes, alert_channel, alert_rules.
Also fixed the now-misleading comment at
dashboard/api/mesh_routes.py:281, which referenced region_labels --
comment only, no code change.
notifications block shipped a whole quiet-hours subsystem that was
deliberately ripped out (commit
|
|||
|
b0d1a76e70 |
feat(dashboard): make the gauge-sites bulk import reachable (#149)
* fix(dashboard): register gauge_sites_import router The gauge-sites bulk-import endpoint (CSV / NWS-AHPS) was fully built and tested but never wired into the app -- server.py never called include_router() for it, so POST /api/gauge-sites/import 404'd in production. Only test_tail_followups.py exercised it, via a raw TestClient bypassing the real app. * test(dashboard): assert gauge_sites import route is reachable via real app Guards against the router silently going unregistered again: builds the actual dashboard app via create_app() and drives a real CSV import through it end-to-end (POST + GET round-trip), instead of only hitting the router in isolation. * feat(dashboard): add gauge-sites bulk import UI Wires a UI onto the now-registered POST /api/gauge-sites/import endpoint, inside the existing GaugeSites tab (no new nav entries/pages). Adds an Import toggle next to Add site, with CSV (paste or file-load into a textarea) and NWS-AHPS (WFO code list) modes. Required/optional CSV columns are documented inline so an operator isn't guessing. Result counts (inserted/updated/skipped/detail_fetched) and any partial-failure errors from the AHPS scrape are always shown; a pending spinner covers the AHPS path's live water.weather.gov calls so a slow response never reads as "imported 0 sites". The list refreshes only after a successful import that actually changed rows. --------- Co-authored-by: Matt Johnson <mj@k7zvx.com> |
|||
|
89a46d520a |
fix(hydro): restore USGS stream-gauge flood alerts (silently dead since the all-native flip) (#156)
* fix(hydro): make native USGS gauge flood alerts renderable env/usgs.py emits stream_flood_warning / stream_high_water Events for elevated stream gauges, but neither category had a registered gating decider or formatter, and event.data was left empty -- so every detected flood/high-water reading was silently dropped before it ever reached the mesh (get_decider/get_formatter both returned None, and compose_mesh_message fell through with nothing to render). - Register the existing hydro.decide()/hydro.format() (already used by the Central-only `stream_flow` category) under stream_flood_warning and stream_high_water too -- same shared-decider/formatter pattern already used for avalanche_warning/_watch, weather_*, wildfire_*, and emergency_*. - Add both categories to cutover.NATIVE_ALWAYS_DECIDE so the decider and formatter actually run unconditionally (mirrors the native WFIGS fire categories): Central never emits these two category strings, so there is no shadow-bake window to wait out. - env/usgs.py's to_event() now populates event.data with the canonical hydro schema (site_id, gauge_name, stage_ft, flow_cfs, unit, threshold_state, reading_time, lat, lon, parameter_code) the shared gate/formatter expect, mapping the adapter's flood_status strings onto the ranked threshold_state vocabulary. - gating/hydro.py's decide() now also OWNS an unconditional gauge_readings INSERT for the native source (source != "nwis"): the table had no writer since Central's nwis_handler stopped running (2026-07-05), so every native prior-state lookup returned "normal" forever and every elevated reading would have rebroadcast on every 15-minute tick instead of once per crossing. The Central source keeps its own inline handler-owned INSERT unchanged (decide() stays read-only for source="nwis"). Left as-is (not this fix): the toggle mapping for these categories (get_toggle() -> "seismic") is unchanged. It already matches the sibling `stream_flow` category and is enforced by test_water_v057.py::test_existing_hydro_entries_unchanged / test_water_categories_have_required_fields; there is no separate water/flood toggle family in VALID_TOGGLES, and inventing one is a config-surface change outside this fix's scope. Known limitation documented in gating/hydro.py: since to_event() only ever emits elevated readings (never a "back to normal" reading), a full recede-to-normal followed by a later re-crossing into the same tier will not re-broadcast until a higher tier is reached -- degrades toward silence, not spam. * test(hydro): cover native gauge flood-alert registration, gating, and wire format Registration tests prove stream_flood_warning/stream_high_water now resolve a decider AND a formatter (the thing that was broken), that neither resolves to the earthquake decider despite sharing the "seismic" toggle name, and that both are in NATIVE_ALWAYS_DECIDE. Gate tests drive env/usgs.py's to_event() through the real gating.hydro.decide(): a first elevated reading broadcasts (graceful no-prior-data handling), a sustained same-band reading suppresses, an escalation broadcasts again, and a routine reading never reaches the gate at all (unchanged pre-fix adapter behavior). A dedicated test confirms the new native-persistence write in decide() does not leak into the Central source="nwis" path. Golden formatter tests pin the wire string for a high-water and two flood-warning tiers, plus the missing-coords drop case -- all rendered through the same formatters.hydro.format() the Central `stream_flow` path uses (test_hydro_refactor.py already proves that formatter is byte-identical to the old central.nwis_handler._render()). Native events never carry flow_cfs (to_event() only ever emits stage/height readings), so that segment's absence is captured explicitly as current behavior, not ported from Central. An end-to-end test drives to_event() -> decider -> compose_mesh_message to prove the NATIVE_ALWAYS_DECIDE gate takes effect for the actual mesh render path, not just formatter/decider resolution in isolation. --------- Co-authored-by: Matt Johnson <mj@k7zvx.com> |
|||
|
d7913fddaa |
feat(dashboard): serve real recommendations in Nodes & Health; give mesh_intelligence one home (#148)
* refactor(mesh_reporter): expose recommendations as list[str]
Add recommendations_list(scope, scope_value) as the canonical source of
recommendation text. build_recommendations() now just joins that list with
the historical "OPTIMIZATION RECOMMENDATIONS:" header/bullet format it
always used.
router.py:1145 injects build_recommendations()'s output into the LLM system
prompt on the live mesh-DM path — verified byte-identical before/after
across mesh/region/node/missing/empty scopes via a synthetic fixture, and
pinned with a literal-string regression test so a future refactor can't
silently change that prompt text.
This unblocks wiring recommendations into the dashboard, which previously
only reached mesh DMs.
* feat(dashboard): serve real recommendations from /api/health
mesh_reporter was never on app.state — add it alongside health_engine etc.
in server.py's existing pattern. mesh_routes.py's health endpoint now
returns mesh_reporter.recommendations_list("mesh") instead of a hardcoded
[] TODO stub.
Add recommendations_available: bool alongside recommendations: string[] so
"the engine ran and found nothing" (healthy mesh) is never indistinguishable
from "the engine couldn't run" (unwired reporter, or an exception — logged
via logger.exception and swallowed so the rest of the health response still
serves). A crashed recommendations engine must not read as "mesh is
healthy" to an operator.
Also delete main.py's phantom `getattr(mh, "recommendations", [])` on the
websocket health_update push: nothing ever set .recommendations on the
mesh_health object (always []), and no frontend consumer reads it — the
REST endpoint above is the supported path.
* feat(dashboard-frontend): render recommendations in Nodes & Health
Meshtastic → Nodes & Health → Health tab used to be a duplicate
mesh_intelligence config editor (the same MeshIntelligenceSection also
edited from Settings → Intelligence, a stale-tab-overwrite hazard). Replace
it with a real operational view: fetch /api/health and render its
recommendations list.
Three states, matching visual conventions from Dashboard.tsx's alerts list
and MeshCoreRouting.tsx's cross-link note:
- recommendations present → Lightbulb list, one card per item
- empty + recommendations_available → "No recommendations — mesh is
healthy" (CheckCircle)
- recommendations_available === false → amber warning, not the healthy
state (AlertTriangle) — a crashed backend must not look healthy
mesh_intelligence config now has exactly one home: Settings. The Health tab
points there via the existing /config?section= deep-link pattern instead of
duplicating the editor. Config.tsx itself is untouched.
MeshHealth.recommendations/.recommendations_available are optional in the
type: the websocket health_update push doesn't include them (REST-only).
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
|
|||
|
ec973b606a |
chore(dashboard): remove orphaned route, redirect 5 legacy routes, fix a misleading docstring (#143)
* chore(dashboard): remove orphaned /adapter-config route
AdapterConfig is already embedded directly in Environment.tsx
(excludeKeys={CURATED_KEYS} hideLlmToggle); no in-app <Link> ever
targeted the standalone /adapter-config route. Drop the route, its
App.tsx import, the extraTitleItems entry in Layout.tsx, and the
now-unused Sliders icon import. pages/AdapterConfig.tsx itself is
kept — it's imported directly by Environment.tsx.
* chore(dashboard): redirect 5 orphaned legacy routes to their tabbed homes
/gauge-sites, /town-anchors, /mesh, /meshtastic/sources, and
/meshcore/companion each duplicate content now tabbed inside Places,
MeshtasticNodes, and MeshCoreContactsCompanion, and have no in-app
<Link> pointing at them (verified with rg). Convert them to
<Navigate replace> redirects, matching the existing /notifications
and /data-sources precedent.
/town-anchors, /meshtastic/sources, and /meshcore/companion land on
their host's first tab rather than deep-linking to the second tab
(no ?tab= support exists) — same accepted trade-off as the existing
/notifications and /data-sources redirects.
Drop the now-dead extraTitleItems entries and the redundant
'/meshcore/companion' pathTitles entry, plus the Radio, Droplets,
Layers, and Bot icon imports that were only used by those entries
(MapPin is kept — still used by the /places nav item). These 5
routes were never nav entries (extraTitleItems only), so the sidebar
stays at 17 items with 5 Meshtastic / 5 MeshCore — MT/MC symmetry
unchanged.
* docs(dashboard-api): fix misleading refresh_toggles docstring
The docstring claimed this endpoint was "kept for backwards-compat
with the dashboard's manual ping path," but rg turns up no frontend
call site for /notifications/refresh-toggles anywhere in
dashboard-frontend. The auto-refresh middleware
(_auto_refresh_toggle_filter, registered via
register_config_routes_hooks) already refreshes the ToggleFilter on
every successful notifications config PUT, covering the normal case.
Endpoint is untouched -- comment only, kept for ops/debug use.
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
|
|||
|
391a6316ad |
chore: remove dead modules and orphaned fixtures (-2,952 LOC) (#142)
* chore: remove dead mesh_sources.py module
Superseded by mesh_data_store.py, whose docstring states it "replaces
mesh_sources.py with a clean three-layer architecture." No remaining
inbound imports (absolute or relative) repo-wide; the only surviving
references to "mesh_sources" are to the unrelated config YAML section
of the same name, which is untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove dead region_tagger.py module
Only reference to "region_tagger" repo-wide was inside its own
module docstring usage example; no live imports found.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove empty meshai/cli package
Package contained only a docstring, no code. No imports of
meshai.cli found anywhere; console entry point is meshai.main:main
(pyproject.toml), and packaging (tool.setuptools.packages.find,
include = ["meshai*"]) has no explicit reference to the cli
subpackage. No MANIFEST.in/setup.py/setup.cfg exist to update.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove orphaned swpc/swpc_last golden fixtures
63 files (40 in fixtures/swpc/, 23 in fixtures/swpc_last/) never
loaded by any test. harness/goldens.py's load_fixtures(hazard) is
the only generic fixture loader and is never called with "swpc" or
"swpc_last" (only "avalanche", "nws", "nws_last", "quake"). The sole
remaining textual reference, a provenance comment in
test_swpc_refactor.py:162 ("Fixture mirrors swpc_last/0003.json..."),
is left untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
f4b717f351 |
chore: untrack dashboard-frontend/node_modules (#141)
The repo's blanket `dist/` .gitignore rule (line 19, inherited from the
Python template) matches directories named `dist` at any depth with no
leading slash, so it silently excluded node_modules/<pkg>/dist/ for every
package whose build output lives there. 21 packages (including
react-router) declare main/module pointing into dist/, so the committed
node_modules was non-functional -- e.g. require.resolve('react-router')
threw MODULE_NOT_FOUND against the tracked copy.
work/Dockerfile already runs `npm ci && npm run build` in its build stage,
so production was never affected by this -- the committed copy was pure
dead weight (8,619 of 9,232 tracked files, 93% of the repo).
package-lock.json is tracked, making `npm ci` fully reproducible, so this
untracks node_modules/ (added a `node_modules/` ignore rule to .gitignore)
rather than trying to fix or narrow the dist/ rule.
History is NOT rewritten -- the old blobs remain reachable via prior
commits, so clone size is unchanged. This only stops new commits from
re-adding node_modules going forward.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
9f06930a0d |
chore: excise the dead Central NATS consumer path (-11,328 LOC) (#144)
* chore: excise the dead Central NATS consumer path
Central was retired and its database dropped 2026-07-15; its NATS broker no
longer exists. Verified against the live CT108 deployment: all 12 adapters
run feed_source=native, zero on central, and central.enabled is False
(default, never overridden). The consumer and its handlers were unreachable.
Removed:
- central/consumer.py and 6 dead handlers (nws, quake, swpc, nwis, avy,
incident) -- their handle_* entrypoints were reachable only from the
consumer's dispatch
- the Central wiring in main.py (init, guarded start, retry loop, stop path)
- the dead config surface: CentralConsumerConfig, EnvironmentalConfig.central,
adapter_config ("central","severity_thresholds") and its display block
- the nats-py dependency (consumer.py was its only importer)
- 19 test files that exercised only the dead path
KEPT -- these live under central/ but are imported directly by native
adapters, and deleting them would break production:
- wfigs_handler.py: firms_handler._handle_pass_boundary() calls its _render()
on the live FIRMS growth-fire path (env/firms.py -> ingest_hotspot_pixel)
- firms_handler, satpass_handler, tle_handler: split files whose handle_*
entrypoints are dead but whose engines are live. Left intact; splitting
them is separate work.
- pass_predictor, budget, idaho_gauge_sites: fully live.
The usgs_quake keys global_mag_floor / regional_mag_floor / regional_centroid
/ regional_radius_mi / broadcast_pager_alerts are NOT removed despite comments
labelling them "CENTRAL-PATH ONLY" -- notifications/gating/quake.py reads them
unconditionally in the native path. Those comments are corrected separately.
Test-count note: the suite drops ~425 tests. Most were migration PARITY tests
whose sole purpose was proving the native rewrite byte-matched the Central
handler (golden byte-parity, cross-source identity, gate-sequence replay).
With the handler deleted there is nothing left to compare against, so they
cannot exist. Native-only tests were kept and reworked where a test reached
for a central symbol incidentally. This is a real coverage loss, accepted
deliberately: the parity harness proved the refactor faithful, and git
history preserves the originals.
Suite: 1984 passed, 6 failed -- the same 6 pre-existing failures as main
(stale SCHEMA_VERSION x3, expired TLE fixtures x2, one order-dependent),
all being fixed on fix/green-test-suite. No new failures.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(nws): restore native-only golden coverage for the wire formatter
Commit
|
|||
|
16f01b29d4 |
fix(persistence): derive SCHEMA_VERSION from migrations; unbreak the red suite (#140)
* fix(persistence): derive SCHEMA_VERSION from the migrations directory db.py hardcoded SCHEMA_VERSION = 26 while migrations/ had already reached v29 (v27 dispatcher floor-drop counter, v28 mesh_observations, v29 IPAWS). The migration runner globs the directory and applies every vN.sql it finds regardless of the constant, so a fresh DB actually landed at 29 while the constant claimed 26 -- a three-version drift that three tests were correctly catching. Derive it from the highest vN.sql present instead of bumping the literal, so it cannot drift again the next time someone adds a migration. Falls back to 0 if the directory is missing so import never fails; the migrations dir sits alongside db.py and ships with the package (Dockerfile COPYs meshai/). Adds a regression guard asserting the constant matches the highest migration file, and updates three tests that hardcoded 26 as a literal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(tle): make TLE fixtures time-relative so they cannot expire The ISS fixtures hardcoded epochs of 2026-06-30/07-01/07-02. tle_handler sets STALE_DAYS = 14 and get_tle_by_norad() filters on epoch >= now - 14d, so the fixtures silently aged out on 2026-07-02 and the tests began failing -- a time bomb, not a regression. Compute epochs relative to wall-clock now (base = now - 2d, +/-1d for newer/older) with correct TLE epoch-field encoding and mod-10 checksum. STALE_DAYS is untouched -- widening it in product code would have changed production behavior to paper over a test bug. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(fire-tracker): pin config + history db to tmp_path load_config() defaults HistoryConfig.database to the relative path "conversations.db", resolved against the process CWD, so every test calling load_config() with no override shares one file for the whole session. The conftest DB-isolation fixture only covers MESHAI_DB_PATH, not this. Point both the config dir and the history database at the test's tmp_path. NOTE: this does NOT resolve the order-dependent failure -- the test still passes standalone and fails in a full run, so the polluting state lives somewhere other than config/history. Left failing rather than weakened; root cause still unidentified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
8460ab50e0 |
fix(ipaws): route county-only CAP alerts + auto-refresh toggles live (#159)
County-only civil CAP alerts (SAME geocode, no <polygon>) had no geometry, so the geometry-based region tagger could not place them: no region -> no region_routes match -> silently not broadcast. Many CEMs / 911 outages / some AMBER alerts are county-only. - Bundle work/meshai/county_centroids.py: Census 2023 national county gazetteer internal points (3,222 counties + DC/territories), FIPS->(lat,lon), with SAME PSSCCC -> 5-digit FIPS helpers. - env/ipaws.py: when a CAP alert has SAME geocode(s) but NO polygon, set a Point (single county) or MultiPoint (multi-county) geometry from the county centroid(s) so the EXISTING coverage/region tagger locates it and tags ALL matching regions. Real polygon geometry always wins (never overridden). - dashboard/server.py: call register_config_routes_hooks(app) in create_app so the toggle auto-refresh middleware is actually wired in prod (was test-only); saving a family toggle now takes effect live without POST /api/notifications/refresh-toggles. Tests: county-only alert tags SW Idaho + matches emergency route cell; multi-county tags all regions; polygon path unchanged; create_app wires the refresh middleware; panhandle coverage-gap documented. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
e244405230 |
feat(dashboard): bring ipaws adapter + emergency family to full GUI parity (#158)
The IPAWS civil-alert adapter shipped backend-complete but frontend-partial: it rendered only in the generic adapter-config page and the Advanced (raw) Data Feeds tab, and the `emergency` family was absent from the curated Data Feeds panel, the family-settings toggles, and the MeshCore routing matrix. Adapter (ipaws), benchmarked against firms/nws: - Environment.tsx: add `ipaws` to AdapterKey union, EnvConfig interface, META (native-only, keyless), a new `emergency` FAMILIES group, PANEL_META_KEY (LLM toggle), and a hand-written renderSettings panel exposing base_url, user_agent, tick_seconds, state_fips, same_codes, exclude_weather, status_actual_only — with coverage-scope handling like the other adapters. - Environment.tsx: IPAWS_DEFAULT backfill so pre-ipaws GET payloads don't crash. - Dashboard.tsx: SOURCE_ICONS entry (Siren/IPAWS) so ipaws events aren't a slug. - ActivityLog.tsx: TABLE_LABELS + CATEGORIES + text-hint so ipaws_alerts rows show labeled "Emergency" and honor the category filter. - dispatcher.py: _SOURCE_TO_TABLE fallback ipaws -> ipaws_alerts so region-routed emergency sends land labeled in the audit feed (not NULL). Family (emergency), benchmarked against fire: - Notifications.tsx: add `emergency` to TOGGLE_FAMILY_META (Siren icon). This cascades to Family Settings, the Meshtastic delivery matrix, and the MeshCore routing matrix (the last was hardcoded to the static list and previously omitted emergency entirely). Backend VALID_TOGGLES/gating/categories were already complete — no backend family change needed. Tests: update the _SOURCE_TO_TABLE exact-match guard and add an ipaws audit-row parity test. Full suite 2407 passed / 6 pre-existing unrelated failures. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
c04daa6e4d |
fix(config): merge partial PUT bodies instead of resetting to defaults (#155)
Saving the "Auto-advert interval" dropdown on the MeshCore Companion page
took BOTH radios offline on 2026-07-17 06:46:52. One click, full outage.
The page PUT a single-key body to /api/config/connection:
{"meshcore_advert_interval_seconds": 10800}
_dict_to_dataclass() builds kwargs only from the keys present in the body
and lets `cls(**kwargs)` default the rest, so every OMITTED field was reset
to its dataclass default and written to disk:
type: tcp -> serial (Meshtastic offline)
tcp_host: 192.168.1.100 -> <lost> (LOCAL_FIELDS, see below)
tcp_port: 4404 -> 4403 (wrong meshmonitor vnode)
meshcore_host: 192.168.1.253 -> '' (MeshCore off; blank = off)
meshcore_conn_type: serial -> tcp (wrong transport)
meshcore_serial_port: /dev/meshcore-rak -> '' (RAK radio lost)
It was silent twice over. `connection` is restart-required, so the running
process kept the good in-memory config while the file sat gutted, waiting
for any restart to detonate. And save_section() writes the domain file
FIRST and local.yaml SECOND: meshtastic.yaml hit the disk already gutted,
then the local.yaml write (which owns connection.tcp_host via LOCAL_FIELDS)
died on `[Errno 13] Permission denied` -- so tcp_host landed in neither
file, and the 500 that would have named the cause was swallowed by the UI.
The operator saw nothing happen.
This was never one page's bug: PUT /api/config/{section} was destructive on
a partial payload for EVERY section. Other callers only survive because they
happen to spread the full object first.
Fixes, in depth:
* Route (the durable fix): merge the body over the CURRENT live section
before coercing, so omitted keys keep their live values while present
keys -- including '' / False / [] -- still apply. The base is the live
config, the same values GET serves, so a partial PUT now lands exactly
where a full-object PUT from that same GET would. Full-object callers are
unaffected. Fixed at the HTTP boundary, not in _dict_to_dataclass():
absent-key-means-default is CORRECT at config-load time, where a file
legitimately omits fields it does not override.
* Nested semantics keyed off the dataclass schema, not "is it a dict":
nested dataclass fields DEEP-MERGE (a partial region_routes must not drop
sibling cells), while bare dict/list fields REPLACE at the key (cells,
toggles, destinations, rules are dynamic maps -- deep-merging them would
resurrect deleted keys and make deletion impossible, the mirror image of
the bug being fixed).
* Page: send the full connection object like every other caller does.
* Errors are visible: the save handler no longer swallows the exception,
and updateConfig() surfaces the server's `detail` rather than a bare
"API error: 500", which is what hid Permission denied from the operator.
* Default advert interval 10800 -> 86400 (24h). 3h is far too frequent a
default for a public mesh; the UI "(default)" label moves to match.
Tests: tests/test_config_partial_save_merge.py reproduces the outage with
the exact payload, and pins merge semantics across connection AND
notifications, intentional clearing, deep-merge, and map-deletion.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
ea4c010967 |
feat(meshcore): report the true connection + add roster/channel management (#153)
self_info() reported host/port straight from config regardless of
conn_type, so a serial companion still advertised whatever stale
meshcore_host sat in the config — the API named a device meshai was not
talking to, which is enough to send an investigation to the wrong radio.
Connection details now come from one _connection_descriptor() shared with
connect(), so the log line and the API can't drift; only the live
conn_type's fields are populated and the rest are null.
meshai's device view is otherwise built once at connect and never re-read
— contacts via ensure_contacts(), channels via _enumerate_channels(). The
lib's contact handler only ever merges (meshcore.py::_update_contacts), so
a cached roster can never shrink, and a channel provisioned on the radio
stays invisible until the process restarts. There was no refetch path at
all. Adds an explicit resync that re-reads BOTH halves: a FULL
get_contacts(lastmod=0) reconciled with replace semantics (absent contacts
are dropped) plus a channel re-enumeration, each reporting what changed.
Also adds a preventive route-health check: every region_routes cell whose
MeshCore target cannot be resolved against the live roster/channel table
is surfaced, since such a send fails silently. Room targets are matched by
pubkey prefix, exactly as the dispatcher resolves them, so a picker-stored
prefix is not misreported as dangling. Same-name/different-pubkey roster
entries are flagged too — a name alone cannot identify a contact, which is
the trap behind a room rebuilt under a new keypair.
Backend:
- meshcore_roster.py: pure reconcile_contacts / check_route_health /
find_name_collisions (no device I/O — unit-testable without a radio)
- transport: _connection_descriptor, resync, refresh_contacts,
remove_contact, import_contact, export_roster, contacts_synced_at;
auto_update_contacts enabled (configurable — it costs one incremental
fetch per advert heard, which is real chatter on a dense mesh)
- API: POST contacts/refresh, DELETE contacts/{pubkey}, GET
contacts/export, POST contacts/import, GET route-health
Frontend (existing Contacts & Companion page — no new page or nav entry):
- dangling-route + name-collision banners; resync/export/add-contact
toolbar with last-synced and the added/removed counts; staleness badges;
search, filters and sortable columns; per-contact delete behind a
confirm; Companion tab shows the real transport + target.
A full pubkey is required to delete or add: the lib resolves by prefix,
and a prefix could silently hit the wrong node.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
bbe97398bc |
feat(ipaws): add FEMA IPAWS-OPEN civil-alert adapter (disabled by default) (#138)
Adds a new native `ipaws` adapter for FEMA IPAWS-OPEN EAS civil alerts. - Two-stage CAP fetch via base_url (direct FEMA or Conduit proxy): Atom index -> per-entry CAP 1.2 documents. - Non-weather civil alerts only (evacuation, Civil Emergency Message, AMBER, 911 outage, law-enforcement, HazMat); NWS/NOAA CAP dropped so weather is never double-broadcast. - Idaho + neighbour statefips scope gate applied before stage-2 fetch. - Own `ipaws_alerts` dedup table (migration v29); reuses the NWS CAP severity + formatter pattern. - Ships enabled=False (no transmit until explicitly enabled). Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
1199b3576a |
fix: make TomTom + FIRMS API keys optional (keyless-capable) (#137)
traffic and firms no longer idle when their key is blank — they build a keyless request (traffic: omit key= param; firms: omit the map_key path segment), matching roads511's existing optional-key pattern. Enables routing these feeds through a key-injecting proxy (Conduit) with the key held only there. Key-set behavior is byte-identical. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
998838bcb2 |
fix: make all native feed URLs config-driven (no hardcoded upstreams) (#136)
Adds a base-URL config field (default = the current value, backward-compatible) to the 9 adapters that hardcoded their upstream URL — nws, swpc (4 endpoints), ducting, fires (perimeter+points), firms, avalanche, usgs streams (3 bases), traffic, satpass/tle_fetch — mirroring the already-compliant roads511 pattern. Every feed URL is now overridable via config, enforcing the "everything configurable" rule and making each adapter live-repointable via a config PUT. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
8b4826f8db |
feat: hot-reload the environmental config section (no restart) (#135)
EnvironmentalStore.apply_config() rebuilds only the changed native adapters in place on a config PUT -- dedup/seen state (store-level) is preserved, unchanged adapters are untouched. Drops "environmental" from RESTART_REQUIRED_SECTIONS; falls back to restart-required only for the narrow feed_source->central case. Cascades nifc/fires -> firms. The old restart requirement was a Central-era coupling, now moot (all-native, central.enabled=false, CentralConsumer inert). Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
4fd431f907 |
fix(reminders): pace the reminder roll-call so N fires don't burst (#130)
The ReminderScheduler is the third fire-broadcast exit and the only one
that does not pass through the EventBus, so FirePacer (which paces the
Central and native fire-event exits to <=1/60s) never sees it. Its tick
is a roll-call: every eligible row is its own broadcast, dispatched in a
plain `for` loop with no gap. Unpaced, N eligible fires produce N
back-to-back mesh transmissions; the only downstream protection is
RadioSendQueue's ~2.2-2.6s per-transport inter-packet jitter, which
prevents packet collision but still lets a roll-call monopolise the mesh.
Not currently firing in production (every overdue fire is filtered by
terminate_when, so the eligible set is 0) -- this is fire-season
hardening against a latent burst, not a live incident.
Adds `spacing_seconds` (adapter_config, default 60 to match FirePacer)
enforcing a minimum gap between consecutive SUCCESSFUL reminder
deliveries. Deliberately a pure spacing change:
* WHAT gets broadcast is untouched; nothing is dropped.
* The ok-gated last_broadcast_at stamp still uses the tick's `now`.
* A failed dispatch sent no packet, so it does not arm the gap.
* Rows filtered by terminate_when/render never burn a spacing slot.
* A lone eligible fire has nothing to pace against -> zero added latency.
* The wait is interruptible by stop(): a 15-fire roll-call holds
tick_once() for ~14 min and stop() awaits the tick task, so a plain
sleep would stall shutdown.
Chose in-loop spacing over routing reminders through FirePacer itself:
reminders re-derive their targets from live DB state every tick and only
clear a row via last_broadcast_at after a confirmed send, so enqueuing
into a 60s-drain FIFO would re-enqueue the same fire on every intervening
tick -- the queue would grow faster than it drains. pacer.py, consumer.py,
store.py and main.py are untouched.
Tests fake the clock end-to-end, so 60s spacing costs the suite nothing.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|||
|
afd045aa96 |
fix(gating): firms decide() spotting/halt severity key, issue #121 (#126)
gating/firms.py's decide() stamped a plain "severity" key in the
data_patch for the wildfire_spotting and wildfire_halted broadcast
paths. central/consumer.py only ever promotes data["_severity_override"]
onto Event.severity -- the plain key is a silent no-op, the same class
of bug as #118 (fixed for firms_handler.py's own inline stamps in
PR #120). Currently inert (MESHAI_CUTOVER_CATEGORIES is unset by
default), but the moment wildfire_spotting/wildfire_halted are cut
over, spotting would silently stop being "immediate".
- gating/firms.py: both data_patch sites now use _severity_override.
Checked the other gating modules (fire.py, avalanche.py, swpc.py,
quake.py, nws.py) -- all already use _severity_override correctly;
firms.py was the only one with the plain-key mistake.
- Fixed the stale module docstring claiming the unattributed-hotspot
cluster path "is DEAD" -- it has been live since
|
|||
|
a85de23af7 |
fix(meshcore): un-shadow _resolve_contact; restore PR #56 refetch-on-miss (#127) (#133)
MeshCoreTransport defined _resolve_contact TWICE: - L171 (PR #56) cache lookup -> on miss, get_contacts(lastmod=0) full refetch -> retry. The DM / path-establishment resolver. - L1227 (PR #92) key-prefix -> by-name lookup, no refetch. The telemetry resolver, added later without noticing the collision. Python silently keeps only the LAST definition in a class body, so the line-171 implementation was dead code and PR #56 was nullified: every DM and path-establishment caller was getting the telemetry resolver instead. No error, no warning, invisible to the linter and the type checker. Fix: rename the telemetry resolver to _resolve_contact_for_telemetry and repoint its sole caller (_req_telemetry_async). The DM path (send_message) and _establish_direct_path now get PR #56's refetch-on-miss behavior back, which is what they need — replying to an inbound DM from a firmware auto-added contact requires the refetch, and re-resolving after path discovery is pointless without it. Deliberately NOT merged into one resolver: telemetry auto-polls on a timer against operator-selected contacts already in the roster, so a full-roster refetch on every miss is recurring airtime for nothing; and its by-name fallback is telemetry-specific and must not widen DM address resolution. The two want different semantics — the bug was the name collision, not that they should be one function. _resolve_contact_async (the MC-event-loop twin) already carried the refetch and was never shadowed, so the async/queue DM send path was unaffected. Add tests/test_no_duplicate_methods.py: AST-walks every ClassDef under work/meshai/ and fails if any class body defines the same method name twice. This failure mode is invisible to review, the linter, and the type checker — which is exactly why it survived. Exempts the legitimate same-name patterns (@property/@setter/@deleter groups, @overload stacks). Verified it flags the bug on the pre-fix source and finds no other duplicates in the tree. Suite: 20 failed -> 17 failed (the 3 meshcore failures gone), 2240 -> 2245 passed (+3 fixed, +2 new guard tests), 72 skipped unchanged. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
|
74a5fa44d4 |
fix(pipeline): let wildfire_spotting skip the grouper (category-scoped) (#131)
wildfire_spotting is the most urgent signal in the system (a fire
throwing embers past its own containment line), but every fire event
carries a group_key (= event_id), so the Grouper held spotting for the
full grouper_window_seconds (60s live) before it could even reach the
dispatcher -- a ~60s latency FLOOR, and ~120s+ once FirePacer queuing
is added on top.
Add a module-level _NEVER_COALESCE_CATEGORIES frozenset and bypass the
coalescing window for the categories in it. Currently: wildfire_spotting
only.
This is safe because spotting is already rate-limited AT THE SOURCE: a
per-fire 1h cooldown (adapter_config.fires.spotting_cooldown_seconds,
default 3600, latched on fires.last_spotting_broadcast_at per irwin_id
in gating/firms.py) gates spotting DETECTION itself, so N active fires
yield at most N spotting alerts per hour. FirePacer (60s interval, with
head-of-line ordering for immediate severity) and the dispatcher's
per-(toggle, category, region) cooldown still apply downstream.
The bypass is scoped by CATEGORY, never by severity. Commit
|
|||
|
c82cceffde |
test: fix 13 stale tests in the red suite, leave 6 real-bug failures (#124) (#129)
Triaged all 20 known-red tests. 13 were stale tests asserting rotted expectations against deliberate, documented behavior changes; fixed by deriving expected values instead of hard-coding, or updating the expectation to match a documented policy change: - test_adapter_config_foundation.py / test_adapter_config_api.py: REGISTRY/API key-count and key-set guards hard-coded magic numbers (59/94/17) that rotted repeatedly. Now derive expectations from REGISTRY itself and, for the schema version, from the migrations directory, so they can't rot the same way again. - test_fire_tracker_phase4.py: two tests hardcoded a nonexistent deployment path (/opt/meshai/meshai/router.py) that matches no Dockerfile WORKDIR in this repo; resolve the module path via importlib.util.find_spec instead. - test_tombstone_broadcast.py: asserted fire severity == "immediate", which commit |
|||
|
8ba8700466 |
fix(fires): wire FirePacer into the native fire broadcast path (#123)
Native fire adapters (env/fires.py source="nifc", env/firms.py source="firms") emitted straight to the EventBus from EnvironmentalStore._emit_event with no rate limiting of their own. FirePacer was only ever attached to CentralConsumer (main.py), which never runs in the actual production deployment (central.enabled=False, all adapters feed_source=native) -- so the <=1/60s throttle + immediate head-of-line behavior fixed for Central in #120 (issue #119) was completely inert in production. A poll that produces several distinct fires/clusters at once (a lightning outbreak, or several tracked fires crossing a satellite-pass boundary together) would dump all of them on the mesh back-to-back instead of at the intended cadence. _emit_event() now routes fire-family Events (source in {"nifc","firms"}, severity in {"priority","immediate"}) through an attached FirePacer, mirroring the exact gate CentralConsumer._handle applies. main.py attaches the same FirePacer instance to env_store right after constructing it. "routine"-severity fire events, non-fire native adapters, and the Central path are all unaffected; a paced event cannot re-enter either gate (native vs Central are mutually exclusive per feed_source), so nothing can be paced twice. Added tests/test_native_fire_pacer.py covering: native fire events route through the pacer, an immediate event jumps an already-queued priority queue with nothing dropped, routine-severity fire events and non-fire native events are NOT paced, and the no-pacer-attached fallback is unchanged. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
|||
|
5dd8266abe |
fix(firms): repair the FIRMS fire-fusion Event contract (issues #117-#119) (#120)
Three independent bugs kept firms_handler's growth/spotting/halt/cluster fusion decisions from reaching a correct mesh Event: - #117: consumer._normalize() computed `category` from the raw Central category BEFORE the per-adapter handler ran and never re-read data["category"] afterward, so every firms_handler category stamp was a silent no-op. Now re-read post-dispatch, validated against the known category registry (unrecognized overrides are logged and ignored). - #118: consumer.py only ever honors data["_severity_override"], but firms_handler's halt/spotting/cluster sites stamped the plain data["severity"] key instead (only growth used the right key). Switched all three sites to `_severity_override` for one consistent contract. This is severity plumbing only -- it does not change which events fire. - #119: FirePacer's gate only matched source in ("fires","wfigs") at severity=="priority", so FIRMS fusion broadcasts (source="firms", growth/spotting at "immediate") never reached the pacer. Broadened the gate to cover "firms" + {"priority","immediate"}, and gave FirePacer head-of-line insertion so an "immediate" event is never stuck behind already-queued "priority" events. Still unbounded/never-drops. Cluster detection is left exactly as main ships it: live, always on, no toggle (PR #73's curated new-fire cluster broadcasts with cold-start silent-seeding). Only its severity-override key changes, under #118. Updated existing tests that asserted the old (buggy) data["severity"] contract, and added tests/test_firms_fusion_event_contract.py covering all three fixes end-to-end through consumer._normalize()/_handle() and FirePacer directly. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
