matt/meshai
1
0
Fork 0
mirror of https://github.com/zvx-echo6/meshai.git synced 2026-08-26 17:31:34 +00:00
Code Issues Projects Releases Packages Wiki Activity
540 commits 59 branches 8 tags 70 MiB
Commit graph

5 commits

Author SHA1 Message Date
malice
c04daa6e4d
 		fix(config): merge partial PUT bodies instead of resetting to defaults (#155) 
Saving the "Auto-advert interval" dropdown on the MeshCore Companion page
took BOTH radios offline on 2026-07-17 06:46:52. One click, full outage.

The page PUT a single-key body to /api/config/connection:

    {"meshcore_advert_interval_seconds": 10800}

_dict_to_dataclass() builds kwargs only from the keys present in the body
and lets `cls(**kwargs)` default the rest, so every OMITTED field was reset
to its dataclass default and written to disk:

    type:                 tcp   -> serial            (Meshtastic offline)
    tcp_host:             192.168.1.100 -> <lost>    (LOCAL_FIELDS, see below)
    tcp_port:             4404  -> 4403              (wrong meshmonitor vnode)
    meshcore_host:        192.168.1.253 -> ''        (MeshCore off; blank = off)
    meshcore_conn_type:   serial -> tcp              (wrong transport)
    meshcore_serial_port: /dev/meshcore-rak -> ''    (RAK radio lost)

It was silent twice over. `connection` is restart-required, so the running
process kept the good in-memory config while the file sat gutted, waiting
for any restart to detonate. And save_section() writes the domain file
FIRST and local.yaml SECOND: meshtastic.yaml hit the disk already gutted,
then the local.yaml write (which owns connection.tcp_host via LOCAL_FIELDS)
died on `[Errno 13] Permission denied` -- so tcp_host landed in neither
file, and the 500 that would have named the cause was swallowed by the UI.
The operator saw nothing happen.

This was never one page's bug: PUT /api/config/{section} was destructive on
a partial payload for EVERY section. Other callers only survive because they
happen to spread the full object first.

Fixes, in depth:

* Route (the durable fix): merge the body over the CURRENT live section
  before coercing, so omitted keys keep their live values while present
  keys -- including '' / False / [] -- still apply. The base is the live
  config, the same values GET serves, so a partial PUT now lands exactly
  where a full-object PUT from that same GET would. Full-object callers are
  unaffected. Fixed at the HTTP boundary, not in _dict_to_dataclass():
  absent-key-means-default is CORRECT at config-load time, where a file
  legitimately omits fields it does not override.

* Nested semantics keyed off the dataclass schema, not "is it a dict":
  nested dataclass fields DEEP-MERGE (a partial region_routes must not drop
  sibling cells), while bare dict/list fields REPLACE at the key (cells,
  toggles, destinations, rules are dynamic maps -- deep-merging them would
  resurrect deleted keys and make deletion impossible, the mirror image of
  the bug being fixed).

* Page: send the full connection object like every other caller does.

* Errors are visible: the save handler no longer swallows the exception,
  and updateConfig() surfaces the server's `detail` rather than a bare
  "API error: 500", which is what hid Permission denied from the operator.

* Default advert interval 10800 -> 86400 (24h). 3h is far too frequent a
  default for a public mesh; the UI "(default)" label moves to match.

Tests: tests/test_config_partial_save_merge.py reproduces the outage with
the exact payload, and pins merge semantics across connection AND
notifications, intentional clearing, deep-merge, and map-deletion.

Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-17 01:28:41 -06:00
malice
ea4c010967
 		feat(meshcore): report the true connection + add roster/channel management (#153) 
self_info() reported host/port straight from config regardless of
conn_type, so a serial companion still advertised whatever stale
meshcore_host sat in the config — the API named a device meshai was not
talking to, which is enough to send an investigation to the wrong radio.
Connection details now come from one _connection_descriptor() shared with
connect(), so the log line and the API can't drift; only the live
conn_type's fields are populated and the rest are null.

meshai's device view is otherwise built once at connect and never re-read
— contacts via ensure_contacts(), channels via _enumerate_channels(). The
lib's contact handler only ever merges (meshcore.py::_update_contacts), so
a cached roster can never shrink, and a channel provisioned on the radio
stays invisible until the process restarts. There was no refetch path at
all. Adds an explicit resync that re-reads BOTH halves: a FULL
get_contacts(lastmod=0) reconciled with replace semantics (absent contacts
are dropped) plus a channel re-enumeration, each reporting what changed.

Also adds a preventive route-health check: every region_routes cell whose
MeshCore target cannot be resolved against the live roster/channel table
is surfaced, since such a send fails silently. Room targets are matched by
pubkey prefix, exactly as the dispatcher resolves them, so a picker-stored
prefix is not misreported as dangling. Same-name/different-pubkey roster
entries are flagged too — a name alone cannot identify a contact, which is
the trap behind a room rebuilt under a new keypair.

Backend:
- meshcore_roster.py: pure reconcile_contacts / check_route_health /
  find_name_collisions (no device I/O — unit-testable without a radio)
- transport: _connection_descriptor, resync, refresh_contacts,
  remove_contact, import_contact, export_roster, contacts_synced_at;
  auto_update_contacts enabled (configurable — it costs one incremental
  fetch per advert heard, which is real chatter on a dense mesh)
- API: POST contacts/refresh, DELETE contacts/{pubkey}, GET
  contacts/export, POST contacts/import, GET route-health

Frontend (existing Contacts & Companion page — no new page or nav entry):
- dangling-route + name-collision banners; resync/export/add-contact
  toolbar with last-synced and the added/removed counts; staleness badges;
  search, filters and sortable columns; per-contact delete behind a
  confirm; Companion tab shows the real transport + target.

A full pubkey is required to delete or add: the lib resolves by prefix,
and a prefix could silently hit the wrong node.

Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-16 22:24:49 -06:00
malice
a7b7f5a6a4
 		feat(transport): per-radio serialized+paced outbound send queue (#93) 
* feat(transport): per-radio serialized+paced outbound send queue

Prevents simultaneous LoRa transmissions when N events arrive at once.

## Mechanism

Two `RadioSendQueue` instances (one MT, one MC), each a FIFO asyncio.Queue
with a long-running drain task.  The MT queue drains on the main asyncio
loop; the MC queue drains on MeshCore's dedicated event-loop thread.

- MT sends: `run_in_executor` offloads the blocking `sendText` call;
  queue started in `set_message_callback`, cancelled in `disconnect`.
- MC sends: drain loop runs pure-async MC lib coroutines directly on the
  MC loop (no `_run_coro` deadlock); cross-loop callers bridge via
  `concurrent.futures.Future` + `asyncio.wrap_future`.
- Pacing: `await asyncio.sleep(pacing_seconds)` between items; read live
  from config per iteration; floor clamped to 0.25 s.
- Config knobs: `meshtastic_send_pacing_seconds` (default 2.0) and
  `meshcore_send_pacing_seconds` (default 2.0) on `ConnectionConfig`.

## Send sites rerouted

All callers now `await connector.send_message_async(...)`:
- `notifications/channels.py` — MeshBroadcast/MeshCoreBroadcast/MeshDM/
  MeshCoreDM deliver(), test_connection(), deliver_test()
- `responder.py` — DM replies in send_response()
- `transport/meshcore_transport.py` — periodic_advert_loop, telemetry
  poll loop, send_advert() → send_advert_async(), req_telemetry()
  → req_telemetry_async() (all queue-routed from main loop)
- `dashboard/api/mesh_send_routes.py` — test-send, advert, telemetry poll

## Audit accuracy

`deliver()` now returns the actual bool from the radio send (not
optimistic True), so `mesh_broadcasts_out` reflects the real result.

## Tests

17 new tests in tests/test_send_queue.py covering FIFO ordering, no drops,
pacing gap, pacing floor enforcement, event-loop non-blocking, serialization,
lifecycle, MT fallback, config round-trip.  Existing test stubs updated to
wire `send_message_async = AsyncMock(side_effect=send_message)` so prior
call_count / call_args assertions remain valid without changes.

Full suite: 2135 passed, 17 pre-existing failures (unchanged), 0 new regressions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(send-queue): resolve MC telemetry self-deadlock + resolve pending futures on teardown/reconnect; composite MC-channel kwarg; audit no-op false

BLOCKER 1 — req_telemetry_async self-deadlock (meshcore_transport.py):
_req_telemetry_async was calling _enqueue_mc_loop_send inside itself;
when _telem_job_outer ran inside the drain it nested another enqueue+await
on the same single-threaded drain — permanent deadlock on first telemetry poll.
Fix: _req_telemetry_async is now fully inline (no _enqueue_mc_loop_send).
_telemetry_poll_loop wraps its call in _enqueue_mc_loop_send for serialization.
req_telemetry_async's outer job calls _req_telemetry_async inline (safe).

BLOCKER 2 — pending futures abandoned on teardown/reconnect:
RadioSendQueue.stop() only cancelled the drain task; queue-sitting items had
their concurrent.futures.Futures left unresolved, causing wrap_future() callers
to hang indefinitely. Fix: stop() drains the remaining queue with get_nowait()
and cancels every pending cfut. _cancel_mc_queue() schedules the same drain-
and-cancel via call_soon_threadsafe. _start_mc_queue() cancels old drain task
and drains old queue cfuts before arming the new queue (reconnect path).
connector.disconnect() now .result(timeout=5) on stop() instead of fire-and-forget.

SHOULD-FIX 3 — composite passes MC channel as wrong kwarg (composite_transport.py):
_broadcast_async no-hint loop was calling send_message_async(channel=child_channel)
for the meshcore child; should be meshcore_channel=child_channel. Silent drop fixed.

NIT 5 — false success on zero-channel MC send (meshcore_transport.py):
send_message_async returned True when meshcore_channel is None (nothing sent).
Now returns False so audit does not record a success for a no-op.

NIT 7 — config comment contradiction (config.py):
meshtastic_send_pacing_seconds comment said "0 disables the floor" while
simultaneously stating "still floored at 0.25". Removed the contradiction.

Regression tests (tests/test_send_queue.py — 3 new, all in TestDeadlockRegression):
- test_telemetry_queue_no_deadlock: drives req_telemetry_async through a real
  _mc_send_queue with fake MC commands; times out on pre-fix code (deadlock).
- test_teardown_resolves_pending_futures: enqueues slow+fast jobs, stops mid-drain,
  asserts every task resolves promptly; hangs on pre-fix code.
- test_reconnect_resolves_old_futures: calls _start_mc_queue twice, asserts old
  cfuts are cancelled; pre-fix leaves them unresolved.

All 17 pre-existing send-queue tests still pass (20 total now).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-07-08 09:38:08 -06:00
malice
284fb5cbf2
 		MeshCore Contacts roster + Companion status (read-only) (#17) 
* feat(dashboard): MeshCore Contacts roster + Companion status (read-only)

Expose the live companion's contact roster (get_contacts) and self/channel
status via /api/meshcore/contacts + /api/meshcore/self. Fill the Contacts
(roster table) and Companion (status + channels) pages. Telemetry auto-poll
comes next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(meshcore): self-advertisement (send-advert + advert-on-connect + periodic)

AIDA now announces itself: send_advert(flood=True) on every connect, an
optional periodic auto-advert (meshcore_advert_interval_seconds), and a
manual "Send Advert" button + POST /api/meshcore/advert. Makes the
companion discoverable/DM-able on the mesh.

---------

Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-03 16:14:10 -06:00
malice
11bac716d0
 		feat(dashboard): send-test-message + MeshCore channel list (#14) 
Add POST /api/mesh/test-send (fire a labeled test broadcast on a chosen
mesh+channel via the live transport) and GET /api/meshcore/channels
(surface the companion's enumerated channel names). "Send test message"
cards on both Connection pages, with the MeshCore one listing real
channels. Lets the operator confirm a mesh's send path on demand.

Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-03 01:06:31 -06:00