diff --git a/engine/lint-report.md b/engine/lint-report.md index c082e93..94a5a6d 100644 --- a/engine/lint-report.md +++ b/engine/lint-report.md @@ -1,6 +1,6 @@ # Vault Lint Report -Generated: 2026-07-02T06:00:06Z | Docs scanned: 99 | Elapsed: 0.0s +Generated: 2026-07-02T12:00:05Z | Docs scanned: 99 | Elapsed: 0.0s ## Summary diff --git a/vault/.obsidian/workspace.json b/vault/.obsidian/workspace.json index 8233811..f4e17bb 100644 --- a/vault/.obsidian/workspace.json +++ b/vault/.obsidian/workspace.json @@ -199,19 +199,21 @@ }, "active": "17bd4a6166f789d0", "lastOpenFiles": [ - "docs/services/services.md.tmp.246153.3cd59d138fdb", - "docs/services/services.md.tmp.246153.f8165c27031c", - "docs/hardware/ip-allocation.md.tmp.246153.ee4d9596c7c1", - "docs/hardware/ip-allocation.md.tmp.246153.5b0369a0250b", - "docs/hardware/environment.md.tmp.246153.4af70760f6b1", + "projects/meshcore-transport.md.tmp.5281.cb5cb7900929", + "projects/meshcore-transport.md.tmp.5281.7a5287ef0bf2", + "projects/meshcore-transport.md.tmp.5281.43dbe09b3b59", + "projects/meshcore-transport.md.tmp.5281.d9ef6ce168ab", + "projects/meshcore-transport.md.tmp.5281.017d34de615d", + "projects/meshcore-transport.md.tmp.5281.1f9a0c39eb6d", + "projects/meshcore-transport.md.tmp.5281.72fa3b1103cf", + "projects/meshcore-transport.md.tmp.5281.ecb9b7a692b9", + "projects/meshcore-transport.md.tmp.5281.d180bc1499a4", + "projects/meshcore-transport.md.tmp.5281.6ebd852456da", + "projects/meshcore-transport.md.tmp.5281.ecbb2b58489d", + "projects/meshai.md", + "projects/meshcore-transport.md", "runbooks/pymc-repeater-kiss-tnc-reenumeration.md", - "runbooks/pymc-repeater-kiss-tnc-reenumeration.md.tmp.5281.e47d0cbf9c9a", "runbooks/headscale-oidc-boot-order.md", - "runbooks/headscale-oidc-boot-order.md.tmp.246153.e2a5642e10dd", - "runbooks/headscale-onboard-node.md.tmp.246153.9adf70459e77", - "runbooks/headscale-onboard-node.md.tmp.246153.0132e66f616c", - "runbooks/headscale-onboard-node.md.tmp.246153.52329ab2810a", - "runbooks/headscale-onboard-node.md.tmp.246153.033a7189f5f5", "runbooks/headless-browser-page-verification.md", "runbooks/central-deploy-cutover.md", "runbooks/fleet-magicdns-resolved-migration.md", @@ -234,8 +236,6 @@ "concepts/split-dns.md", "concepts/vector-database.md", "concepts/meshtastic.md", - "concepts/ocr.md", - "archive/projects/mmud/mmud-prompts/mmud-prompts/01-update-planned.md", "assets/echo6yellow_logo_422x422_square.png", "assets/echo6yellow_logo_422x81.png", "assets/echo6_logo.png", diff --git a/vault/projects/meshai.md b/vault/projects/meshai.md new file mode 100644 index 0000000..86c8624 --- /dev/null +++ b/vault/projects/meshai.md @@ -0,0 +1,63 @@ +--- +title: meshai +type: project +tags: + - mesh + - ai +aliases: + - meshai + - MeshAI +related: + - [[services]] + - [[meshcore-transport]] + - [[central]] +updated: 2026-07-02 +--- + +# meshai + +meshai is the LLM-powered LoRa-mesh assistant/gateway for the local Treasure/Magic Valley Meshtastic mesh — it ingests real-world event feeds and broadcasts concise alerts, and answers interactive LLM queries from mesh users (see [[services]]; MeshCore dual-transport design in [[meshcore-transport]]). + +## What it is + +meshai (bot name **AIDA**) attaches to the mesh as a physical node and does two jobs at once: + +- **Outbound alerts.** It consumes real-world situational-awareness feeds (weather, wildfire, traffic/511, space weather, earthquakes, water, satellite passes, …), normalizes them into events, and broadcasts short, chunked alerts to the mesh. +- **Interactive assistant.** Mesh users message it directly and get data-driven LLM answers over LoRa — mesh-health questions ("how's the mesh?"), knowledge/weather queries, and command handlers (`!health`, `!region`, `!neighbors`, `!fires`, `!quakes`, `!space`, `!water`, subscriptions, etc.). + +The LLM backend is Gemini 2.5 Flash with Google Search grounding (multi-backend capable — Gemini / OpenAI / Anthropic / local via LiteLLM). Conversation memory is a rolling window persisted to SQLite. + +## Where it runs + +- **Host:** utility, **CT 108** — `192.168.1.144` / Tailscale **`100.64.0.32`**. +- **Runtime:** Docker, image **`work-meshai`** (local build, *not* `ghcr.io/zvx-echo6/meshai:latest`). Working tree + compose live in `/home/zvx/meshai/work/`. +- **Ports:** `8080` — dashboard / web API (`/api/status`); `7682` — config TUI (ttyd, `meshai --config`). +- **Liveness:** container healthcheck is a PID-file liveness probe (`/tmp/meshai.pid`), not an HTTP endpoint. +- **Mesh link:** connects over **IP to MeshMonitor's virtual-node (vnode) service**, which fronts the actual Meshtastic radio (a meshtasticd node — ultimately AIDA-N2 / channel 8 on **aida-nebra**). Everything is over the network; there is no radio physically attached to the meshai host. meshai *can* also connect directly to a meshtasticd over TCP, but the deployed configuration routes over IP/TCP to MeshMonitor's virtual node. +- **Source:** GitHub `origin` = `zvx-echo6/meshai`. Deploy = git pull + `docker compose build && up -d` (survives reboot). +- **Distinct from** the [[central]] service (utility CT 104) — meshai is a *consumer* of Central's feed, not Central itself. + +## Architecture + +The pipeline is feeds → events → notifications → mesh, plus a separate inbound interactive path. + +- **Central consumer** (`meshai/central/`). `CentralConsumer` (`consumer.py`) subscribes to [[central]]'s NATS JetStream firehose and hands each event to a per-adapter handler that normalizes it into a meshai event. Real handlers present: + - `nws_handler` — NWS weather alerts + - `wfigs_handler` — WFIGS wildfire perimeters/incidents + - `firms_handler` — NASA FIRMS satellite fire hotspots + - `incident_handler` — traffic incidents / Idaho 511 + - `swpc_handler` — SWPC space weather + - `quake_handler` — earthquakes + - `avy_handler` — avalanche advisories + - `nwis_handler` (+ `idaho_gauge_sites`) — USGS NWIS stream gauges / hydro + - `satpass_handler` (+ `pass_predictor`, `tle_handler`) — satellite pass prediction from TLEs +- **Notifications pipeline** (`meshai/notifications/`). Events flow through a bus/dispatcher with grouping, digesting, inhibition, pacing, scheduling and toggle filtering (`pipeline/`), then a composer → renderers → channels. The **mesh renderer** (`renderers/mesh.py`) does the `(k/N)` chunking that fits an alert into LoRa-sized packets; other renderers cover email/webhook. +- **Transport (outbound).** `MeshConnector` (`connector.py`) wraps the `meshtastic` package (serial/TCP, no BLE) and broadcasts to the mesh channel. Message-size assumptions are currently duplicated across several sites — the consolidation to a single `transport.max_chars` is part of [[meshcore-transport]]. +- **Inbound / interactive.** meshtastic pubsub → `MessageRouter` (`router.py`) builds a `MeshMessage`, routes commands/LLM queries → the LLM → `Responder` (`responder.py`) sends the reply back over the same connector. +- **Network awareness.** meshai pulls a live picture of the mesh from **MeshView** and **MeshMonitor** (`mesh_sources.py`, `meshmonitor.py`, `mesh_health.py`, `mesh_data_store.py`) to feed the LLM situational context and compute a 5-pillar mesh-health score. + +## Active / planned work + +- [[meshcore-transport]] — proposed dual Meshtastic + MeshCore transport (send/receive on both meshes simultaneously; uniform message sizing to the smaller radio budget). Design agreed; open on hardware, the `meshcore` dependency, and branch timing. +- NWS severity normalization — CAP-severity pre-filter removed; NWS breadth is now governed solely by the dashboard Weather toggle threshold (warnings broadcast immediately). +- Fire-path correctness — WFIGS/FIRMS fire correlation and drain/pacer spam controls on the fire feed. diff --git a/vault/projects/meshcore-transport.md b/vault/projects/meshcore-transport.md new file mode 100644 index 0000000..22a838a --- /dev/null +++ b/vault/projects/meshcore-transport.md @@ -0,0 +1,123 @@ +--- +title: MeshCore transport for meshai (dual Meshtastic + MeshCore) +type: project +tags: + - mesh +aliases: + - meshai meshcore transport + - meshai meshcore + - MeshCore transport for meshai +related: + - [[meshai]] + - [[services]] +updated: 2026-07-02 +status: proposed +--- + +# MeshCore transport for meshai (dual Meshtastic + MeshCore) + +Design for adding MeshCore as a second mesh transport to the meshai LLM mesh assistant ([[meshai]]), running alongside Meshtastic. + +## Status + +Proposed / design — not yet built. Decisions below are agreed with Matt; three prerequisites remain open (hardware, dependency, branch timing — see Open items). + +## Goal + +Let meshai speak both **Meshtastic** and **MeshCore**, config-selectable as either or **both at once**. Target: full two-way parity (outbound announcements AND inbound interactive LLM replies) on both networks simultaneously. + +## Key decisions (agreed) + +1. **Uniform messages across both networks.** Every outbound message is formatted ONCE, sized to fit the SMALLER radio's budget (MeshCore, ~120 chars), and sent identically to both meshes. No per-network rendering, no divergent content. This is deliberately chosen over per-transport rendering (which was judged fragile: it would make every message formatter budget-polymorphic, double the test surface, run composition per-transport, and produce divergent content). +2. **Run mode: both simultaneously** — send every announcement to Meshtastic AND MeshCore. "Both" = the existing Meshtastic path (meshai over IP to MeshMonitor's vnode fronting the radio) + a **MeshCore Companion node reachable over IP/TCP**, paralleling that same all-over-IP model. The MeshCore radio need NOT be on the meshai host — it just needs to be network-reachable. +3. **The transport is the only thing that knows its size and the only place that fragments a message** (single source of truth for size = `transport.max_chars`; kill the 4 hard-coded 200s + the stray [:195]). Because messages are pre-fit to the smaller budget, both radios normally send a single packet; the transport's (k/N) splitter is a safety net. +4. **Network awareness is per-network, not uniform.** Outbound *announcements* (weather/fire/511/alerts) are outside-world info → uniform to both. But *network-status* answers from the LLM ("who's on the mesh / mesh health") are inside-the-network info and must be scoped to whichever network the question arrived on. A MeshCore user gets MeshCore answers; a Meshtastic user gets the existing MeshView/MeshMonitor-based answers. + +## Character limits + +- Meshtastic text payload: ~200–237 bytes (meshai currently budgets 200). +- MeshCore: hard cap `MAX_TEXT_LEN = 160 bytes`; usable channel-message text is ~120–150 chars (channel messages embed `: ` + timestamp/flags), docs round to "~133". No auto-chunking in the companion chat path — over-length text is hard-truncated, so meshai must split itself. +- Practical uniform budget when both are active: ~120 chars. + +## Companion vs client (how we connect to MeshCore) + +MeshCore firmware is role-specific (flashed, not runtime): **Companion**, **Repeater**, **Room Server**. The **Companion** node is the one a computer attaches to and drives — analogous to Meshtastic's phone+node model. meshai attaches to a Companion-firmware radio via the official `meshcore` Python lib (asyncio). **TCP to a Companion node is the natural fit** here — either native-TCP MeshCore firmware or a `ser2net`/serial-to-IP bridge — matching meshai's all-over-IP deployment (the Meshtastic side already runs over IP to MeshMonitor's vnode). A USB-on-host radio is not how this deployment works. Avoid BLE on Linux. Broadcast to a channel with `send_chan_msg(index, text)`; channel 0 = "Public" (well-known PSK) = the broadcast primitive. Do NOT attach to a Repeater or Room Server for messaging. + +## Connecting to MeshCore via pyMC (companion TCP frame server) + +- meshai's MeshCore link will be a **TCP connection to the pyMC repeater's companion frame server**, not a directly-attached radio. "pyMC" = **pyMC_core** (Python MeshCore library) + **pyMC_Repeater** (the repeater daemon, also branded "openHop"), which runs on **aida-nebra** as `pymc-repeater.service` (node EchoBase, RAK4631). pyMC_core includes a `CompanionFrameServer` that exposes the MeshCore **Companion Protocol frames over a TCP socket** — so a companion client connects over TCP as if to a radio. +- **Wire compatibility is confirmed:** the `meshcore` Python lib's TCP transport (`tcp_cx.py`) is byte-for-byte compatible with pyMC's frame server — outbound `>`(0x3E)+LE16+data, inbound `<`(0x3C)+LE16+data. So meshai connects with `MeshCore.create_tcp(host, port)`; no framing shim needed. +- **Target:** meshai connects to the companion identity named **AIDA** on the pyMC repeater — aida-nebra at **100.64.0.9** over Tailscale, **port 5050** (the AIDA companion on aida-nebra; pyMC's out-of-box default is 5000). The repeater's radio node is **EchoBase**; **AIDA** is the companion frame-server identity meshai attaches to (set under `identities.companions[]` in the repeater's config). The frame server is confirmed enabled. No authentication on the frame server. Endpoint: `100.64.0.9:5050`. +- **⚠️ Single-client gotcha (important):** pyMC's companion frame server is **single-client** — a new connection **evicts** the existing one. So meshai and MeshMonitor (which we also want on MeshCore for network awareness) **cannot share one companion frame server** — they'd kick each other off. Resolution: pyMC's config supports **multiple companion identities** (`identities.companions[]` is a list), each on its own `tcp_port` — run a **dedicated companion identity/port for meshai** and a separate one for MeshMonitor. (This mirrors the Meshtastic side, where meshai and MeshMonitor coexist via MeshMonitor's vnode.) meshai's companion endpoint is the **AIDA** identity at `100.64.0.9:5050` — if MeshMonitor also needs MeshCore awareness it must use a **separate** companion identity/port so the two don't evict each other. + +## Network awareness: MeshCore vs Meshtastic + +- **Meshtastic (today):** rich, global, passive picture via MeshView + MeshMonitor (fed by Meshtastic's MQTT firehose). +- **MeshCore:** no MQTT firehose and no passive "every node ever heard" nodeDB — by design (privacy/routing model). Awareness is LOCAL/contact-scoped and mostly PULL-based. +- **Convenient fit:** MeshMonitor (the same tool meshai already uses for Meshtastic) supports MeshCore as a first-class source since v4.5+, over USB/TCP, with a REST API (`/api/nodes`). So MeshCore awareness reuses the existing MeshMonitor pattern. MeshMonitor is already central to meshai's mesh connectivity — meshai's Meshtastic link itself runs over IP through MeshMonitor's vnode — which makes reusing MeshMonitor for MeshCore awareness an especially natural fit. +- **What we CAN give a MeshCore user's LLM query:** contact roster (name, node type, last-advert, position if shared, known path/hops via `get_contacts()`), per-message SNR/RSSI, own device telemetry, on-demand telemetry from other nodes (`req_telemetry`/`req_status`, Cayenne LPP), trace/path discovery. Repeater stats (uptime/airtime/neighbors) only if the operator enabled guest access. +- **What's missing vs Meshtastic:** no global/passive view, no firehose, advert SNR/position not inline, companion has no neighbor table (only Repeaters do). +- **Verdict:** less than Meshtastic's effortless global view, but a real local picture — plan MeshCore awareness around active polling of a curated contact/repeater set, not passive ingestion. + +## Architecture & code seams (current meshai) + +- Today: one concrete `MeshConnector` (`meshai/connector.py`) hard-wired to the `meshtastic` package (serial/TCP; no BLE), imported directly by `main.py`, `router.py`, `responder.py`, `channels.py`, and the pipeline. No transport interface. +- Send path: `MeshConnector.send_message()` → `interface.sendText(...)`; broadcast = `BROADCAST_NUM` on a channel index. +- Inbound: meshtastic pubsub → `_on_receive` builds a `MeshMessage` (carries a meshtastic-specific `packet` dict) → `set_message_callback` → `MeshAIBot._on_message` → `MessageRouter`. +- Size assumptions duplicated in 4 places: `chunker.py` (`max_chars=200`), `MeshRenderer` + its `(k/N)` splitter (`renderers/mesh.py`), digest accumulator (`pipeline/digest.py`), and the newer `single_packet_max_chars` (`adapter_config/defaults.py`, used by `central/nws_handler.py`); plus a stray `[:195]` in `notifications/router.py`. +- Config: dataclasses in `meshai/config.py` (`ConnectionConfig`), loaded from `meshtastic.yaml` via `config_loader.py`. Exactly one connector is built in `main.py`. + +## meshai config: the MeshCore menu + +- There is **no MeshCore code in meshai yet** (greenfield; nothing to reconcile). +- meshai config is **two-layer**: (1) file — `meshai/config.py` `ConnectionConfig` dataclass (`type` serial|tcp, `tcp_host`, `tcp_port`), loaded from `meshtastic.yaml` via `config_loader.py`; (2) **live dashboard** — a React Config page (`dashboard-frontend/src/pages/Config.tsx`) with a "Connection" section, saved at runtime via `PUT /config/{section}` (`dashboard/api/config_routes.py`), with a restart banner when needed. +- The **MeshCore menu** = a new sub-form in the dashboard's Connection section (plus a matching config block). Fields it needs: + - transport enable / select (meshtastic | meshcore | both) + - `connection_type: tcp` (the pyMC path is TCP; BLE/serial exist in the lib but aren't used here) + - `host` (aida-nebra, `100.64.0.9`), `port` (`5050` — the AIDA companion on aida-nebra; pyMC's out-of-box default is 5000) + - `channel_index` (index into the companion's channel table — required to send/receive on a public channel) + - `channel_name` (optional label) + - `channel_psk` / secret — **store in `credentials`, NOT in this doc or any graph-tracked file**; only needed if meshai must provision the channel rather than relying on one pre-configured on the repeater + - `max_chars` (MeshCore per-chunk cap — est. ~130–150; verify against companion max) + - `auto_reconnect` + `max_reconnect_attempts` (passed to `create_tcp`) + - optional self/node name for the companion identity + +## Implementation plan (phased) + +- **Phase 1 — Transport abstraction (pure refactor, testable without new hardware).** Extract `MeshTransport` ABC (`connect/disconnect/send_message/set_message_callback/my_node_id/connected/get_node_name/get_node_position`). Rename `MeshConnector` → `MeshtasticTransport` (behavior identical; assert byte-identical output in tests). Generalize `MeshMessage` (drop meshtastic-only `packet`; add a `transport` tag + generic reply-address token). Config-driven transport factory. Existing tests stay green. +- **Phase 2 — MeshCoreTransport (in + out).** Wrap the `meshcore` lib behind the same sync surface via a dedicated event-loop thread. Outbound `send_chan_msg` for broadcast, DM-by-pubkey for replies. Inbound: MeshCore channel/contact events → normalized `MeshMessage` → same callback. Contact/pubkey handling from adverts; `my_node_id` equivalent = our node pubkey (self-filter). +- **Phase 3 — Uniform sizing + one splitter.** Consolidate the 4 size sites into `transport.max_chars`; message formatting targets the minimum active budget (~120) so output is uniform; transport does final (k/N) safety split. Remove the stray [:195]. +- **Phase 4 — CompositeTransport ("both") + reply routing.** Fan `send_message` to both children; merge both inbound streams; each reply goes back over the transport it arrived on (via the `MeshMessage.transport` tag). +- **Phase 5 — Per-network awareness + config + deploy.** Scope LLM network-status context to the originating network (MeshCore via MeshMonitor REST, mirroring the Meshtastic path). Add `transport: meshtastic|meshcore|both` + a `meshcore` config section (serial/tcp, channel index/PSK, max_chars). Add the `meshcore` pip dependency to the image. Deploy to CT 108 once the second radio is attached; smoke-test both meshes. + +## Open items / prerequisites + +1. **Hardware:** a **MeshCore Companion node reachable over IP/TCP** (native-TCP firmware or a `ser2net`/serial-to-IP bridge). Location is flexible — it need NOT be the meshai host — and no USB/udev pinning is required since the connection is over the network. "Both" can't run live until this exists — Phases 1–4 can be built/unit-tested without it. +2. **Dependency:** adds the `meshcore` pip package to the meshai image (needs approval per no-deps-without-permission policy). +3. **Timing:** Phase 1 touches core files (`connector.py`, `main.py`, `router.py`, `responder.py`, `channels.py`) near the in-flight `fix/fire-spam-drain-pacer` work; land this AFTER that branch merges to main to avoid a refactor conflict on the live host. +4. ✅ **RESOLVED — pyMC companion frame server:** confirmed **enabled** on aida-nebra, on **port 5050**, companion identity **AIDA** (repeater radio node EchoBase, RAK4631). +5. ✅ **RESOLVED — aida-nebra Tailscale IP = `100.64.0.9`** (confirmed authoritatively from Tailscale: `tailscale status`, `tailscale ip -4 aida-nebra`, MagicDNS `aida-nebra.echo6.mesh`, and `getent hosts` all agree). The old 100.64.0.18 figure is wrong. +6. **Single-client separation (still open):** ensure meshai's **AIDA** companion identity/port (`100.64.0.9:5050`) is not also used by MeshMonitor. If MeshMonitor needs MeshCore awareness, it must use a **separate companion identity/port** so the two don't evict each other. +7. Map the repeater's **channel table** (which index = which named channel/PSK). +8. Confirm **MeshCore max text length** to set `max_chars`. + +## Risks + +- Async bridge (meshcore asyncio ↔ meshai sync send). +- Reply routing correctness (right mesh). +- MeshCore firmware/lib version churn — pin the `meshcore` lib + Companion firmware together; constants (`MAX_TEXT_LEN`, opcodes) are firmware-defined. +- Uniform-to-smaller means Meshtastic messages don't use their extra capacity (accepted tradeoff for uniformity). + +## References + +- meshcore Python lib: github.com/meshcore-dev/meshcore_py ; pypi.org/project/meshcore +- Companion protocol: docs.meshcore.io/companion_protocol/ ; firmware: github.com/meshcore-dev/MeshCore +- MeshMonitor MeshCore support: meshmonitor.org/features/meshcore.html +- Roles overview: nodakmesh.org/blog/meshcore-how-it-works-guide +- pyMC_core (companion frame server): github.com/pyMC-dev/pyMC_core — src/pymc_core/companion/frame_server.py +- pyMC_Repeater / openHop: github.com/rightup/pyMC_Repeater +- meshcore_py TCP transport: github.com/meshcore-dev/meshcore_py — src/meshcore/tcp_cx.py + +## Deploy context + +meshai runs on CT 108 (Tailscale 100.64.0.32) as a docker compose build in /home/zvx/meshai/work/ — NOT the Central service on CT 104. Mesh connectivity is all over IP: no radio is attached to CT 108. The Meshtastic link runs over IP to MeshMonitor's vnode (which fronts the radio), and the planned MeshCore link is likewise a Companion node reached over TCP — so neither transport requires hardware on the meshai host. See the deploy topology note.