Commit graph

96 commits

Author SHA1 Message Date
a97d346449 feat: Add node-to-node distance calculation
- Add build_distance() method to MeshReporter
- Uses _haversine_km() for GPS distance calculation
- Formats output with both km and miles
- Handles missing nodes or GPS positions gracefully

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 22:16:43 +00:00
79ff756a38 fix: Remove hard-coded token limits on LLM responses
- Add max_response_tokens config (8192) to LLMConfig
- Use config value in router.py instead of hardcoded 500
- Update base.py default from 300 to 8192
- Lets LLM generate full responses; chunker handles size limits

Fixes truncated responses like Here are three nodes in the freq

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 22:06:50 +00:00
70c0ab3047 fix: Restore build_lora_compact, list_regions_compact, build_region_compact, build_node_compact
These methods were called by commands/health.py and main.py but missing
from mesh_reporter.py, causing crashes on !health and !region commands.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 21:50:18 +00:00
99c952b432 feat: Show hop count and GPS distance in node detail and single-gw listings
- Added hops_away to Connectivity section in node detail
- Added nearest infra distance after Position in node detail
- Added distance from reference infra to single-gw client listings
- Added _haversine_km and _format_distance helper functions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 21:16:55 +00:00
6ac21d5f0e fix: LLM prompt — use local names, concise node listings, no redundant regions
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 21:09:21 +00:00
29a57b459a fix: 6 reporter bugs — missing methods, undefined var, indentation, key types
1. Added _find_node() — delegates to health_engine.get_node()
2. Added _find_region() — fuzzy match with config aliases
3. Fixed undefined unified var in _node_recommendations
4. Fixed env recommendation indentation (was inside MQTT uplink check)
5. Fixed 6 string-vs-int key mismatches on health.nodes lookups
6. Fixed estimated_position_interval — compute inline from packets_by_type

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 20:58:41 +00:00
cc474e3bb3 feat: Show source name for single-gateway nodes
Single-gateway nodes now display "via {source_name}" to indicate
which data source they depend on. This helps identify coverage gaps
and understand node visibility.

Adds source info to:
- Tier 1 region summary (infra and client nodes)
- Node distribution section (infrastructure nodes)
- Region detail view (single gateway list)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 20:23:18 +00:00
1d9c90911b feat: Feeder-level gateway awareness from /api/packets_seen
Samples recent packets and calls /api/packets_seen to discover which
physical MQTT gateways hear each node. Per-gateway RSSI and SNR.

UnifiedNode:
- feeder_gateways list with gateway_id, gateway_name, avg_rssi, avg_snr
- feeder_count, feeder_best (strongest signal), feeder_worst

MeshviewSource:
- Added feeders to ENDPOINT_SCHEDULE (every 20 ticks / 10 min)
- _fetch_feeders() samples 20 packets and queries packets_seen
- Auto-disables if endpoint returns 404

MeshDataStore:
- _enrich_feeder_data() aggregates gateway data across all sources
- _normalize_node_id() helper for hex/decimal conversion
- get_feeder_map() shows per-gateway coverage statistics
- get_node_feeders() returns sorted gateway list for a node

MeshReporter:
- Node detail shows feeder gateways with signal strength
- Tier 1 shows total unique gateways and avg per node

Discovered gateways: AIDA, BKBS, STLR, N7MH, stor, JTS

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 19:55:23 +00:00
b3c79f12da feat: Tick-based staggered polling for all sources
Both MeshviewSource and MeshMonitorDataSource now use tick-based
staggered polling instead of batch-every-5-minutes:

MeshviewSource (30s ticks):
- Packets: every tick (30s)
- Nodes: every 4 ticks (2 min)
- Stats/Edges: every 6 ticks (3 min)
- Traceroutes: every 10 ticks (5 min)

MeshMonitorDataSource (30s ticks):
- Packets: every 2 ticks (60s)
- Nodes/Telemetry: every 4 ticks (2 min)
- Traceroutes/Channels/Network/Topology: every 10 ticks (5 min)
- Solar: every 20 ticks (10 min)

Features:
- Source health status (avg_response_ms, tick_count, backed_off)
- Source coverage analysis (unique vs shared nodes)
- Tier 1 DATA SOURCES section shows all source health
- Node detail shows source visibility
- Incremental packets and telemetry with dedup
- Rate limit detection (429) with backoff
- Consecutive error exponential backoff
- polite_mode config option for shared instances

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 19:16:00 +00:00
bc644b3ac2 fix: Name single-gateway client nodes in Tier 1 (top 10 per region)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 17:04:49 +00:00
aef78996a9 feat: Named single-gw nodes, infra-only monitoring, commands overhaul
Coverage:
- Single-gateway infra nodes named as critical risks per region
- Client single-gw nodes counted but not individually named
- Mesh-wide single-gw infra summary

Monitoring rules by node type:
- Infrastructure: full detail - battery, offline, coverage, neighbors, hardware
- Clients causing problems: named - high util, top senders
- Clients otherwise: counted per region, not individually tracked
- POWER breakdown now infra-only

Commands:
- Removed hardcoded command list from config.py system_prompt
- Dynamic command list in router.py from dispatcher (only enabled commands)
- MeshMonitor commands no longer listed as MeshAI commands
- !help overhaul: grouped by category, per-command detailed help
- LLM explicitly told to only mention listed commands

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 15:33:14 +00:00
56536f55c8 feat: Rich Tier 1 data - named infra per region, problem nodes, expanded recommendations
Tier 1 now includes:
- Every infrastructure node BY NAME per region with status/battery/util/gateways
- Problem nodes section: offline infra, critical battery, high util, coverage risks
- Per-region coverage with gateway counts and single-gw counts
- Environmental data per region
- All 5 pillars with weights
- Expanded recommendations with specifics (10 max, up from 5)
- LLM prompt simplified: data speaks for itself

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 14:32:43 +00:00
8d1a48ea08 fix: Short sentence instruction + chunker splits instead of truncating
- Added CRITICAL instruction to keep sentences under 150 chars
- Chunker now splits long sentences at word boundaries instead of truncating
- No words lost when splitting

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 07:22:52 +00:00
51969050da fix: Replace deleted REGION_CONTEXT with config method, replace hardcoded city map with config aliases
- Line 763: REGION_CONTEXT.get() → self._region_context() (same method used elsewhere)
- Deleted _CITY_TO_REGION hardcoded dict
- Scope detection now uses config aliases/cities from RegionAnchor
- Fixed Sun Valley/Ketchum geography (was Central ID, should be South Central ID)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 07:11:08 +00:00
de400068dd feat(mesh): config-driven regions with stale purge and coverage fix
- Extended RegionAnchor with local_name, description, aliases, cities
- Moved region geographic context from hardcoded Python to config.yaml
- Added 7-day stale node purge in _do_refresh (556 → 267 nodes)
- Fixed coverage lookup: str(node_num) → node_num (int key)
- Added bidirectional neighbor lookup for better region assignment
- Dynamic geography building in router from config
- Reporter reads region context from config instead of hardcoded dict

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 06:33:38 +00:00
abcf2d88e2 feat: Geographic context — local names, accurate Idaho/Utah geography, no Unknown gaps
- Magic Valley, Treasure Valley, Panhandle, Dixie — local names in all output
- Southern Idaho correctly maps to Magic Valley, not lumped with Boise/IF
- Unlocated nodes excluded from coverage gap recommendations
- LLM response rules override brevity for mesh questions
- Node detail shows region with local context

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 06:08:58 +00:00
45630f2cc6 fix: Override LLM brevity for mesh questions — give detailed data responses
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 05:29:09 +00:00
69f3d5d284 fix: Replace 5 remaining node_id refs with node_id_hex after NodeHealth removal
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 05:24:26 +00:00
4183abe755 refactor: DELETE NodeHealth — reporter uses UnifiedNode directly
NodeHealth is gone. MeshHealth.nodes is now dict[int, UnifiedNode].
Reporter reads all fields from UnifiedNode: coverage, environment,
neighbors, hw_model — everything available without cross-referencing.
This eliminates the entire category of field missing on NodeHealth bugs.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 05:12:45 +00:00
a384fd7a20 Implement 5-pillar health scoring with coverage
- Update HealthScore with coverage pillar (20% weight)
- Adjust weights: Infra 30%, Util 25%, Coverage 20%, Behavior 15%, Power 10%
- Add coverage metrics: avg_gateways, single_gw_count, full_count
- Add health score fields to UnifiedNode for direct sync
- compute() now syncs scores back to UnifiedNode objects
- Coverage scoring penalizes single-gateway nodes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 04:59:41 +00:00
ac2bb87473 fix: Coverage breakdown now uses str(node_num) for health node lookup
Health engine stores node IDs as decimal strings, but UnifiedNode.node_num
is an integer. The lookup health.nodes.get(n.node_num) was failing, causing
all nodes to show as Unlocated. Fixed by converting to string first.
2026-05-05 04:40:12 +00:00
d8189e2e4d fix: Add hw_model and missing fields to NodeHealth
- hw_model, neighbor_count, packets_sent_24h fields
- node_id_hex, battery_trend, packets_by_type, predicted_depletion_hours properties
- Populate hw_model from node data when creating NodeHealth
- Fixes reporter crash on node detail view
2026-05-05 04:26:52 +00:00
e9ddfa7e26 fix: Add node_num property to NodeHealth class
NodeHealth stored node_id as hex string but mesh_reporter.py
expected node_num integer. Added computed property to convert.
2026-05-05 04:16:00 +00:00
cb61c4199c feat: Geographic coverage breakdown in mesh reporter
- Show per-region coverage stats in tier1 summary
- List single-gateway nodes in region detail
- Add coverage status to node detail view
- Add coverage gap recommendations
2026-05-05 04:07:19 +00:00
62f04a3e09 fix: Align parameter names between data_store and source_manager
- create_dispatcher uses data_store=
- MessageRouter uses source_manager=
- Added get_all_traceroutes() and get_all_channels() compat methods

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 03:50:58 +00:00
800e4d9e7e fix: Add get_all_traceroutes and get_all_channels compat methods
Completes backwards compatibility layer for mesh_health.py.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 03:45:48 +00:00
c756727cad feat: Node source overlap for gateway coverage metrics
Replaces broken per-packet gateway sampling with node-level source counting.
Each Meshview/MeshMonitor source represents a gateway view of the mesh.
If a node is seen by N sources, its packets are reaching N gateways.

- Removed _sample_gateway_coverage() (required non-existent API)
- Rewrote _enrich_deliverability() to use node.sources count
- Per-node: avg_gateways, max_gateways, source_reach, deliverability_score
- Mesh-wide: avg 4.16 gateways/node with 7 sources
- Fixed edge.timestamp -> edge.last_seen in get_all_edges()

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 03:31:48 +00:00
f30cd0a8bf fix: Add backwards compatibility methods for mesh_health.py
- Add hasattr check for fetch_recent_packets in gateway sampling
- Add get_all_nodes(), get_all_telemetry(), get_all_packets(), get_all_edges() methods
- These methods return data in dict format expected by mesh_health.py

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 03:09:30 +00:00
af2f66d71d Merge subscriptions from main into feature branch (with full data pipeline) 2026-05-05 02:57:41 +00:00
Ubuntu
b20dea60e2 feat(subscriptions): Add Phase 4 subscription system for scheduled reports
- Create subscriptions.py with SubscriptionManager class for SQLite storage
- Add subscribe.py commands: !sub, !unsub, !mysubs with aliases
- Update dispatcher.py to register subscription commands
- Modify main.py with scheduler tick (60s) and _check_scheduled_subs()
- Add build_node_compact() and build_region_compact() to mesh_reporter.py
- Support daily, weekly, and alerts subscription types
- Support mesh, region, and node scope filtering
- 5-minute matching window for schedule tolerance
- Dedup via last_sent tracking

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-05 02:26:12 +00:00
e06d71036f fix: !neighbors shows infra links without requiring SNR, edge SNR enrichment from traceroutes
- Removed SNR-required filter from !neighbors command
- Show all infra neighbors, add signal quality when available
- Enrich edges with SNR from traceroute snrTowards/snrBack data
- Fallback: use node-level SNR for edges without traceroute data
- Sort by SNR when available, alphabetically otherwise

Co-Authored-By: Claude <noreply@anthropic.com>
2026-05-05 01:18:37 +00:00
8c3b6a1f09 fix: Fundamental ID matching — packets, telemetry, and utilization now work
Root cause: health engine keyed nodes by database row IDs instead of
Meshtastic node numbers. Packets and telemetry could never match.

Fixed:
- Store _node_num on all normalized nodes (mesh_sources.py)
- Key health engine node dict by _node_num (mesh_health.py)
- Fix packet field names: from_node not from/fromId
- Fix telemetry parsing: handle telemetryType/value structure
- Increase packet/telemetry fetch limits for 24h coverage
- Fix utilization formula to compute actual airtime percentage
2026-05-04 21:47:18 +00:00
3959444a09 feat: Complete data pipeline — utilization, behavior, power, solar, traceroutes all wired 2026-05-04 21:22:30 +00:00
df197cc395 fix: Scope detection, follow-up context, utilization calculation, duplicate disambiguation
- router.py: Fixed region scope detection to match longest region name first
- router.py: Added region abbreviations (SCID, SWID, etc.) for quick matching
- router.py: Added city name mapping (Boise -> South Western ID, etc.)
- router.py: Fixed node longname matching (case-insensitive substring)
- router.py: Added follow-up message context tracking (_user_mesh_context)
- router.py: Added more mesh keywords (noisy, traffic, packets, etc.)
- mesh_reporter.py: Added disambiguation for duplicate shortnames in region detail
- mesh_health.py: Added util_data_available flag to track packet data presence
- mesh_health.py: Passes has_packet_data through score computation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 20:56:54 +00:00
ece8bfd4d9 fix: Normalize role enums, GPS fields, and timestamps across sources
Add _normalize_node() function to mesh_sources.py that standardizes:

- Role field: Map integer enums to string names using correct Meshtastic
  protobuf values (0=CLIENT, 1=CLIENT_MUTE, 2=ROUTER, 3=ROUTER_CLIENT,
  11=ROUTER_LATE, etc.). Now detects 18 infrastructure nodes.

- GPS fields: Check latitude/longitude, then last_lat/last_long (Meshview
  scaled integers), then lat/lon. Filter out invalid 0,0 coordinates.
  Now 238 nodes with GPS (was 201).

- Timestamps: Normalize to last_heard as epoch seconds. Handle
  microseconds (last_seen_us), milliseconds, and seconds formats.
  Now 527 nodes with timestamps (was 0).

- Hardware model: Prefer string hw_model over integer hwModel.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 20:22:07 +00:00
44c74ccfd4 feat: Phase 3 - LLM mesh health integration, recommendations, and health commands
New files:
- mesh_reporter.py: MeshReporter class for prompt injection
  - build_tier1_summary(): ~500-800 token mesh health summary
  - build_region_detail(): Detailed region breakdown
  - build_node_detail(): Single node info with recommendations
  - build_recommendations(): Optimization suggestions
  - build_lora_compact(): Short format for LoRa messages
  - list_regions_compact(): Region list with scores

- commands/health.py: !health and !region commands
  - !health: Quick mesh summary (no LLM)
  - !region [name]: Region info or list all regions

Modified files:
- router.py: Mesh question detection and prompt injection
  - _is_mesh_question(): Keyword/phrase matching
  - _detect_mesh_scope(): Node/region/mesh scope detection
  - Inject Tier 1/2 data for mesh questions
  - Add mesh awareness instructions to LLM

- main.py: Create MeshReporter, pass to dispatcher/router

- commands/dispatcher.py: Register health/region commands

- mesh_health.py: Fix role type (int -> str)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 19:19:42 +00:00
89315a8008 Fix configurator: region_labels -> regions
Update main menu to use new regions attribute instead of old region_labels.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 18:56:13 +00:00
959c128c6f Add node/edge deduplication to MeshSourceManager
- Add _extract_node_num() helper to normalize node IDs across formats
- Add _normalize_edge_key() helper for undirected edge deduplication
- Update get_all_nodes() to deduplicate by nodeNum with _sources field
- Update get_all_edges() to deduplicate by sorted (from, to) tuple
- Update get_all_telemetry() to deduplicate by (nodeNum, timestamp)
- Add get_dedup_stats() method for raw vs dedup counts
- Add get_stats_by_source() method for per-source statistics
- Change _source field to _sources (list) for multi-source tracking

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 18:02:47 +00:00
06c2b9e4e2 feat: Expand region defaults to ID/UT
10 regions covering Idaho and Utah:
- Northern ID (Coeur d alene)
- Central ID (Salmon)
- South Western ID (Boise)
- South Central ID (Twin Falls)
- South Eastern ID (Idaho Falls)
- Northern UT (Ogden)
- Central UT (Salt Lake City)
- Eastern UT (Vernal)
- Western UT (Tooele)
- Southern UT (St. George)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 17:43:59 +00:00
584616c8a5 feat: Infer node region from neighbors
Nodes without GPS can be assigned to a region based on their
neighbors. Uses edge data to build neighbor graph, then assigns
unlocated nodes to the most common region among their neighbors.
Iterates until no more assignments possible.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 17:38:57 +00:00
c3f1347b4b refactor: Replace auto-clustering with fixed region anchors
- Regions are now user-defined anchor points (name + lat/lon)
- Nodes assigned to nearest region, no distance limits
- Removed auto-naming and region_labels/infra_overrides
- Added Idaho region defaults in TUI
- Simpler, deterministic, user-controlled

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 17:35:28 +00:00
fe2b77097b fix: Handle Meshview GPS and timestamp formats
- last_lat/last_long are scaled integers (divide by 1e7)
- last_seen_us is in microseconds (divide by 1e6)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 17:21:51 +00:00
fd9ba21f4a fix: Handle Meshview API wrapper format
Meshview API returns {"nodes": [...]} and {"edges": [...]} wrapper
dicts, not raw lists. Added _extract_list() helper to unwrap.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 17:13:57 +00:00
5c8aeb3974 feat: Add User-Agent header to API requests
Adds MeshAI/x.x User-Agent for good API citizenship and easier
traffic identification by Meshview/MeshMonitor operators.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 17:06:14 +00:00
a7c409e406 feat: Add Phase 2 - Geographic Hierarchy and Health Scoring
Implements mesh intelligence with geo clustering, four-pillar health scoring,
and auto-naming regions from GPS data.

New: geo.py, mesh_health.py
Modified: config.py, main.py, router.py, configurator.py, config.example.yaml

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 16:43:12 +00:00
b945558ba3 feat: Phase 1 — multi-source data aggregation from Meshview and MeshMonitor APIs
- Add MeshviewSource class for fetching nodes, edges, stats from Meshview API
- Add MeshMonitorDataSource class for fetching nodes, channels, telemetry,
  traceroutes, network stats, topology, packets, solar from MeshMonitor API
- Add MeshSourceManager for managing multiple sources with aggregation
- Add MeshSourceConfig dataclass and mesh_sources list to config
- Integrate source_manager into main.py with periodic refresh
- Add source_manager parameter to MessageRouter (for future Phase 3)
- Add Mesh Sources TUI menu with add/edit/remove/test functionality
- Update config.example.yaml with mesh_sources section

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 16:26:58 +00:00
root
584d1b199d docs: Update advBBS link to zvx-echo6/advbbs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 14:29:43 +00:00
root
0e36869a5f feat: Hybrid RAG knowledge base, sentence-aware chunking, MeshMonitor HTTP sync
Knowledge Base:
- Hybrid FTS5 + vector search using sqlite-vec and bge-small-en-v1.5
- Reciprocal Rank Fusion for result merging
- Domain-aware query construction handles typos
- Configurable weights for keyword vs semantic matching

Message Chunking:
- Sentence-aware splitting respects message boundaries
- Continuation prompts for long responses
- Natural follow-up detection (yes, ok, continue, more, etc.)
- Per-user continuation state management

MeshMonitor Integration:
- HTTP API trigger sync (replaces file-based triggers.json)
- Dynamic refresh interval
- Trigger injection into LLM prompt

Other:
- Updated system prompt for better response length control
- Simplified responder to handle message lists
- Updated README with new features and architecture diagram
- Cleaned up config.example.yaml

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-04 07:44:12 +00:00
root
e65a558c6f docs: Add MeshMonitor integration section and credit to README
- Add MeshMonitor Integration section with features and configuration
- Add credit line in Features list
- Link to github.com/Yeraze/meshmonitor

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-03 07:10:21 +00:00
root
5f66b69c9c feat: Dynamic identity system prompt from bot config
- Build system prompt dynamically using bot.name and bot.owner from config
- Reorder prompt: identity -> static prompt -> MeshMonitor (conditional) -> mesh context
- MeshMonitor description only injected when meshmonitor.enabled is true
- Update default system_prompt to static parts only (commands, architecture, rules)
- Fix meshmonitor.py to handle trigger arrays (not just strings)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-03 05:45:58 +00:00