# NAV-INTEGRATION-v4.md — Echo6 Navi Module

**Status:** Active  
**Created:** 2026-04-17  
**Updated:** 2026-04-19  
**Author:** Matt + Claude  
**Repo:** forge.echo6.co/matt/refactored-recon  

---

## System Context

- **RECON** — Backend of everything. Acquires, processes, enriches, embeds, files. Manages all data including Navi datasets.
- **Aurora** — Eyes, ears, and mouth. Speaks to humans via Open WebUI or mesh. Queries tools, synthesizes answers.
- **Navi** — Navigation module. Routing, geocoding, tiles, mesh bridge, weather, trail/land overlays.
- **Kiwix** — Offline internet. ZIM files via kiwix-serve. Aurora integration via tool-callable search.
- **Library** — files.echo6.co. PDFs across 21 domains.

---

## Infrastructure

- **VM 130** — 192.168.1.130 on data node (192.168.1.240)
- **OS:** Ubuntu 24.04 LTS (migrated from LXC to VM for Docker support)
- **RAM:** 16 GB
- **vCPU:** 4
- **Boot disk:** 80-100 GB
- **Docker:** Yes
- **User:** zvx
- **Mounts:**
  - /mnt/library/ — ~67 GB library (virtiofs from data host /mnt/data/library/)
  - /mnt/nav/ — ~200 GB nav data (virtiofs from data host /mnt/data/nav/)
  - /mnt/kiwix/ — Kiwix ZIM storage (virtiofs from data host /mnt/data/kiwix/)
  - /mnt/nas/ — pi-nas NFS share (192.168.1.245)

---

## Completed Work

### Phase H1a: Valhalla Routing Engine ✅
- **Docker:** `ghcr.io/valhalla/valhalla-scripted:latest` v3.6.3
- **Port:** 8002
- **Data:** Idaho PBF, 540 tiles built
- **Validated:** Buhl → Boise (127.1 mi, 144 min, auto + pedestrian costing)
- **Branch:** feature/navi on forge.echo6.co/matt/recon

### Phase H1b: Photon Geocoding ✅
- **Service:** systemd unit, Java jar, Xmx10g
- **Port:** 2322
- **Data:** Full planet import, 281.4M documents, 85 GB index
- **Validated:** Forward + reverse geocoding, worldwide coverage

### Phase H2: Aurora Nav Tools ✅
- **Files:**
  - /opt/recon/lib/nav_tools.py — route(), reverse_geocode()
  - /opt/recon/lib/aurora_nav_tool.py — Open WebUI tool wrapper
- **Registered:** Navigation tool visible in Open WebUI
- **Tested:** 5/5 tests passing

### Phase H2b: Semantic Query Router ✅
- **Files:**
  - /opt/recon/lib/query_router.py — standalone router (38 example queries, 4 routes)
  - recon_rag_tool.py v4.2.0 on cortex — router gate integrated into inlet()
- **Routes:**
  - nav_route (0.735 confidence) → Valhalla directions, skip RAG
  - nav_reverse_geocode (0.871) → Photon reverse, skip RAG
  - direct_answer (0.877) → pass through, skip RAG
  - rag_search (0.751) → full RAG pipeline
- **Safety:** TEI down → falls to RAG. Nav fails → falls to RAG.
- **Expandable:** Adding new routes = embed 10-20 examples, compute centroid, add to dict. No retraining.

### Pi-nas Country Index ✅
- **File:** /export/data/nav/photon-country-index.txt (also as .md)
- **Coverage:** 282.5M lines indexed by country code
- **US records:** 52.8M starting at line 122,520,379
- **Use:** Enables regional Photon builds for Pi deployment

---

## Active Services on VM 130

| Service | Type | Port | Status |
|---------|------|------|--------|
| recon.service | systemd (native) | 8420 (dashboard), 8888 (files) | Running |
| Valhalla | Docker | 8002 | Running (540 tiles, Idaho) |
| Photon | systemd (Java) | 2322 | Running (281M docs, 85 GB) |

---

## Git Repos

| Repo | What | Language | Branch |
|------|------|----------|--------|
| `matt/recon` | Backend, pipeline, nav_tools, mesh_bridge, router | Python | feature/navi |
| `matt/refactored-recon` | Design docs, bibles, plans | Markdown | main |
| `matt/navi` | Web map frontend (TO CREATE) | JS/HTML/CSS | — |
| `matt/navi-mobile` | Ferrostar Android app (FUTURE) | Kotlin | — |

---

## Priority Tiers (Revised)

### HIGH — Core Proposition (DONE)
1. ✅ Aurora gives turn-by-turn directions
2. ✅ Semantic router for intelligent tool selection
3. Meshtastic waypoint delivery (BLOCKED — needs hardware decisions)

### MEDIUM — Enhanced Capabilities (IN PROGRESS)
4. Address book in RECON — pre-Photon geocoding for saved locations
5. Netsyms Address DB — 160M USPS-validated addresses, SQLite
6. Web frontend — navi.echo6.co (NEW REPO: matt/navi)
7. Ferrostar mobile app — Android turn-by-turn pointed at Valhalla
8. TomTom traffic integration — routing-level, not just overlay
9. Self-hosted weather (Open-Meteo)
10. Selectable offline region downloads for designated AO
11. Trail and land ownership overlays (USFS/BLM/PAD-US)

### LOW — Future Enrichment
12. Wilderness data layers (foraging, water, fauna)

---

## Next Phases (Ordered)

### Phase M-AB: Address Book + Netsyms Download

**Goal:** Saved locations resolve instantly without geocoding. Netsyms provides USPS-validated address precision.

**Address Book:**
- YAML or SQLite in /opt/recon/data/ or /opt/recon/config/
- Checked BEFORE Photon in nav_tools geocoding chain
- Feeds: Aurora web ("how do I get home"), mesh bridge ("nav home"), Navi frontend (starred locations)
- Structure: key, name, lat, lon, address, aliases[]
- Future expansion: contacts, callsigns, frequencies (RECON-managed rolodex)

**Netsyms:**
- Download: https://dl.netsyms.net/gis/addresses/2025/AddressDatabase2025.zip (11 GB compressed, 35 GB uncompressed)
- SHA256: 3deb85a37c6a4d027dd35fcbc1084e577b06a95be471042acc17fe21dedc3d8e
- Format: SQLite, 160M addresses (US + Canada), USPS ZIP+4 validated
- Schema: zipcode, number, street, street2, city, state, plus4, country, latitude, longitude, source
- Source data: National Address Database + OpenAddresses.io + USPS ZIP+4
- License: Public domain (facts cannot be copyrighted under US law)
- Comparison: Photon (OSM crowd-sourced) vs Netsyms (government records). Test side-by-side.
- Pi deployment: "lite" version at 6.7 GB compressed, no lat/lon, optimized for autocomplete

**Photon vs Netsyms vs Address Book — geocoding chain:**
1. Address book → exact match on saved locations (instant, zero network)
2. Netsyms SQLite → street address precision (local query, no service)
3. Photon → place names, POIs, worldwide ("Boise Airport", "Sawtooth NF")

### Phase M-WEB: Web Frontend (navi.echo6.co)

**Goal:** Google Maps-style web experience with self-hosted tiles, routing, search.

**Repo:** forge.echo6.co/matt/navi (new)
**Deploy:** nginx on VM 130, port 8440, navi.echo6.co

**Stack:**
- MapLibre GL JS — map rendering
- PMTiles — vector tiles served by nginx with Range Requests
- Valhalla API at :8002 — routing with polyline + maneuvers
- Photon API at :2322 — search/geocode
- Netsyms SQLite — address autocomplete (via small API or direct query)
- TomTom traffic overlay — raster tiles via API key (visual only for now)
- Address book locations — starred markers on map

**MVP Features:**
- Search bar (Photon forward geocode)
- Click-to-route (two points → Valhalla → draw polyline + maneuver list)
- Mode selector (auto, pedestrian, bicycle)
- GPS dot (browser Geolocation API)
- TomTom traffic tile overlay
- Responsive mobile layout

**Future Features:**
- Layer switcher (street / topo / satellite)
- Saved locations from address book
- Turn-by-turn panel with voice (Web Speech API)
- Offline PWA with Service Worker caching
- USFS trail / BLM land / PAD-US overlays
- Contour/hillshade overlay

**Tile Data (needs building):**
- Basemap PMTiles from Planetiler or `pmtiles extract` from Protomaps daily builds
- Start with Idaho regional extract, expand to CONUS
- Contours from SRTM via phyghtmap (future)

### Phase M-APP: Ferrostar Mobile App

**Goal:** Native Android turn-by-turn navigation app pointed at your Valhalla.

**Repo:** forge.echo6.co/matt/navi-mobile (future)

**Stack:**
- Ferrostar SDK (BSD license, Rust core, Kotlin/Jetpack Compose UI)
- Built-in Valhalla route provider: `WellKnownRouteProvider.Valhalla("http://192.168.1.130:8002/route/v1", "auto")`
- MapLibre Native for map rendering
- GPS snapping, off-route detection, automatic rerouting
- Voice guidance via SpokenInstructionObserver
- Your PMTiles for offline map tiles

**Effort:** Fork the Ferrostar demo app, swap three URLs (Valhalla, tile source, search). Half-day to functional, a few days to polish.

### Phase M-TRAFFIC: TomTom → Valhalla Traffic Integration

**Goal:** Valhalla routes based on real-time traffic, not just static road speeds.

**Architecture:**
1. Cron job (every 5 min while internet available) → poll TomTom Traffic Flow Segments API for AO
2. Mapping service → translate TomTom segment IDs to Valhalla edge IDs (built once via `valhalla_ways_to_edges`, stored as lookup table)
3. Traffic tile writer → pack speeds into Valhalla's binary traffic tile format → update traffic.tar
4. Valhalla hot-reloads traffic data without restart

**TomTom free tier:** 2,500 requests/day — sufficient for single AO polled every 5 min during waking hours.

**Key insight:** Traffic-aware routing is a peacetime feature. Grid-down = no TomTom feed = Valhalla routes without traffic = fine (traffic patterns irrelevant without grid).

**Effort:** 2-3 CC sessions. Requires understanding Valhalla's internal edge ID system and binary traffic tile format.

**Reference:** Christian Beiwinkel's "Ultimate Guide to Traffic in Valhalla" and his "Valhalla Orbis Tools" for TomTom integration.

### Phase H3: Meshtastic Mesh Bridge + Waypoints + Position Logging

**BLOCKED on hardware decisions:**
1. Which physical Meshtastic node is the gateway?
2. Gateway IP on LAN?
3. Private nav channel PSK

**Architecture (Option B — Aurora-mediated):**
```
Mesh DM → mesh_bridge receives text
  → Injects sender GPS + context
  → Sends to Aurora (via Ollama API or Open WebUI API)
  → Semantic router classifies intent
  → Aurora calls appropriate tool (nav, RAG, weather, etc.)
  → Response formatted + compressed
  → Waypoints + text sent back over mesh DM
```

Fast-path bypass for `n`, `next`, `cancel`, `sitrep` — skip Aurora.

**Position logging:**
- Subscribe to POSITION_APP packets
- SQLite table: node_id, lat, lon, altitude, speed, heading, timestamp
- Enables: "get me home" (earliest position), breadcrumb trails, team tracking

**Waypoint delivery:**
- Sliding window (3 at a time, 6s pacing on MEDIUM_FAST)
- Compressed names ≤30 chars ("L Main St 0.3")
- Emoji icons per maneuver type
- 24h TTL expiry
- Private channel to avoid polluting public LongFast

**Gateway config:**
- Role: CLIENT or ROUTER_CLIENT
- TCP: Enabled :4403
- Channel 0: Public LongFast
- Channel 1: Private nav, PSK-protected, MEDIUM_FAST
- GPS: Enabled

---

## Data Sources Available

### Geocoding
| Source | Records | Size | Coverage | Best For |
|--------|---------|------|----------|----------|
| Photon (running) | 281M | 85 GB | Planet | Place names, POIs, worldwide |
| Netsyms (to download) | 160M | 35 GB | US + Canada | Street addresses, USPS precision |
| OpenAddresses.io | 600M+ | Varies | Worldwide | Future international expansion |
| Address book (to build) | User-defined | <1 MB | Personal | Saved locations, zero latency |

### Trail and Land Data (Phase M future)
| Dataset | Source | License | Size as PMTiles |
|---------|--------|---------|----------------|
| USFS NFS Trails | data.fs.usda.gov | Public domain | ~150-300 MB |
| USFS MVUM Roads | data.fs.usda.gov | Public domain | ~250-450 MB |
| USFS MVUM Trails | data.fs.usda.gov | Public domain | ~50-100 MB |
| PAD-US 4.1 | usgs.gov | Public domain | ~400-800 MB |
| BLM SMA | gbp-blm-egis.hub.arcgis.com | Public domain | ~80-150 MB |
| IDFG Hunt Units | idfg.idaho.gov | Free | ~10 MB |
| IDL State Trust | idl.idaho.gov | Free | ~20 MB |
| IDPR OHV/Snow Trails | idpr-data-idaho.hub.arcgis.com | Free | ~10 MB |

**Conversion pipeline:** shapefile → ogr2ogr (EPSG:4326) → tippecanoe → PMTiles → nginx
**Keep as display-only overlays.** Do NOT merge into OSM PBF for routing.

---

## Pi 5 Deployment Target

Everything built on the homelab should have a "Pi profile" — regional extracts, configurable scope, no 16+ GB RAM dependency.

| Component | Homelab | Pi 5 (8 GB) |
|-----------|---------|-------------|
| Valhalla | CONUS (60 GB, Docker) | Regional (2-5 GB, native or Docker) |
| Geocoding | Photon planet (85 GB, JVM 10g) | Netsyms SQLite (35 GB, zero overhead) |
| Place names | Photon | Lightweight OSM place extract (~500 MB) |
| Tiles | CONUS PMTiles (15-20 GB) | Regional PMTiles (2-5 GB) |
| LLM | Qwen3 8B on cortex GPU | 1-3B model on CPU (slow but functional) |
| Embeddings | bge-m3 on cortex GPU | Dense-only, 256-dim, int8 |
| Meshtastic | TCP to gateway | USB serial to node |
| Weather | Open-Meteo (50-100 GB) | Subset or skip |
| Storage | 1 TB SSD shared | 1-2 TB NVMe |

---

## Semantic Router — Current Routes

| Route | Confidence | Action | Expandable |
|-------|-----------|--------|------------|
| nav_route | 0.735 | Valhalla directions, skip RAG | ✅ |
| nav_reverse_geocode | 0.871 | Photon reverse, skip RAG | ✅ |
| direct_answer | 0.877 | Pass through, skip RAG | ✅ |
| rag_search | 0.751 | Full RAG pipeline | ✅ |
| kiwix_search | — | PLANNED: kiwix-serve search | Add 10-20 examples |
| weather | — | PLANNED: Open-Meteo query | Add 10-20 examples |
| mesh_sitrep | — | PLANNED: node positions summary | Add 10-20 examples |

---

## Storage Budget (Shared 938 GB SSD)

| Mount | Current | Planned Additions | Total |
|-------|---------|-------------------|-------|
| /mnt/library/ | 67 GB | — | 67 GB |
| /mnt/kiwix/ | ~40 GB | +110 GB (Wikipedia + others) | ~150 GB |
| /mnt/nav/ | ~111 GB (Valhalla 2 GB + Photon 85 GB + sources + PBF) | +35 GB Netsyms, +20 GB PMTiles, +60 GB CONUS Valhalla | ~226 GB |
| Other host data | ~400 GB | — | ~400 GB |
| **Total** | | | **~843 GB** |
| **Free** | | | **~95 GB** |

**Action needed:** Trim Photon to US-only to reclaim ~70 GB, bringing free to ~165 GB. Or accept planet coverage and manage tightly.

---

## Risk Register

| Risk | Impact | Mitigation |
|------|--------|------------|
| Valhalla CONUS tile build OOMs | Expansion blocked | Build on cortex, rsync to data node |
| Photon 85 GB crowds out Kiwix ZIMs | Storage pressure | Trim to US-only (~15 GB), reclaim ~70 GB |
| TomTom free tier rate limit | Traffic routing degraded | Single AO polling only, cache aggressively |
| Meshtastic gateway TCP flaky | H3 degraded | Reconnect loop with backoff |
| Ferrostar SDK breaking changes (pre-1.0) | App maintenance | Pin SDK version, update deliberately |
| Netsyms lacks place names/POIs | "Boise Airport" fails | Keep Photon as fallback for non-address queries |
| Address book file not synced to cortex | OWUI can't use saved locations | Inline in Valve JSON or add HTTP endpoint |

---

## Estimated Effort (Remaining)

| Phase | CC Sessions | Blocked By |
|-------|------------|------------|
| M-AB: Address book + Netsyms | 1 | Nothing |
| M-WEB: Web frontend MVP | 2-3 | PMTiles built |
| M-APP: Ferrostar mobile | 1-2 | PMTiles built |
| M-TRAFFIC: TomTom → Valhalla | 2-3 | M-WEB validated |
| H3: Mesh bridge | 2-3 | Meshtastic hardware |
| Trail/land overlays | 1-2 | tippecanoe installed |
| Weather (Open-Meteo) | 1 | Nothing |
| Photon trim to US-only | 0.5 | Nothing |
| Expand Valhalla to CONUS | 0.5 | Cortex RAM for build |