---
title: Separating navi from recon
type: project
tags:
  - recon
aliases: []
related:
  - [[navi-lift-to-media]]
  - [[navi]]
  - [[fleet-storage-memory-upgrade]]
  - [[deployment]]
  - [[environment]]
updated: 2026-08-15
---
# Separating navi from recon

[[recon]] and [[navi]] are separate functions but they currently live in **one VM**, so neither can be relocated without the other. This documents what actually belongs to which, established 2026-08-13/14, and what a split would cost.

Platform docs: [[recon]], [[navi]].

---

## Current state

`recon-vm` (VMID 1130) on data — 4 cores, 24 GB allocated, 180 GB disk on local-lvm. It runs roughly twenty [[services]] across both platforms:

- **[[recon]]** — `recon.service`, `recon-watchdog.service`
- **[[navi]]** — `navi-admin`, `navi-config`, `navi-contacts`, `navi-geo`, `navi-landclass`, `navi-offroute`, `navi-places` (gunicorn, all bound to `127.0.0.1:84xx`)
- **geo stack** — `photon.service`, `argus-resolver.service`, PostgreSQL 16, plus Docker running Nominatim v5 and Valhalla
- **content** — `kiwix.service`
- `apache2` fronts the localhost-bound [[services]]

Calling a relocation of this VM "moving recon" badly undersells it.

---

## Ownership, measured

Three virtiofs shares are mounted from data's `/mnt/data` (the 1 TB SATA SSD at 93% full):

| Share | Size | Belongs to | Evidence |
|---|---|---|---|
| `nav` | 625 GB | **navi** | held open by `postgres`, `valhalla_`, `java` (Photon) |
| `kiwix` | 138 GB | kiwix-serve, also read by recon | `kiwix-ser` holds it; recon references `/mnt/kiwix/library` |
| `library` | 59 GB | **recon** | `/opt/recon` code references `/mnt/library/Acquired`, `_ingest`, `_acquired` |

`/mnt/nav` breakdown: `overture` 252 GB (contains PostgreSQL's `data_directory` at `/mnt/nav/overture/pgdata`), `tiles` 128 GB (Valhalla), `worldcover` 85 GB, `sources` 67 GB, `addresses` 36 GB, `nominatim-v5` 29 GB, `photon` 21 GB, `padus` 3.8 GB.

**recon's own footprint is 25 GB** — `/opt/recon`, of which `data` is 24 GB. It lives on the VM's disk, **not** on the full volume. Its [[environment]] holds only a profile setting, four Gemini keys and a PeerTube password. **recon uses no database at all** — the only `psycopg2` hits are files inside its venv. PostgreSQL belongs entirely to navi.

---

## Which one should move

**navi**, if either does.

- **Storage** — navi is 625 GB of the 821 GB on data's full SATA SSD. Moving recon frees *zero bytes* from the volume that is actually full. Moving navi takes it from 93% to roughly 21%.
- **Memory** — the 18 GB of page cache in that VM serves Nominatim, Valhalla, Photon and PostgreSQL. recon is a modest Python service. The VM uses 4.8 GB of 23 GB with the rest as working cache.
- **Speed** — navi's data sits on SATA SSD today. NVMe is materially faster for the random reads a geocoder and routing engine live on.

Counterpoint worth noting: recon holds a PeerTube password and feeds the PeerTube instance on media, so recon has a mild affinity *toward* media, not away from it.

---

## Why this is a rebuild, not a migration

**virtiofs is machine-local.** The shares are wired through raw QEMU `args:` to unix sockets on the data host, backed by three systemd units (`virtiofsd-nav/kiwix/library.service`) each running `/usr/libexec/virtiofsd --socket-path=/run/virtiofsd-<n>.sock --shared-dir=/mnt/data/<n> --cache=auto --announce-submounts`. It cannot cross hosts, and it blocks live migration outright. `vzdump` does not capture the contents either — see [[pve-guest-park-and-adopt]].

The VM config references socket *paths*, not host-specific IDs, so the same units on another host with `--shared-dir` repointed would work unchanged. Moving the whole VM is therefore tractable. Splitting the two platforms apart is not — it means standing up PostgreSQL with the overture and padus databases, the Nominatim and Valhalla containers, Photon, and the seven `navi-*` services on a new host, then removing them from the original.

Unresolved before planning: the `navi-*` services bind to `127.0.0.1` behind `apache2`, so the external entry point moves with navi. Whether recon calls navi's APIs has not been confirmed.

**Memory accounting caution:** the `virtiofsd` process for `nav` shows ~16.7 GB RSS, but `RssAnon` is only 20 MB — it is almost entirely `RssShmem`, the guest's own RAM mapped through the `memory-backend-memfd,share=on` object that virtiofs requires. Counting it on top of the VM's 24 GB is double-counting.

---

## Viewshed — planned, not built

No viewshed or line-of-sight code exists anywhere on the VM. Sizing for the planned feature, since it drives the memory budget:

Elevation data present is **SRTM 1 arc-second (~30 m)** — 48 `.hgt` tiles of 3601×3601 int16, 1.2 GB total, covering **41–50°N, 110–118°W** (roughly 620 × 400 miles, ~163,000 sq mi). Moving to 10 m 3DEP over the same footprint costs 7–22 GB depending on format; contour generation is where storage actually goes (10–60 GB of vector tiles, plus 100–150 GB of transient scratch during `gdal_contour`/`tippecanoe`, which is why cortex carries a 32 GB swapfile). 3DEP 10 m is US-only.

Memory depends entirely on algorithm:

| Approach | 250 mi radius at 10 m |
|---|---|
| Ray-cast, tile cache | ~8 GB (32-tile LRU) — algorithm itself under 200 MB |
| Full-grid raster | **~84 GB** across DEM + clutter + RSSI + working layers |

A 250-mile disc is 80,400² ≈ 6.5 billion cells; full-grid is quadratic in radius, ray-casting is linear. Tools in this space — including `xarray-spatial`, which mesh_terrain uses — implement full-grid, which is why they pull in `dask` for out-of-core work.

**Budget 32 GB** if building this. That covers full-grid at 100–150 miles at 10 m, 250 miles at 30 m, or dask-chunked 250 miles at 10 m. Uncompromised 250 miles at 10 m in memory is not reachable on current hardware.

Physical note: with 4/3-Earth refraction, `d(km) = 4.12(√h₁ + √h₂)`. A 250-mile path needs ~2,380 m at both ends — summit-to-summit only. Realistic siting at 1,500 m gives about 200 miles station-to-station.

This budget does not fit on data, which is capped at 32 GB and already runs the whole geo stack. It fits comfortably on media at 64 GB. See [[fleet-storage-memory-upgrade]].

---

## Execution plan

Decided 2026-08-15: **lift the whole VM and strip each copy**, rather than rebuilding navi's ~20 services from scratch. Step-by-step procedure with rollback points is [[navi-lift-to-media]].

Two findings that made this tractable, both verified rather than assumed:

- **The front door is a single upstream.** [[caddy]] on utility CT 101 sends all of `navi.echo6.co` to `100.64.0.24:8440` (nginx inside recon-vm, which fans out to the gunicorns, Valhalla, Nominatim and Photon). Cutover is four `reverse_proxy` lines in one Caddyfile block — no [[dns]] change, no [[authentik]] change. `apache2` on the VM is only a default vhost and fronts nothing.
- **recon has no dependency on navi.** Grepping `/opt/recon` for every navi port (84xx, 8002, 8010, 2322) and hostname returns nothing, so splitting them creates no cross-host call path.