diff --git a/engine/lint-report.md b/engine/lint-report.md index e3daf1c..e220c96 100644 --- a/engine/lint-report.md +++ b/engine/lint-report.md @@ -1,6 +1,6 @@ # Vault Lint Report -Generated: 2026-06-23T12:00:09Z | Docs scanned: 95 | Elapsed: 0.0s +Generated: 2026-06-23T18:00:08Z | Docs scanned: 95 | Elapsed: 0.0s ## Summary diff --git a/vault/.obsidian/workspace.json b/vault/.obsidian/workspace.json index 09aa3ff..5b3aa2b 100644 --- a/vault/.obsidian/workspace.json +++ b/vault/.obsidian/workspace.json @@ -199,6 +199,9 @@ }, "active": "17bd4a6166f789d0", "lastOpenFiles": [ + "docs/software/navi.md.tmp.5281.13900bd73182", + "docs/software/navi.md.tmp.5281.4b9a09bef001", + "projects/nominatim-v5-reimport.md.tmp.5281.7f47896f6abc", "runbooks/fleet-magicdns-resolved-migration.md.tmp.5281.0b86b5de4a11", "runbooks/fleet-magicdns-resolved-migration.md.tmp.5281.f94befc79ce2", "runbooks/fleet-magicdns-resolved-migration.md.tmp.5281.e8ec4cd814f7", @@ -207,9 +210,6 @@ "runbooks/fleet-magicdns-resolved-migration.md.tmp.5281.3a384be4f42c", "runbooks/fleet-magicdns-resolved-migration.md.tmp.5281.32d6e661d1df", "runbooks/fleet-magicdns-resolved-migration.md.tmp.5281.e72f1c289265", - "runbooks/fleet-magicdns-resolved-migration.md.tmp.5281.5b1645835a01", - "runbooks/fleet-magicdns-resolved-migration.md.tmp.5281.d0e7c45a9ac5", - "runbooks/fleet-magicdns-resolved-migration.md.tmp.5281.96dd988d42a7", "runbooks/fleet-magicdns-resolved-migration.md", "projects/fleet-platform-baseline.md", "runbooks/toc-cortex-pve9.2-update.md", diff --git a/vault/docs/software/navi.md b/vault/docs/software/navi.md index c9444fb..562ea8b 100644 --- a/vault/docs/software/navi.md +++ b/vault/docs/software/navi.md @@ -3,7 +3,7 @@ title: navi — Offline Navigation Platform type: reference tags: [recon] related: ["[[recon]]", "[[services]]", "[[environment]]"] -updated: 2026-06-18 +updated: 2026-06-23 --- # navi — Offline Navigation Platform @@ -53,12 +53,14 @@ Shared backend modules: `shared/auth.py` (Authentik header validation), `shared/ ## Geo Backends (co-resident on recon-vm) -| Service | Port | Runtime | Purpose | -|---------|------|---------|---------| -| Valhalla | :8002 | Docker | Routing engine | -| Nominatim | :8010 | Docker | Geocoder | -| Photon | :2322 | Direct | Geocoder (embedded Elasticsearch :9201) | -| PostgreSQL/PostGIS | :5432 | System | DBs: `padus` (land classification), `overture` (OSM enrichment) | +| Service | Version | Port | Runtime | Purpose | Consumed by | +|---------|---------|------|---------|---------|-------------| +| Valhalla | `ghcr.io/valhalla/valhalla-scripted:latest` | :8002 | Docker (`valhalla`) | Routing engine | navi-offroute | +| Nominatim | 5.3.2 (`mediagis/nominatim:5.3.2`) | :8010 | Docker (`nominatim-v5`) | Geocoder — western-11 US states | navi-places | +| Photon | 1.1.0 (jar) | :2322 | Direct (Java process) | Reverse geocoder (komoot dump, embedded ES :9201) | navi-geo | +| PostgreSQL/PostGIS | — | :5432 | System | DBs: `padus` (land classification), `overture` (OSM enrichment) | navi-landclass, navi-places | + +**Nominatim coverage note:** western-11 states only (AZ, CA, CO, ID, MT, NV, NM, OR, UT, WA, WY). Queries outside those states return country-level fallback only. Nominatim was upgraded from v4.5 → v5.3.2 on 2026-06-23 (fresh re-import; same coverage as before). Photon is independent of Nominatim — it runs from a komoot dump and was not changed. ## Data diff --git a/vault/projects/nominatim-v5-reimport.md b/vault/projects/nominatim-v5-reimport.md index 97bdab5..4443ced 100644 --- a/vault/projects/nominatim-v5-reimport.md +++ b/vault/projects/nominatim-v5-reimport.md @@ -1,42 +1,76 @@ --- -title: "Nominatim v5 Re-import (deferred project)" +title: "Nominatim v5 Re-import" type: project tags: [recon, storage] related: [] -status: planned -updated: 2026-06-19 +status: complete +updated: 2026-06-23 --- -# Nominatim v5 Re-import (deferred project) +# Nominatim v5 Re-import -Spun off from the [[fleet-patch-audit]] (2026-06-19). Explicitly out of scope for that patch pass — to be scheduled as its own maintenance window. +Spun off from the [[fleet-patch-audit]] (2026-06-19). Completed 2026-06-23 as a standalone maintenance window. + +## Deployed 2026-06-23 + +Nominatim was upgraded from **v4.5 → v5.3.2** on recon-vm (VM 1130). This was a full fresh re-import; no in-place upgrade path exists between 4.x and 5.x due to data model changes. + +### What shipped + +| Parameter | Value | +|-----------|-------| +| Container name | `nominatim-v5` | +| Image | `mediagis/nominatim:5.3.2` | +| Port | 8010 (mapped from container :8080) | +| Coverage | Western-11 US states (see below) | +| PG version | 16 | +| IMPORT_STYLE | full | +| FREEZE | true | +| OSM data date | 2026-06-21 | +| pgdata path | `/mnt/nav/nominatim-v5/postgres-data/pgdata` | +| DB size (post-import) | ~29 GB | +| Restart policy | `--restart unless-stopped` + systemd unit `/etc/systemd/system/nominatim.service` (targets `nominatim-v5`) | + +**Western-11 coverage:** arizona, california, colorado, idaho, montana, nevada, new-mexico, oregon, utah, washington, wyoming. Outside these states the service returns only country-level ("United States") fallback — this was also the case for v4.5; coverage did not change. + +**Build artifact** (retained on pi-nas): +- Tarball: `pi-nas:/export/data/nav/nominatim-staging/western-us-11states-v5.3.2-pgdata.tar.gz` +- SHA-256 prefix: `cc0f0497` +- `BUILD-INFO.txt` co-located with tarball +- Built on matt-desktop (~3h41m, 14 threads) from a 3.13 GB merged PBF + +### Old v4.5 cleanup (2026-06-23) + +The prior v4.5 container, its ~26 GB DB, and the `mediagis/nominatim:4.5` image were removed after cutover validation. `/mnt/nav` has ~70 GB free post-cleanup. + +### Coverage clarification + +The v4.5 deployment was also western-11 only — this was not always clearly documented. The v5 upgrade was a like-for-like data freshness + engine refresh, not a coverage expansion. + +### Photon — not coupled this upgrade + +**Photon was left at v1.1.0 (port 2322) and was not touched.** Photon is independent: it runs from a komoot dump, not built from the Nominatim DB. No version coupling applied for this data refresh. The prior doc note about mandatory Photon 1.2.0 co-upgrade was based on an assumption that Photon reads Nominatim's internal tables directly — that does not apply here. + +## Full-US expansion (future option, not done) + +Full-US coverage is the next natural scope expansion but requires significant resources that are not currently available. + +### Requirements + +| Resource | Estimate | +|----------|----------| +| Final DB size | ~120 GB+ (likely 150–250 GB — dense eastern address data) | +| Peak build disk | ~350–600 GB (scratch + output) | +| RAM during build | 48–64 GB | +| Build time | ~15–24 h | + +### Blockers + +1. **recon-vm `/mnt/nav` disk**: ~70 GB free — insufficient for a full-US build or DB. Would require either reclaiming existing storage or relocating the volume. +2. **matt-desktop build-disk capacity**: the gating number for build scratch space is unconfirmed. Must be established before committing to a full-US build. + +Until both blockers are resolved, full-US is deferred. ## Context -Nominatim on **recon-vm** (VM 1130, data node) is running **4.5.0** via Docker image `mediagis/nominatim:4.5` (~14 months old). Latest upstream is **5.3.2**. - -This is NOT a routine version bump. Nominatim v5 (released February 2025) changed the underlying data model. Moving from 4.x to 5.x requires deploying the `mediagis/nominatim:5.x` image and running a **full OpenStreetMap re-import** — there is no in-place upgrade path. - -## Coupled dependency: Photon - -**Photon must be upgraded from 1.1.0 → 1.2.0 in coordination.** Photon 1.2.0 was released specifically to read the Nominatim v5 data model. Upgrading only one side risks incompatibility; both must move together at cutover. - -## Constraints and risks - -- **Disk is the primary blocker.** The data node is at ~92% capacity (~73 GB free). A full OSM re-import is large and disk-heavy. The import must target external or provisioned storage — not the existing data pool. Disk and time budgets must be planned before starting. -- Re-imports are slow; schedule a maintenance window with adequate runway. -- The 4.5 deployment must remain live until validation is complete — do not destroy it before cutover. - -## Suggested steps (high level) - -1. Provision sufficient disk on or attached to recon-vm for the import scratch space and new data volume. -2. Stand up `nominatim:5.x` alongside the running 4.5 deployment (parallel, not replacement). -3. Run a fresh OSM import into the new deployment. -4. Upgrade Photon to 1.2.0 and point it at the new Nominatim v5 data. -5. Validate geocode and reverse-geocode parity between old and new stacks. -6. Cut over (update any consumers pointing at Nominatim/Photon endpoints). -7. Retire the 4.5 deployment and reclaim its disk. - -## Status - -Deferred. Not scheduled. Pick this up as a standalone maintenance window separate from routine fleet patching. +Nominatim v5 (released February 2025) changed the underlying data model relative to 4.x. A full OSM re-import is required; there is no in-place upgrade path.