The backend (navi-places wiki_rewrite) already turns each OSM wiki tag into a
complete URL in extratags.{wikipedia,wikidata,wikivoyage} -- a local Kiwix URL
(https://wiki.echo6.co/content/...) when the article is mirrored, otherwise the
public URL -- and records which in sources.wiki_rewrites[tag] = local|public.
PlaceCard and PlaceDetail ignored those rewritten values and rebuilt/linked from
the wrong fields, causing all three bugs. Verified against the live
/api/place/R/121355 (Twin Falls) response before and after.
- Bug 1 (Wikipedia "(local)" href was public): the link used the public
wiki-index field wiki_url with a hardcoded "(local)" badge, while the actual
local URL sat unused in extratags.wikipedia. Now links to extratags.wikipedia
and shows "(local)" only when sources.wiki_rewrites.wikipedia === "local".
- Bug 2 (Wikivoyage "(local)" href was public): same hardcoded badge on the
public wikivoyage_url. Now prefers the rewritten extratags.wikivoyage (local
when mirrored) and badges from sources.wiki_rewrites.wikivoyage; falls back to
the public wikivoyage_url with no "(local)" badge (Twin Falls has no mirror).
- Bug 3 (Wikidata URL doubled): the link prepended https://www.wikidata.org/wiki/
onto extratags.wikidata, which is already that full URL -> .../wiki/https://...
Now uses the value directly (only builds the URL for a bare Q-id).
New frontend/src/utils/wiki.js centralizes the builders (wikipediaLink,
wikivoyageLink, wikidataHref); both components import them, and the duplicated
public-URL builder (wikiUrl) is removed from each. No backend changes. Frontend
build clean; navi-places tests still 13 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extract surface-category boundaries along the winning single-mode polyline as
additional multi-modal-Auto transition candidates, so Auto can suggest "pull off
where the pavement turns to dirt and switch vehicles" trips even with no MVUM
trailhead nearby. Candidates share the trailhead record shape, so _try_hybrid_auto
consumes them with no restructuring.
Backend-only:
- mvum_surface_change.py: get_surface_change_candidates(coords, valhalla_url) walks
the polyline through Valhalla trace_attributes (action=include, costing=auto,
edge.surface/road_class/use/begin_shape_index/end_shape_index). classify_surface
buckets each edge into PAVED/UNPAVED/TRACK/TRAIL; adjacent edges are grouped into
runs, runs shorter than MIN_STRETCH_M (100 m, measured by haversine along the input
coords) are collapsed to suppress noise, and each surviving category boundary emits
{lat, lon, name: "Surface change: <from>-><to>", road_class}. Capped at 10. Adds an
encode_polyline6 helper (the inverse of the router _decode_polyline method).
- router.py: _try_hybrid_auto concatenates trailheads + surface-change candidates,
then re-sorts by distance to the route and applies the existing
HYBRID_MAX_TRAILHEADS cap. Probing logic unchanged.
Verified trace_attributes on the live Valhalla before coding (returns the requested
edge fields). Two empirically-driven deviations from the spec, flagged:
1. This Valhalla normalizes OSM surface tags into its own enum (paved_smooth/paved/
paved_rough/compacted/dirt/gravel/path/impassable); classify_surface keys on that
enum AND the raw OSM names for robustness.
2. Urban alleys come back as road_class=service_other with surface=paved_smooth, so
the service_other->TRACK rule is gated on a non-paved surface to avoid classifying
paved alleys as tracks.
Tests: test_mvum_surface_change.py (6) -- classify spot-check, paved->unpaved boundary,
sub-100 m noise suppression, uniform-surface empty, encoder round-trip vs the router
decoder, and hybrid integration (both trailhead + surface candidates probed). Full
offroute suite: 77 passed.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Auto now also considers "drive in to a trailhead, switch vehicles, continue
on foot/2w/4w" trips and picks one when it is meaningfully faster than the
single-mode winner. Implicit — no new chip; Auto just returns the fastest plan.
Backend:
- mvum_transitions.py: TrailheadIndex (STRtree over trail_entry_points), built
once per process via load_trailheads() (mirrors the MVUMSpatialIndex singleton).
query_trailheads_near_line(coords, buffer_m=2000) with a precise distance filter.
- router.py: _route_auto, after the single-mode probe and only when the winner is
ok AND total_distance_km >= MIN_HYBRID_DISTANCE_KM (8.0), tries hybrids. For each
candidate trailhead near the winning polyline (closest first, capped at 20) and
each (drive, offroad) pair in HYBRID_PAIRS, it routes both legs (annotate_mvum
off) and sums leg times with NO transition cost. A hybrid wins only if it beats
the single-mode winner by >= HYBRID_MIN_TIME_SAVINGS_MIN (15 min); trivial
offroad detours (< HYBRID_MIN_OFFROAD_KM = 0.8 km) are skipped. The winner is
combined into a new "multi" scenario: leg1 features + a kind=transition marker
+ leg2 features; summary carries total_*, per-leg legs[], summed MVUM counts;
selected_mode="hybrid". Each leg is annotated separately.
- app.py / offroute_route.py: load + inject the trailhead index singleton.
Frontend (additive — no api.js signature change):
- DirectionsPanel: per-leg breakdown row for hybrid/multi ("Drive X mi (Ymin)
-> 4W X mi (Zmin) - total Wmin", lucide Repeat between legs); existing Auto
badge still shows.
- MapView: network polylines colored by network_mode (vehicle/auto blue, 4w
orange, 2w green, foot red); transition points rendered as a circle marker with
the lucide Repeat icon + "Switch to <mode>" tooltip; bounds fit skips Points.
Tests: test_mvum_transitions.py — index load, near-line close-only query, short
trip stays single-mode, big-savings hybrid wins, trivial-detour + below-threshold
+ no-trailheads all fall back. 7 new tests; full offroute suite 71 passed.
Note: the DB column is trail_entry_points.highway_class; surfaced as record
"road_class" per the spec.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operational scripts only (no service code, no tests).
- backend/scripts/mvum_backfill.py: cleaned, repo-ready version of the P2/P3 NFS-centerline
backfill. argparse --db-path (default /mnt/nav/navi.db) / --nfs-gdb (default the EDW
Trans_Trail_NFS_Publish.gdb) / --dry-run; no /tmp, no test-DB paths, no prod-refusing
guard (intended for prod, gated by --dry-run + the README snapshot step). Extracts NFS
geometry from the .gdb via ogr2ogr into a cache beside it (no GDAL Python bindings here),
matches null mvum_trails rows by TRAIL_NO+TRAIL_NAME within the forest ADMIN_ORG prefix,
merges segments, writes WKB. Idempotent (UPDATE WHERE shape IS NULL). Prints
rows_attempted/rows_updated/rows_skipped_no_match/rows_skipped_nfs_null_geom.
- backend/scripts/README-mvum-ingest.md: source URLs, initial ingest (ogr2ogr shapes),
refresh, repair (the NULL-shape gap), snapshot-first + stop-service guardrails, and the
produce-vs-consume pointer to mvum.py / mvum_annotate.py / mvum_exclude.py.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Backend-only. For strict-boundary motorized routes, turn MVUM-closed segments into
buffered Valhalla exclude_polygons so routing actively avoids them. pragmatic/emergency
keep Layer-1 annotate-only behavior; foot is never excluded.
- mvum_exclude.py (new): build_exclude_polygons(start,end,mode,spatial_index,on_date,
boundary_mode) -> GeoJSON Polygon dicts, or None when not applicable (foot / non-strict
/ no index / unmappable mode). Queries the Layer-0 index over a 5km-expanded bbox,
keeps only features closed to the mode (via mvum_annotate._status_for_feature), buffers
each ~15m (lat-corrected), emits one Polygon per part (MultiPolygon split). Caps at 500
with a warning.
- router.py: route() computes self._exclude_polygons once per call (after mode validation;
has boundary_mode), so each Auto candidate probes against its own exclusions. Both
Valhalla /route builders (_route_D_network_only and _valhalla_route) inject
exclude_polygons in array-of-rings form (outer ring per Polygon); omitted when None/empty.
- 6 tests: strict builds polygons, pragmatic/emergency/foot -> None, open not excluded,
1000 closed -> capped at 500 + warning.
No frontend changes (Layer-1 closure warning still fires for residual closures). Wilderness
pathfinder, Layer-0 index, and Layer-1 annotation untouched.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* MVUM Layer 1: per-edge annotation + accurate closure counts
Annotate each network-leg segment with its MVUM access status for the selected mode,
using the Layer-0 MVUMSpatialIndex. No routing-decision or Valhalla changes.
- mvum_annotate.py (new): annotate_network_edges(coords,(lat,lon)), mode, spatial_index,
on_date) -> [EdgeAnnotation{coord_pair_index, matched_features, mvum_status}]. Walks
consecutive pairs, queries the index (10m buffer), applies a parallelism filter (acute
angle to the edge <=45deg, both directions), resolves per-mode access via the existing
check_access/get_mode_field/symbol_to_access (worst/most-restrictive across matches).
Mode map: foot->open (skip), 2w->e_bike_class1, 4w->atv, vehicle->highclearancevehicle;
auto is already resolved to a concrete candidate upstream.
- router.py: _route_D_network_only and _build_response annotate the network leg using the
injected self.spatial_index (getattr-guarded; skipped + debug-logged if None), attach
edge_mvum to the network feature, and add summary.mvum_closed_crossings +
summary.mvum_segments_annotated.
- offroute_route.py: inject app.config[MVUM_SPATIAL_INDEX] onto the router per request.
- DirectionsPanel.jsx: warning row when mvum_closed_crossings>0.
- tests/test_mvum_annotate.py: parallel match, perpendicular reject, seasonal closure,
symbol fallback, summary count.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix seasonal date default + hoist annotation, single annotate for Auto
- Default mvum_on_date to datetime.now() at annotation time so seasonal MVUM
openings/closings actually fire in production (was effectively None -> no seasonal).
- Hoist per-edge annotation out of _route_D_network_only and _build_response into a new
central OffrouteRouter._annotate_network_segments(result, mode), invoked once at the end
of route() (annotate_mvum=True). _route_auto probes with annotate_mvum=False and
annotates only the winning candidate -> Auto runs annotation once instead of up to 4x.
Removed the inline annotation/edge_mvum/summary blocks from both scenario handlers.
Note: the central pass filters network features on properties.segment_type == "network"
(the actual tag) rather than the spec-suggested "kind", which is not a field here.
1 new test: test_route_auto_annotates_only_winner.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* MVUM Layer 0: spatial index foundation
Additive only — no routing logic, no response-format, no Valhalla changes.
- mvum.py: add MVUMSpatialIndex. Loads mvum_roads + mvum_trails from navi.db (read-only),
decodes the pure-WKB shape blobs with shapely, builds a shapely.strtree.STRtree, and
keeps a parallel list of full feature records (all columns minus the blob, plus the
parsed geometry) with a by_id lookup. Exposes query_bbox(min_lat,min_lon,max_lat,max_lon)
and query_buffered_line(coords, tolerance_m) returning candidate records (coarse bbox +
buffer; full parallelism filter is a TODO for PR-B). Reports road_count, trail_count,
bbox, build_time_seconds, memory_estimate_mb (RSS delta).
- app.py: build the index once per process (singleton) at service init; stored on
app.config[MVUM_SPATIAL_INDEX]. Failure is logged and degrades to None.
- admin.py: GET /api/admin/mvum-spatial/info (Authentik-gated, read-only) returning the
counts/bbox/build-time/memory stats.
- tests/test_mvum_spatial.py: index loads, query_bbox returns Boise-area features,
query_buffered_line returns a feature, admin endpoint returns counts.
Diagnostic before coding (read-only): roads_with_shape=150568/null=68,
trails_with_shape=21995/null=6746 (green), shape blobs are pure WKB MultiLineString.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix lat-aware buffer + count WKB parse failures
- query_buffered_line: replace the static _DEG_PER_M with _buffer_degrees_for_meters(),
which scales longitude degrees by cos(lat) and uses the larger lat/lon equivalent;
buffer at the polyline avg latitude. Early-return [] for empty coords.
- _load_table: count WKB parse failures and logger.warning once per table when > 0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_route_auto now probes every eligible candidate (eligibility filter unchanged) and
picks the one with the smallest summary.total_effort_minutes, instead of returning the
first that succeeds. Fixes the case where a wilderness start + on-road end fell through
to foot and computed the entire network leg at foot pace. selected_mode = winning
candidate; ties keep AUTO_MODE_PRIORITY order; all-fail still returns the last error.
Wilderness leg still always foot (unchanged).
Per-leg breakdown: summary now carries wilderness_minutes + network_minutes (mirrors the
existing wilderness_effort_minutes/network_duration_minutes) in _build_response (A/B/C)
and _route_D. Frontend DirectionsPanel shows a transition badge "Auto: Foot Xmin +
<mode> Ymin" when wilderness_minutes>0 and selected_mode!=foot, else the existing
"Auto chose <mode>".
Tests: add min-time pick + per-leg breakdown; update probe-all assertions (no early
return) and make the selected-mode test time-based.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Rename mode constants: mtb->2w, atv->4w
Rename the OFFROUTE travel-mode identifiers mtb->2w and atv->4w across backend and
frontend. MVUM vehicle-access classes/columns (atv, motorcycle) are a separate
vocabulary and are left untouched. UI labels (MTB/ATV) and the cost.py __main__ demo
local variables (cannot be digit-initial identifiers) are unchanged.
Backend: MODE_PROFILES, MODE_TO_COSTING, MODE_TO_VALID_HIGHWAYS, AUTO_MODE_PRIORITY,
_MODES_* sets, CATEGORY_ELIGIBLE_MODES, route()/compute_cost*/_pathfind_wilderness
Literals, VALID_MODES, and tests. offroute_route.py adds a backward-compat shim mapping
legacy mode=mtb->2w and mode=atv->4w before validation so bookmarked URLs still work.
Frontend: store.js routeMode doc, DirectionsPanel TRAVEL_MODES ids + SELECTED_MODE_LABEL
keys, Panel TRAVEL_MODES ids, ManeuverList network_mode->verb map keys, api.js jsdoc.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix user-visible labels: MTB->2W, ATV->4W + grep cleanup
Update the user-facing travel-mode labels to match the renamed ids:
- DirectionsPanel TRAVEL_MODES (2w -> "2W", 4w -> "4W") + SELECTED_MODE_LABEL values.
- Panel TRAVEL_MODES labels.
- Stale test comment: mode=mtb -> mode=2w.
Grep pass over frontend/src + backend/services/navi_offroute found no remaining
mode-identifier string literals to rename. Residual hits are all out-of-scope: MVUM
vehicle-access vocabulary (mvum.py, /api/mvum output, single-quoted test fixtures), the
cost.py __main__ demo locals, the offroute_route.py back-compat shim, and comments
referencing the historical names.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A single noisy DEM cell could fabricate a huge fake slope and make an edge
unconditionally impassable, forcing the pathfinder to route around passable terrain.
Replace the hard cliff (|grade|>max_grade -> skip) with a smooth exponential penalty:
no penalty up to max_grade, then base_time *= exp(overshoot * SLOPE_PENALTY_SCALE);
only grades whose penalty exceeds SLOPE_PENALTY_CAP (true bad data / vertical) are
dropped. Routing can now see through noisy cells while still strongly avoiding real
cliffs. Penalty only raises edge cost, so the heuristic stays admissible.
2 tests: smooth penalty traverses a >max_grade gap (finite, raised cost) yet routes
around / drops a past-cap grade; exactly-at-threshold grade incurs no penalty.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace MCP_Geometric in _pathfind_wilderness with a numba-jit anisotropic A* (new
astar.py): signed-slope speed (climbing != descending; tobler peaks at -0.05), hard
cliff, per-edge avg context multiplier, trail-takes-both via 256-entry lookup, per-edge
barriers (strict/pragmatic/emergency), multi-goal A* (first popped wins) with admissible
distance/base-speed heuristic. New compute_cost_multiplier_grid (slope-free context
multiplier) + exponential inflation (sigma=1.8; inf->HARD=50*p95 for blur, inf re-imposed).
numba>=0.59 added (numba 0.65.1).
fix: wilderness leg is always foot effort; mode parameter reserved for future flexibility.
_pathfind_wilderness keeps the mode param (threaded from _route_A/B/C) but hardcodes
cost_mode=foot for the cost grid, trail friction, speed function, base speed, and max
grade. Off-trail math for MTB/ATV/vehicle is not well-grounded and real-world wilderness
traversal is foot regardless (push the bike, walk past where the vehicle stops). User mode
still drives entry-point eligibility (query_radius highway filter) and Valhalla network
costing. Matches the original pre-#17 design: wilderness ALWAYS uses foot.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The old ST_DWithin(50km) scan returned ~226k candidate points near dense areas before
sorting (~3-10s/call). Replace with PostGIS k-NN ordering: ORDER BY geom::geography <->
point LIMIT k, which the existing GiST index on (geom::geography) walks nearest-first and
stops after K rows. SELECT still computes ST_Distance AS distance_m so callers see real
meters. radius_km is kept as a Python soft cap applied after fetch (drops rows beyond it),
preserving the caller expanded-radius fallback (now effectively a no-op).
2 tests: SQL contains <-> + LIMIT and no ST_DWithin; radius_km soft cap filters beyond-cap rows.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The three wilderness-scenario guards used table_exists() OR get_entry_point_count()==0
to check the index is non-empty — a full SELECT COUNT(*) that scanned the entire
2.94M-row table (~9-73s depending on load/cache). Add EntryPointIndex.has_entry_points()
using SELECT EXISTS (SELECT 1 ... LIMIT 1), which short-circuits at the first row, and
swap it into _route_A/_route_B/_route_C. get_entry_point_count() kept for admin-info/tests.
3 new tests: table missing -> False, empty -> False, rows -> True.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Labeled-feature setSelectedPlace: category from props.kind:kind_detail (or kind:* when
only kind is present), else null.
- Pick-from-map route input (fetchReverse .then): category from reverse-geocode
raw.osm_key:osm_value when present, else null. The .catch fallback has no raw -> null
implicitly (key omitted).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Populate place.category = osm_key:osm_value (else null) where geocoder results
become routable places, so Auto mode can take the type-hint fast path instead of
the spatial /locate probe:
- LocationInput.selectResult (direct setRouteStart/End)
- SearchBar.setSelectedPlace (preview -> PlaceCard -> startDirections) + pending addStop
- DirectionsPanel.handleDragEnd now preserves category through origin/dest reconstruction
Coord-only pins (radial menu, GPS, dropped pins) and basemap-label features carry no
osm_key:value, so category is null and Auto falls back to the spatial check. setRouteStart/
setRouteEnd store places wholesale (category preserved); startDirections already preserves it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Align spatial eligibility with Valhalla's actual /locate vocabulary:
- road grade lives in classification.classification (8-value enum incl service_other,
which was missing from PAVED_HIGHWAY_CLASSES); track/path/footway are NOT grades,
they live in classification.use.
- _locate_on_network now also returns use (defensive .get chain; None in fallback).
- Renamed TRACK/PATH_HIGHWAY_CLASSES -> TRACK_USE_VALUES/PATH_USE_VALUES; atv/mtb now
match on use, vehicle still on the paved grade.
- 4 tests: service_other->vehicle, use=track->atv/mtb/foot, use=footway->mtb/foot,
none->foot.
Live-verified: the Boise end point (43.626,-116.215, service_other) now yields
[atv,mtb,vehicle,foot] instead of [foot]; Auto intersection picks vehicle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #10s _spatial_eligible_modes read edge.get("road_class"), but Valhalla /locate
returns no class without verbose, and even with verbose=true the class is NOT named
road_class — it lives at edge.classification.classification (lowercase, e.g.
"secondary"). Without it, every paved/track/path gate failed and Auto collapsed to
foot for untyped endpoints.
Fix: add verbose=true to the /locate body and read the class via a defensive
edge.get("edge",{}).get("classification",{}).get("classification") chain, kept under
the same road_class key so downstream eligibility is unchanged. Live-verified against
Valhalla: (43.621,-116.205) -> road_class=secondary, snap 5.0m, PAVED=True.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the 3-tier vehicle-only discriminator with a per-endpoint eligible-mode-set
intersection:
- CATEGORY_ELIGIBLE_MODES maps OSM key:value (and key:* wildcards) to eligible modes;
_eligible_modes_from_category resolves a hint, or None when untyped.
- Untyped endpoints fall back to _spatial_eligible_modes: parallel Valhalla /locate
(auto/pedestrian/bicycle) + 3-tier snap/road-class rules (vehicle needs paved, and
paved+flat in the 5-100m grace zone; atv paved/track; mtb paved/track/path; foot always).
- _route_auto intersects both endpoints eligible sets, probes AUTO_MODE_PRIORITY within
it, and adds selected_mode + selected_mode_set. Both untyped endpoints resolve in parallel.
- _locate_on_network now returns road_class.
- api_offroute accepts optional start_category/end_category and forwards them (auto only).
Frontend: requestOffroute takes startCategory/endCategory; computeRoute passes
routeStart/routeEnd.category; startDirections preserves place.category.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Vehicle is pure Valhalla road routing — Valhalla snaps endpoints to the nearest
road automatically, so the off-network classifier (OFF_NETWORK_THRESHOLD_M) is
irrelevant for it. Add an early branch in route() that sends vehicle straight to
_route_D_network_only, instead of bandaiding the threshold. Foot/MTB/ATV keep the
gate so users can intentionally pin backcountry points; Auto inherits the skip via
its recursive mode=vehicle probe. Threshold stays at 10.
Frontend: boundary-mode chips now show for all modes except Drive (vehicle).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
unifiedList previously gated the origin/destination rows behind routeStart/
routeEnd being set, so an unset endpoint had no input row. Always push origin
(first) and destination (last) with data:null when unset; LocationInput renders
its placeholder for value={null}, and the rows bind routeStart/routeEnd directly
(never item.data), so null is safe.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Relabel the auto chip to "Auto" (Zap icon) and the vehicle chip to "Drive";
render an "Auto chose <mode>" badge below the travel-mode row when the backend
returns selected_mode. Update routeMode/requestOffroute docs to list auto.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When mode="auto", OffrouteRouter._route_auto() probes AUTO_MODE_PRIORITY
(vehicle -> atv -> mtb -> foot) and returns the first mode whose network can
serve the route, tagging the result with selected_mode. route() already errors
when a mode cannot reach an endpoint, so the first status==ok is the most
road-capable feasible mode. Adds 4 isolation tests (route monkeypatched).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Combines zvx-echo6/navi-backend into this repo as a monorepo. Backend code now lives under backend/; frontend was retroactively relocated under frontend/ via git-filter-repo. Full commit history of both sides is preserved with original authors, dates, and messages — only file paths in commits were rewritten (so SHAs differ from the originals).
Co-authored-with: navi-backend@b5079fd
navi-offroute (:8428, extraction #8) was missing from fleet.py's SERVICES list,
so it never appeared in /api/admin/fleet. Add the one (name, port) row — the
single source of truth that build_fleet, dependency_summaries, and the
self-info fanned_services all derive from. 8427/navi-admin is the aggregator
itself (self-poll), correctly not in its own fan-out.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR-A of decouple #4-REWRITE — the LAST navi→recon coupling. navi-places now
decides "is this wiki article in the local Kiwix mirror?" in-process instead of
HTTP-calling recon's /api/wiki-rewrite. Intra-process swap, no nginx changes.
Mirrors decouple #4-READ (which moved wiki_index.db reads the same way).
- NEW services/navi_places/wiki_rewrite.py: verbatim port of recon's
lib/wiki_rewrite.py. Only adjustments: setup_logging -> stdlib logging;
KIWIX_BASE -> NAVI_KIWIX_BASE_URL env; KIWIX_PUBLIC_BASE -> NAVI_KIWIX_PUBLIC_BASE
env; cache DB -> NAVI_WIKI_CACHE_DB (default /var/lib/navi-backend/wiki_cache.db,
auto-created); + a reset() to match the place_cache/wiki_index per-worker pattern.
No logic changes — same classify, same lazy hourly catalog refresh, same HEAD
timeout, same status values (local/public/original), same cache semantics.
- place_detail.py: _enrich_wiki_links_via_http -> _enrich_wiki_links; calls
wiki_rewrite.rewrite_wiki_link(tag,value) (TUPLE) and unpacks it, replacing
the dict-returning HTTP client. Import + docstrings updated.
- app.py: wiki_rewrite.reset() per worker/test (alongside place_cache/wiki_index).
- DELETE services/navi_places/wiki_rewrite_client.py (HTTP shim dead).
- tests: the 2 wiki-rewrite tests now monkeypatch the local
wiki_rewrite.rewrite_wiki_link (tuple) instead of the deleted client.
Recon's endpoint stays live but unused until PR-B (safe co-existence).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR-A of decouple #4-READ. navi-places now reads its own wiki_index.db directly
(NAVI_WIKI_INDEX_DB) instead of HTTP-calling recon's /api/wiki-enrich — same
pattern it already uses for place_cache.db. The 2.1GB DB was copied to
/var/lib/navi-backend/wiki_index.db out-of-band (5,061,763 rows verified).
- NEW services/navi_places/wiki_index.py: verbatim port of recon's
lookup_wiki_index + _get_wiki_index_db, reading NAVI_WIKI_INDEX_DB, mirroring
place_cache.py's db_path()/lazy-conn/reset() pattern. Returns the same
{wiki_summary, wiki_population, wiki_url, wikivoyage_url} shape /api/wiki-enrich
did, so it's a drop-in for the HTTP client.
- place_detail.py: _enrich_with_wiki_via_http -> _enrich_with_wiki_index; call
wiki_index.lookup() instead of wiki_client.enrich_via_recon(); docstrings.
- app.py: wiki_index.reset() per worker/test (alongside place_cache.reset_cache()).
- admin.py: drop the recon-wiki-enrich dependency probe; add NAVI_WIKI_INDEX_DB
env + a read-only filesystem entry. (recon-wiki-rewrite probe kept — separate
decouple.)
- DELETE wiki_client.py (fully replaced).
- test_place.py: convert the wiki test from a monkeypatched HTTP client to a
hermetic tmp wiki_index.db.
Internal localhost migration — no nginx/edge involvement. recon's /api/wiki-enrich
stays live until PR-B (deploy PR-A first so nothing calls the route after removal).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR-A of the overture-import relocation. The Overture Places ETL moves from
recon (where it produced data nothing in recon consumes) to navi-backend (the
side that owns the consumer, navi-places). Additive: recon's copy stays live
until PR-B; this just establishes the navi-side copy + deps + docs.
- scripts/overture_import.py: verbatim port of recon's script (recon master
879df84). The ONLY non-verbatim change is the line-8 docstring usage hint,
swapped from `/opt/recon` + venv to the navi-backend path + .venv.
- pyproject.toml: add `duckdb>=1.5` (recon runs 1.5.2; psycopg2-binary already
present). It's the only new dep.
- scripts/README.md: document the manual-only trigger + invocation.
Source release is pinned in-code: OVERTURE_RELEASE = '2026-04-15.0'.
No tests (ETL; none on recon either). Per cleanup #29, the script has no lib/
imports — fully self-contained (stdlib + duckdb + psycopg2).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR-A of the 2-PR whoami migration. Net-new, additive endpoint in navi-admin
matching recon's existing handler shape exactly. Recon's handler stays live in
this PR; once nginx routes /api/auth/whoami to :8427 (out-of-band) and recon's
handler is removed (PR-B), navi-admin is the sole owner.
- New services/navi_admin/auth_route.py with its own blueprint (navi_admin_auth):
GET /api/auth/whoami reads X-Authentik-Username, returns {authenticated,
username}. NOT @require_auth — it's the "am I logged in?" check, must answer
the unauthenticated case (mirrors recon).
- app.py: register the new blueprint (2 lines).
- test_auth.py: header-present + header-absent cases.
Kept in its own blueprint/file so admin_route.py's "all routes @require_auth"
invariant stays true. recon and nginx untouched (additive only).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per Matt's directive that navi-* should not call any /api/* on recon.
navi-admin was the only navi service doing so (polling recon's /api/health
and surfacing it in /api/admin/recon/info + the /api/admin/fleet fan-out).
navi-admin is now the navi-only fleet view; recon has its own dashboard for
recon-pipeline health.
- admin_route.py: delete the /api/admin/recon/info handler; drop the recon
config entry + RECON_HEALTH_URL/RECON_REPO_PATH env entries from
/api/admin/navi-admin/info; refresh docstrings.
- fleet.py: remove recon constants, recon_health_url/recon_repo_path/
recon_git_sha/wrap_recon_health, the now-unused shared.git_sha import, and
the recon arms in build_fleet + dependency_summaries. /api/admin/fleet now
reports only the 6 navi-* services.
- tests: drop the 2 recon/info tests + recon scaffolding; strip recon
assertions from fleet + self-info tests. 12 relevant tests pass (10 admin
+ 2 git_sha).
shared/git_sha.py KEPT unchanged — it's a generic git_short_sha(path) helper
used by every service's create_app(), not recon-specific.
RECON_HEALTH_URL + RECON_REPO_PATH in /etc/navi-backend/navi-admin.env are now
dead — flagged for out-of-band post-merge cleanup.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add navi-offroute service (extraction #8 — the last one)
Faithful port of recon's /api/offroute (POST) + /api/mvum (GET) and the
runtime offroute modules into a new :8428 service. Closes the loop: after
this, navi-frontend talks only to navi-backend.
Ported: router.py (OffrouteRouter, EntryPointIndex, 4 route strategies,
in-Python MCP_Geometric least-cost path, Valhalla integration, per-request
osmium extract), mvum.py (MVUMReader over navi.db), cost.py, friction.py,
trails.py, and barriers.py (runtime BarrierReader/WildernessReader only).
NOT ported (per Phase A §3/§15): prototype.py (dead at runtime), barriers.py
build_*_raster (offline GDB→raster prep). DEM imported from shared/dem.py
(PR #9), not duplicated.
Behaviour-faithful changes: hardcoded paths/URLs → env vars; the
profile.offroute.* config (osm_pbf_path/postgis_dsn/densify_interval_m) →
dedicated env vars (router drops deployment_config). Both routes public (no
auth, matching recon). PADUS via libpq peer-auth DSN (dbname=padus) — NO
secret. Owns no DB.
15 hermetic tests (offroute validation + mocked-router shape + close-always;
fixture-SQLite MVUM roads/trails/fallback/null; admin auth + no-secrets +
probe shape). Full suite 119 passed / 1 skipped. Adds scikit-image + rasterio.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* navi-offroute: PR #10 review cleanups (4 faithful-port deviations)
1. trails.py — drop recon-era "Run the Phase B rasterization script"
reference from the not-found error (confusing in navi-offroute context).
2. friction.py — add FileNotFoundError-before-rasterio-open check to
match barriers/trails consistency.
3. mvum.py — remove dead try/except shapely import + warnings.warn at
2 sites (shapely is a hard pyproject dep; the fallback was unreachable).
4. router.py — declare psutil in pyproject, drop the silent fallback;
the MEMORY_LIMIT_GB safety check was silently disabled in prod.
Adds test_friction_reader_raises_file_not_found_when_missing (16 navi-offroute
tests; full suite 120 passed / 1 skipped).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: zvx-echo6 <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure refactor, no behavior change. Moves services/navi_geo/dem.py to
shared/dem.py (verbatim logic + env override; only docstring + location
changed) and re-points navi-geo's two imports (geo_route.py, admin.py) to
`from shared.dem import ...`.
Per extraction-8-phase-a.md §5/§13.1: navi-offroute (#18) needs the same
DEMReader, so a single source of truth in shared/ beats a third copy. Second
shared/ promotion after PR #7 round-2's shared/git_sha.py; navi-offroute will
`from shared.dem import DEMReader` directly.
Adds shared/tests/test_dem.py (dem_path default + NAVI_DEM_PMTILES override).
navi-geo behavior unchanged (test_reverse_bundle mocks geo_route._DEM, agnostic
to DEMReader's location). Full suite: 104 passed / 1 skipped (+2).
Co-authored-by: zvx-echo6 <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #6 round-1 fixup #3 wired netsyms.health() (COUNT + DISTINCT on
35 GB) into _netsyms_fs_entry, adding >3s latency to cold admin-info
calls. navi-admin's fleet fan-out (3s timeout) caught it after #7
deploy. Reverting to the cheap _file_entry shape; deleting health()
per the no-dead-code rule (the original fallback option).
Co-authored-by: zvx-echo6 <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add navi-admin service (extraction #7)
Net-new fleet admin aggregator on :8427 — no port from recon (recon has no
/api/admin route; Phase A §3). Three @require_auth routes:
GET /api/admin/fleet fan-out to all 6 navi-* /api/admin/<svc>/info
+ recon /api/health, merged; never 5xx
(failures land in errors[])
GET /api/admin/recon/info recon /api/health wrapped in the info shape
GET /api/admin/navi-admin/info self-describe
Fan-out forwards the caller's X-Authentik-Username so the @require_auth
upstreams accept it; per-service admin endpoints stay localhost-only (this is
the single edge-exposed admin surface). Service discovery: hardcoded list in
fleet.py (Option B). No secrets, no DB.
Deploy artifacts (NOT applied here): navi-admin.env.example, systemd unit,
nginx ^~ /api/admin snippet, and deploy/caddy notes for the @authed_api edit
(first Caddy change since #2).
12 hermetic tests (fleet happy-path, per-service timeout/500 → errors[],
auth-header forwarding, recon-down degraded-not-5xx, self-info no-secrets,
auth-required). Full monorepo suite green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR #7 review fixes
1. Symmetric degraded-entry handling in fleet.build_fleet — every
probed service now appears in `services` with a uniform degraded
dict on failure (matches recon's existing pattern), AND in errors[].
Operators see "everything I tried + which broke" consistently.
2. Catch ValueError specifically in _get_json — non-JSON 200 responses
now surface as `error: 'invalid JSON'` instead of opaque 'ValueError'.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR #7 review fixes (round 2)
1. Unified degraded shape: wrap_recon_health calls _degraded_entry on
failure — no more runtime.status vs runtime.recon_status asymmetry.
Every probed service has the same shape on failure
(runtime.status == 'unreachable'). recon-specific runtime fields
(recon_status/recon_uptime/pipeline) remain only on the success path.
2. DRY'd git short-SHA helper into shared/git_sha.py — was duplicated in
7 service app.py files + fleet.recon_git_sha. One implementation,
one place to fix when behavior changes. Adds shared/tests (testpaths
now includes "shared").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: zvx-echo6 <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add navi-geo service (extraction #6)
Faithful port of recon's geocode/reverse family to a new :8426 service:
GET /api/geocode?q=&limit=&lat=&lon=&zoom= Photon-first ranked search
GET /api/reverse?lat=&lon= reverse geocode (Photon)
GET /api/reverse/<lat>/<lon> reverse enrichment bundle (Central)
Ported modules: geocode.py (engine), netsyms.py (address SQLite), dem.py
(planet-DEM reader), address_book.py (reader copy), and the three handlers +
four bundle helpers from netsyms_api.py. All three routes public, behaviour-
identical to recon.
Behaviour-changing edges (both pre-decided in Phase A/B, called out in the PR):
- landclass: in-process call replaced with HTTP GET to navi-landclass :8424,
reading .summary (the same most-specific unit-name string). First navi→navi
edge after landclass itself.
- hardcoded paths/URLs → env vars (PHOTON_URL, NAVI_NETSYMS_DB,
NAVI_TIMEZONE_DB, NAVI_DEM_PMTILES, NAVI_ADDRESS_BOOK_YAML,
NAVI_LANDCLASS_URL); rerank trace log opt-in (NAVI_GEO_RERANK_TRACE_LOG,
default off — recon always wrote /tmp).
No secrets in this service: PADUS_DB_* disappears because landclass is HTTP-
delegated (Phase A §10). Address book uses Option B (shared-file read), the
same pattern navi-contacts already uses.
Bundle 9-key contract preserved exactly (name/city/county/state/country/
postal_code/timezone/landclass/elevation_m), same null-on-component-failure
semantics, same in-memory TTLCache(10_000, 86_400) per worker.
Tests: 28 passing, 1 skipped (real timezone DB, off-box). Ported the 9 recon
reverse-bundle tests + added the HTTP-landclass coupling tests + hermetic
geocode reranker/intent-classifier tests (recon's geocode_test.py was a live
smoke test). Adds usaddress/rapidfuzz/cachetools/shapely/numpy/Pillow/pmtiles
to deps.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR #6 review fixes
1. Rename geocode._setup_trace_logger → setup_trace_logger (public hook)
2. Hoist `import requests as http_requests` to module level in geo_route.py
3. Wire netsyms.health() into admin.py (enriches the netsyms filesystem entry
with row_count/file_size_bytes/indexed_countries; no shared-builder change)
4. Fix misleading LANDCLASS_TIMEOUT_S comment (recon had no timeout)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: zvx-echo6 <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New services/navi_places/ on :8425 — the heaviest extraction. Ports recon's
/api/place family with the two wiki dependencies decoupled to HTTP.
Routes (public, mirroring recon):
GET /api/place/<osm_type>/<int:osm_id> (Nominatim -> Overpass fallback + enrich)
GET /api/place/wikidata/<wikidata_id> (Wikidata entity)
-> 200 / 400 / 404 / 502, same response shapes as recon.
Enrichment chain (recon order): Overture (PostGIS) -> Google Places -> wiki
rewrite -> wiki index. The two wiki paths are now HTTP to recon (the 2.1 GB
wiki_index.db and Kiwix/wiki_cache stay in recon — see [[reference-echo6-edge-topology]]):
- wiki_client.enrich_via_recon -> recon /api/wiki-enrich (PR #8) [has_kiwix_wiki]
- wiki_rewrite_client.rewrite_via_recon -> recon /api/wiki-rewrite (PR #9) [has_wiki_rewriting]
(per-tag loop over the <=4 wiki extratags, mirroring recon's _enrich_wiki_links)
Both clients degrade gracefully (None / status 'original') on error/timeout.
Data ownership (see [[feedback-navi-backend-data-ownership]]):
- place_cache.db migrates to /var/lib/navi-backend/place_cache.db (env
NAVI_PLACE_CACHE_DB). place_cache.py auto-creates the FULL schema on first
open — place_cache (incl. the google_place_id/google_data/google_fetched_at
columns recon added by migration) + google_api_calls — so a fresh DB serves
both cache_put and the Google daily-cap/cache. WAL, shared module conn.
- overture stays in external PG (OVERTURE_DB_* env), verbatim port of recon's
pool (1,3) + _pool_failed latch, with reset_pool()+probe_db() added.
- wiki_index.db / Kiwix stay in recon, reached via the two HTTP endpoints.
Modules: place_cache.py, overture.py (verbatim+probe), google_places.py
(daily cap via env GOOGLE_PLACES_DAILY_CAP; DB via shared place_cache conn),
wiki_client.py + wiki_rewrite_client.py (HTTP, RECON_BASE_URL default
http://127.0.0.1:8420), osm_categories.py (vendored for humanize_category),
place_detail.py (orchestrator), config.py (feature flags from the vendored
profile via NAVI_PROFILES_DIR), place_route.py, admin.py, app.py.
Feature gates read from the vendored profile (config.py), matching recon:
has_overture_enrichment / has_google_places_enrichment / has_kiwix_wiki /
has_wiki_rewriting — flag off => that enricher is skipped entirely.
admin.py (§4.5): 2 secrets masked (OVERTURE_DB_PASSWORD, GOOGLE_PLACES_API_KEY);
3 dependency probes — overture-postgis (SELECT 1), recon-wiki-enrich and
recon-wiki-rewrite (GET with no params, expect HTTP 400 = route alive).
Deploy: systemd unit (:8425) + nginx snippet (^~ /api/place, no trailing slash,
no proxy_cache; public, no Caddy edit — TIER 2 already through nginx since #2).
Tests (13; recon had zero for this module): validation (400), cache hit (no
upstream), nominatim hit, nominatim-miss->overpass fallback, both-fail 502,
not-found 404, wikidata happy + invalid, overture gated-off (no PG call),
wiki-rewrite-via-http local hit + original pass-through, wiki-enrich-via-http
field merge. Full suite 59. See ../recon_refactor/extraction-5-phase-a.md,
-wiki-enrich-investigation.md, -wiki-rewrite-investigation.md, and PRs #8/#9.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
