* navi-offroute: HPA tile DB manifest pattern (H6a-pre, unblocks continental rollout)
Replaces the single-file NAVI_OFFROUTE_HPA_DB env var with a directory-based
manifest layout (NAVI_OFFROUTE_HPA_DIR + manifest.json) so one deployment can
carry multiple regional tile DBs side by side and look up which one(s) cover
a route bbox. The two-level HPA* kernel (astar_hpa_multimode) is unchanged;
router.py asks the new hpa_manifest module for matching tile DBs and forwards
the path to the existing kernel.
Manifest schema (version 1):
{"version": 1, "tile_dbs": [
{"name": "idaho", "path": "idaho.db",
"chunk_bounds": {"min_x": -8550, "max_x": -8430, "min_y": 3100, "max_y": 3260}}
]}
- path is relative to the manifest directory.
- chunk_bounds is inclusive in chunk-index space; null = covers everywhere.
- Multiple entries may overlap; v1 dispatch only handles single-region routes,
multi-region falls through to the unified kernel (UNION across DBs is a
future PR — needs astar_hpa_multimode signature change).
Backward-compat: if NAVI_OFFROUTE_HPA_DB is set and NAVI_OFFROUTE_HPA_DIR is
not, the loader synthesizes a single unbounded entry and warns (deprecation).
Current deploy keeps working byte-identically; env-file migration happens as
a separate follow-up step.
Touches:
+ hpa_manifest.py (new, 144 LOC) — schema, loaders, lookup, lazy conn cache
~ router.py (-24 +31) — _HPA_MANIFEST replaces HPA_TILE_DB; dispatch site
consults manifest.dbs_for_route_bbox; _hpa_eligible uses manifest.enabled
+ tests/test_hpa_manifest.py (new, 117 LOC) — 8 tests covering all paths
~ tests/test_hpa_runtime.py (4 lines) — monkeypatch _HPA_MANIFEST instead
No touches to hpa_build, astar, transitions, cost, mvum. No schema change.
No service restart, no env-file edit (deploy migration happens after merge).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* navi-offroute: drop unused connection cache from HPAManifest (cleanup)
The connection cache field (_conns) and get_connection() method were
shipped in PR #53 "ready for future use" but never consumed by the v1
dispatch path: astar_hpa_multimode opens its own sqlite connection per
call (astar.py:752) and changing that signature was out of scope.
Per self-review of PR #53: shipping dead code in a PR is wrong even
when the rationale is "future PRs will use it." Removing it here keeps
the manifest module focused on lookup; whoever needs the cache later
can add it alongside the use site.
- Drop _conns field, get_connection() method.
- Drop sqlite3 + Dict imports (no longer referenced).
- Drop test_get_connection_caches_per_path.
23 deletions, 2 insertions. No production behaviour change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Ubuntu <zvx@recon-vm.echo6.co>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the per-road-cell Python 8-neighbour scan with a single 3x3 binary
dilation of the off-network mask (scipy.ndimage), AND'd with the road/track
mask. `border_value=0` treats out-of-bounds neighbours as on-network, matching
the scalar version's OOB skip -- NOT np.roll, which would wrap the raster edges
and fabricate phantom neighbours. A road cell is never off-network itself, so
dilating with the centre included is equivalent to the loop's strict-neighbour
test; the surviving cell set and the 2 directed foot<->vehicle tuples per cell
are identical (only emission order differs; cap/kernel are order-independent).
Synthetic eyeball benchmark (1234x470, dense road block, worst case for the
loop's early-break): scalar 843 ms -> vector 9.3 ms; set-equal True. Targets the
~3.75s road_terminus stage of Route B's gather_transition_cells.
Adds test_road_terminus_dilation_no_wrap (np.roll regression guard); existing
test_road_terminus_transitions_pure_raster + gather/cost parity tests unchanged.
Co-authored-by: mj <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* navi-offroute: Auto classify-once / route-once (kill 4-mode contest)
PR 1 of the Auto rewrite. _route_auto no longer routes all four modes and keeps a
min-time winner; it classifies each endpoint (category map, spatial probe only as
untagged-click fallback), picks the first AUTO_MODE_PRIORITY mode in the eligible
intersection, and routes ONCE. This removes the measured ~3.03s in-town 4-mode
probe (single-mode probing line in journald) -- in-town Auto drops from ~6s toward ~1s.
Trade-off (intentional, PR1): no routing-failure fall-through and no min-time
refinement -- if the capability-picked mode cannot route, the error is returned.
The hybrid path (unchanged here, still gated on the 24km MIN_HYBRID_DISTANCE_KM)
recovers road->offroad plans. Semantic hybrid gate is PR 2.
Scope: router.py (_route_auto contest loop only) + test_offroute.py (contest tests
-> capability-pick tests + new tagged-no-spatial and untagged-spatial-once tests).
_try_hybrid_auto body, the hybrid gate, AUTO_MODE_PRIORITY/MODE_PROFILES, and all
other modules untouched. Full offroute suite: 84 passed.
Design: recon_refactor/auto-rewrite-plan.md (artifacts dir).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Auto: foot-as-last-resort fallback when picked mode fails to route
If the capability-picked mode cannot route, retry foot ONCE (foot always routes
modulo bbox limits) instead of surfacing a wall to the user. On success, ship the
foot route tagged with auto_fallback_from=<picked mode> for the UI; on foot failure,
return the original error. No fallback when the picked mode is already foot.
selected_mode_set still reflects the original capability intersection.
Tests: no_fallthrough -> falls_back_to_foot_on_error; + both-fail returns error;
+ no-fallback-when-picked-is-foot. Full offroute suite: 86 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #34 added per-stage Auto timing logs (single-mode probing took..., hybrid
candidate gathering..., hybrid probing took...) at logger.info level, but
navi-offroute has no logging config, so Python's last-resort handler dropped
everything below WARNING and the lines never reached stderr/journal.
This one-line logging.basicConfig(level=logging.INFO) in app.py opens the gate so
those lines surface in journald, letting us localize the 6-7s baseline Auto
latency on in-town routes. No other changes.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PROBLEM 1 (latency): short in-town Auto routes (Twin Falls->Filer ~7 mi) took 60+
seconds because urban OSM-parking density exploded hybrid candidate evaluation.
Fixes in router.py:
- MIN_HYBRID_DISTANCE_KM 8.0 -> 24.0 (~15 mi): in-town trips never enter hybrid eval
at all -- this alone eliminates the reported latency on Matt's routes.
- HYBRID_MAX_TRAILHEADS 20 -> 8: fewer candidates even on long trips.
- HYBRID_OVERALL_TIMEOUT_S = 6.0: a wall-clock check inside the candidate loop bails
hybrid eval past 6 s (logger.warning) and keeps the single-mode / best-so-far winner.
- HYBRID_EARLY_ABORT_MIN = 30.0: once a candidate beats the single-mode winner by 30+
min, stop probing the rest and ship it.
- Per-stage timing logs (logger.info) in _route_auto / _try_hybrid_auto:
"single-mode probing took Xs", "hybrid candidate gathering: N candidates in Xs",
"hybrid probing took Xs across N tested candidates".
PROBLEM 2 (hybrid render): investigated the missing network polyline. The stated
hypothesis (a hybrid drive leg using a non-"network" segment_type) is DISPROVEN --
_build_hybrid_response emits segment_type=="network" for BOTH the drive and offroad
legs (verified against the live response), the OFFROUTE_NETWORK_LAYER filter matches
it, MODE_COLORS is fully defined, and the store passes data.route correctly. The one
real fragility is the MapLibre color match: if network_mode is ever null the whole
layer can fail to paint (wilderness still draws via its static color -- matching the
exact symptom). Hardened it with ["to-string", ["get","network_mode"]] so a
missing/unknown mode falls through to the blue fallback and the layer always paints.
I could not reproduce the exact missing-leg render headlessly (all backend shapes +
frontend filters are correct), so a Chrome MCP repro is recommended to confirm #2 is
resolved; if a render issue remains it should be diagnosed in-browser.
Tests: hybrid synthetic-trip distances bumped 20 -> 30 km (past the new 24 km gate);
new test_hybrid_early_abort_stops_probing; surface-change integration savings lowered
into the 15-30 min band so both candidate sources are still probed (not early-aborted).
Full offroute suite: 84 passed. npm run build: clean.
PROBLEM 3 (off-road wilderness timeout) is out of scope -- separate follow-up; the
wilderness pathfinder is untouched here.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When PR #30 (wikivoyage name-based discovery) landed, every place cached before
the fix kept returning stale (no-wikivoyage) responses until a manual TRUNCATE of
place_cache. A TTL makes enrichment changes propagate automatically.
- place_cache.py: cache_get now treats a hit older than the TTL as a miss, so the
caller refetches + re-enriches and cache_put overwrites the row (no delete on
read). TTL is NAVI_PLACE_CACHE_TTL_DAYS (default 30), via _ttl_seconds(). Entries
with unknown age (cached_at 0/NULL, e.g. legacy rows) are treated as expired.
- No column migration needed: the schema already has cached_at INTEGER NOT NULL and
cache_put already writes now(). Added an idempotent guard in get_conn anyway
(PRAGMA table_info check -> ALTER TABLE ADD COLUMN cached_at INTEGER DEFAULT 0)
so a hypothetical legacy on-disk DB predating the column self-heals; on the live
DB and fresh DBs it is a no-op since CREATE TABLE already includes cached_at.
Tests (test_place.py): within-TTL hit served from cache (no refetch); past-TTL hit
refetches + refreshes cached_at; NAVI_PLACE_CACHE_TTL_DAYS=1 override expires a
2-day-old entry. Full navi-places suite: 21 passed.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Apply the proven OSMParkingIndex (PR #31) memory-pack pattern to TrailheadIndex,
which the 2026-05-26 memory audit flagged at ~300-450 MB/worker using the old
list[dict]+list[Point] storage.
- mvum_transitions.py: store coords as packed float64 numpy arrays (_lats/_lons)
and attributes as interned lists (_names/_road_classes); build candidate record
dicts lazily via _record(i) in query_trailheads_near_line instead of holding
740k dicts + 740k shapely Point objects. Points are built only to construct the
STRtree, then released. Adds a records property (lazy, for tests/introspection)
and tracks build_time_seconds + memory_estimate_mb (psutil RSS delta) like
OSMParkingIndex. Query logic (coarse STRtree bbox + precise degree-distance
check) unchanged.
- admin.py: GET /api/admin/trailhead/info -> {count, build_time_seconds,
memory_estimate_mb}, mirroring /api/admin/osm-parking/info.
Tests: existing test updated (len(records)==count; the removed _points assertion)
plus a numpy-backing test (_lats/_lons dtype float64, len==count). Full offroute
suite: 83 passed.
Real-DB sanity (not deployed): index loads 740,430 entry points in ~4.8 s using
~285 MB RSS (down from the audit's inferred ~300-450 MB; same ~40% pack ratio as
parking), query returns 317 trailheads on a Redfish Lake corridor.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* MVUM Layer 3b: OSM parking as multi-modal Auto transition candidates
Adds OSM parking lots as a third multi-modal-Auto transition source alongside
MVUM trailheads (3a) and surface-change points (3c), so Auto can suggest
"drive to a parking lot, switch to foot/2w/4w" trips where no MVUM trailhead
exists -- BLM/state land, urban edges, anywhere OSM has parking but the USFS
trailhead layer does not. Backend-only; consumes the already-ingested
/mnt/nav/osm-parking.db read-only (no data-pipeline change).
- mvum_parking.py: OSMParkingIndex (process-wide singleton via load_parking_index)
over a shapely STRtree of parking points, mirroring MVUMSpatialIndex /
TrailheadIndex. Read-only SQLite. Drops access in (private,no,permit) at load.
query_parking_near_line(coords, buffer_m=2000) with the same coarse-bbox +
precise-distance filter as TrailheadIndex. Records carry
{lat, lon, name, road_class="parking", parking_type, access}.
Perf note: the ingest already stored representative_point() in lat/lon, so the
STRtree is built straight from those columns -- parsing the 1.6M WKB blobs at
boot would add minutes for an identical point.
- router.py: _try_hybrid_auto generalized to gather candidates from each AVAILABLE
source (trailhead index if present + surface-change always + parking index if
present) instead of hard-returning when trailhead_index is None, so parking-only
candidates still work. Combined list keeps the existing closest-first sort +
HYBRID_MAX_TRAILHEADS cap. Signature unchanged; record shape already compatible.
- app.py / offroute_route.py: load + inject the OSM parking singleton, mirroring
MVUM_SPATIAL_INDEX / MVUM_TRAILHEAD_INDEX. Failure logs a warning, degrades None.
- admin.py: GET /api/admin/osm-parking/info -> {count, build_time_seconds,
memory_estimate_mb}, mirroring /api/admin/mvum-spatial/info.
- backend/scripts/ingest_parking.py + README-osm-parking-ingest.md: the
data-pipeline ingest lifted to the repo with argparse (--geojsonseq/--db, no
/tmp) + the download/filter/export/ingest/restart refresh recipe.
Tests: test_mvum_parking.py (loads, near-line close-only, private/no/permit
filtered, null-access kept) + test_offroute.py::test_hybrid_consumes_parking_
candidates (parking-only source probed as a leg-1 destination). Full offroute
suite: 82 passed.
Real-DB sanity (not deployed): index loads 1,489,054 usable parking objects
(182,945 access-blocked dropped) in ~12 s using ~950 MB RSS per worker; a Redfish
Lake/Sawtooth corridor query returns 8 lots. The ~950 MB/worker memory cost is
notable -- flagging for review.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Numpy-pack OSMParkingIndex coords + lazy records to cut RSS (~950->~570 MB/worker)
Store parking coords as packed float64 numpy arrays (_lats/_lons) and the
attribute columns as interned lists (_names/_parking_types/_accesses), and build
candidate record dicts lazily in query_parking_near_line instead of materializing
1.5M dicts + 1.5M shapely Point objects up front. road_class is the constant
"parking" so it is not stored per row.
Measured on the real /mnt/nav/osm-parking.db (1,489,054 usable rows):
RSS/worker ~950 MB -> ~570 MB (~40%), build ~11 s. Across 2 gunicorn workers that
is ~1.9 GB -> ~1.14 GB.
NOTE: this does NOT reach the ~250 MB originally targeted. The remaining cost is
the shapely STRtree itself: it permanently retains the input geometries
(tree.geometries len == row count), so the transient `del points` does not free
them. Attribution on the real DB: columns-only 137 MB, retained Point objects
+230 MB, STRtree index +110 MB. Reaching ~250 MB would require dropping the
shapely STRtree for a coordinate-only structure (e.g. scipy cKDTree over the
lon/lat arrays), which changes the line-buffer query into a per-vertex radius
query -- a behavior change beyond this fix-up's scope. Flagged for a follow-up.
Tests unchanged except one assertion (`len(idx.records) == idx.count`); full
offroute suite 82 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Places without an OSM wikivoyage tag (the entire current dataset -- 0 of placex
rows carry the tag, vs 32,862 wikipedia) never got a local Wikivoyage link even
when the article exists in the mirror by name. This implements the long-standing
discover_wikivoyage_article stub (a never-finished placeholder ported verbatim
from recon -- not disabled for any flaw) so a place name can resolve to a local
Kiwix article.
- wiki_rewrite.py: discover_wikivoyage_article(name, ...) normalizes the name to
a MediaWiki title and runs it through the existing check_kiwix_has_article
('wikivoyage', ...) path -- same catalog discovery, HEAD probe, and positive
cache as tag rewriting. Returns (kiwix_url, "local") on a HEAD 200, else
(None, None). No public fallback: without an OSM tag we can't confirm a public
Wikivoyage article exists, so a name miss yields no link rather than a guessed
(possibly dead) public URL.
- place_detail._enrich_wiki_links: after the tag-rewrite loop, when extratags has
no wikivoyage value, attempt name-based discovery on result["name"] and, on a
local hit, set extratags["wikivoyage"] + sources.wiki_rewrites["wikivoyage"] =
"local". Tag rewrite always wins when a tag is present (discovery only fills the
gap). Gated by the existing has_wiki_rewriting flag (discovery lives inside that
already-enabled, flag-gated function) -- no new flag / cross-repo profile edit;
the docstring-only has_wiki_discovery flag was never defined in any profile.
Tests (test_place.py): finds-local (HEAD 200 -> local URL + source), 404 ->
no link / no source, and runs-only-when-tag-missing (tag present -> tag rewrite
wins, discovery not consulted). Full navi-places suite: 18 passed.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Twin Falls Wikivoyage investigation surfaced two coordinated gaps that kept
OSM wikivoyage tags from rewriting to local Kiwix URLs even though the
wikivoyage ZIM is loaded and serving.
Fix 1 (pagination) -- wiki_rewrite.py: append ?count=-1 to the Kiwix OPDS catalog
fetch. kiwix-serve's /catalog/v2/entries defaults to the first 10 entries; the
library has 17, so wikivoyage (and other page-2 ZIMs) were never seen by
_discover_zims and never entered _zim_map, so their tags always fell back to public.
Fix 2 (tag passthrough) -- place_detail.py: add wikivoyage to both nominatim
extratags whitelists, mirroring the existing wikipedia/wikidata lines. The
rewriter (classify_wiki_link / build_kiwix_url / rewrite_wiki_link) and the
_enrich_wiki_links loop were already source_type-generic and covered wikivoyage;
the only missing link was that the nominatim parser dropped the wikivoyage tag
before enrichment ever saw it. No rewriter refactor was needed.
Tests (test_place.py):
- test_catalog_url_requests_full_library: the OPDS fetch URL contains count=-1.
- test_wikivoyage_tag_rewrites_to_local: a wikivoyage OSM tag for a mirrored
article rewrites to https://wiki.echo6.co/content/wikivoyage_en_all_maxi_<date>/...
with sources.wiki_rewrites.wikivoyage == "local" (Kiwix mocked).
Full navi-places suite: 15 passed.
Follow-up (separate ops step, not in this PR): prune 3 dangling library.xml
entries on the Kiwix host (wikiloc.com_eng_2026-04_18, meshtastic.org_eng_2026-04_14,
meshtastic.org_eng_2026-04_15) whose ZIM files are absent; kiwix-serve silently
skips them.
Note: Twin Falls (R/121355) itself has no OSM wikivoyage tag, so it still will not
get a local Wikivoyage link from tag rewrite -- that needs the separate name-based
discovery feature (discover_wikivoyage_article stub).
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The backend (navi-places wiki_rewrite) already turns each OSM wiki tag into a
complete URL in extratags.{wikipedia,wikidata,wikivoyage} -- a local Kiwix URL
(https://wiki.echo6.co/content/...) when the article is mirrored, otherwise the
public URL -- and records which in sources.wiki_rewrites[tag] = local|public.
PlaceCard and PlaceDetail ignored those rewritten values and rebuilt/linked from
the wrong fields, causing all three bugs. Verified against the live
/api/place/R/121355 (Twin Falls) response before and after.
- Bug 1 (Wikipedia "(local)" href was public): the link used the public
wiki-index field wiki_url with a hardcoded "(local)" badge, while the actual
local URL sat unused in extratags.wikipedia. Now links to extratags.wikipedia
and shows "(local)" only when sources.wiki_rewrites.wikipedia === "local".
- Bug 2 (Wikivoyage "(local)" href was public): same hardcoded badge on the
public wikivoyage_url. Now prefers the rewritten extratags.wikivoyage (local
when mirrored) and badges from sources.wiki_rewrites.wikivoyage; falls back to
the public wikivoyage_url with no "(local)" badge (Twin Falls has no mirror).
- Bug 3 (Wikidata URL doubled): the link prepended https://www.wikidata.org/wiki/
onto extratags.wikidata, which is already that full URL -> .../wiki/https://...
Now uses the value directly (only builds the URL for a bare Q-id).
New frontend/src/utils/wiki.js centralizes the builders (wikipediaLink,
wikivoyageLink, wikidataHref); both components import them, and the duplicated
public-URL builder (wikiUrl) is removed from each. No backend changes. Frontend
build clean; navi-places tests still 13 passed.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extract surface-category boundaries along the winning single-mode polyline as
additional multi-modal-Auto transition candidates, so Auto can suggest "pull off
where the pavement turns to dirt and switch vehicles" trips even with no MVUM
trailhead nearby. Candidates share the trailhead record shape, so _try_hybrid_auto
consumes them with no restructuring.
Backend-only:
- mvum_surface_change.py: get_surface_change_candidates(coords, valhalla_url) walks
the polyline through Valhalla trace_attributes (action=include, costing=auto,
edge.surface/road_class/use/begin_shape_index/end_shape_index). classify_surface
buckets each edge into PAVED/UNPAVED/TRACK/TRAIL; adjacent edges are grouped into
runs, runs shorter than MIN_STRETCH_M (100 m, measured by haversine along the input
coords) are collapsed to suppress noise, and each surviving category boundary emits
{lat, lon, name: "Surface change: <from>-><to>", road_class}. Capped at 10. Adds an
encode_polyline6 helper (the inverse of the router _decode_polyline method).
- router.py: _try_hybrid_auto concatenates trailheads + surface-change candidates,
then re-sorts by distance to the route and applies the existing
HYBRID_MAX_TRAILHEADS cap. Probing logic unchanged.
Verified trace_attributes on the live Valhalla before coding (returns the requested
edge fields). Two empirically-driven deviations from the spec, flagged:
1. This Valhalla normalizes OSM surface tags into its own enum (paved_smooth/paved/
paved_rough/compacted/dirt/gravel/path/impassable); classify_surface keys on that
enum AND the raw OSM names for robustness.
2. Urban alleys come back as road_class=service_other with surface=paved_smooth, so
the service_other->TRACK rule is gated on a non-paved surface to avoid classifying
paved alleys as tracks.
Tests: test_mvum_surface_change.py (6) -- classify spot-check, paved->unpaved boundary,
sub-100 m noise suppression, uniform-surface empty, encoder round-trip vs the router
decoder, and hybrid integration (both trailhead + surface candidates probed). Full
offroute suite: 77 passed.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Auto now also considers "drive in to a trailhead, switch vehicles, continue
on foot/2w/4w" trips and picks one when it is meaningfully faster than the
single-mode winner. Implicit — no new chip; Auto just returns the fastest plan.
Backend:
- mvum_transitions.py: TrailheadIndex (STRtree over trail_entry_points), built
once per process via load_trailheads() (mirrors the MVUMSpatialIndex singleton).
query_trailheads_near_line(coords, buffer_m=2000) with a precise distance filter.
- router.py: _route_auto, after the single-mode probe and only when the winner is
ok AND total_distance_km >= MIN_HYBRID_DISTANCE_KM (8.0), tries hybrids. For each
candidate trailhead near the winning polyline (closest first, capped at 20) and
each (drive, offroad) pair in HYBRID_PAIRS, it routes both legs (annotate_mvum
off) and sums leg times with NO transition cost. A hybrid wins only if it beats
the single-mode winner by >= HYBRID_MIN_TIME_SAVINGS_MIN (15 min); trivial
offroad detours (< HYBRID_MIN_OFFROAD_KM = 0.8 km) are skipped. The winner is
combined into a new "multi" scenario: leg1 features + a kind=transition marker
+ leg2 features; summary carries total_*, per-leg legs[], summed MVUM counts;
selected_mode="hybrid". Each leg is annotated separately.
- app.py / offroute_route.py: load + inject the trailhead index singleton.
Frontend (additive — no api.js signature change):
- DirectionsPanel: per-leg breakdown row for hybrid/multi ("Drive X mi (Ymin)
-> 4W X mi (Zmin) - total Wmin", lucide Repeat between legs); existing Auto
badge still shows.
- MapView: network polylines colored by network_mode (vehicle/auto blue, 4w
orange, 2w green, foot red); transition points rendered as a circle marker with
the lucide Repeat icon + "Switch to <mode>" tooltip; bounds fit skips Points.
Tests: test_mvum_transitions.py — index load, near-line close-only query, short
trip stays single-mode, big-savings hybrid wins, trivial-detour + below-threshold
+ no-trailheads all fall back. 7 new tests; full offroute suite 71 passed.
Note: the DB column is trail_entry_points.highway_class; surfaced as record
"road_class" per the spec.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operational scripts only (no service code, no tests).
- backend/scripts/mvum_backfill.py: cleaned, repo-ready version of the P2/P3 NFS-centerline
backfill. argparse --db-path (default /mnt/nav/navi.db) / --nfs-gdb (default the EDW
Trans_Trail_NFS_Publish.gdb) / --dry-run; no /tmp, no test-DB paths, no prod-refusing
guard (intended for prod, gated by --dry-run + the README snapshot step). Extracts NFS
geometry from the .gdb via ogr2ogr into a cache beside it (no GDAL Python bindings here),
matches null mvum_trails rows by TRAIL_NO+TRAIL_NAME within the forest ADMIN_ORG prefix,
merges segments, writes WKB. Idempotent (UPDATE WHERE shape IS NULL). Prints
rows_attempted/rows_updated/rows_skipped_no_match/rows_skipped_nfs_null_geom.
- backend/scripts/README-mvum-ingest.md: source URLs, initial ingest (ogr2ogr shapes),
refresh, repair (the NULL-shape gap), snapshot-first + stop-service guardrails, and the
produce-vs-consume pointer to mvum.py / mvum_annotate.py / mvum_exclude.py.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Backend-only. For strict-boundary motorized routes, turn MVUM-closed segments into
buffered Valhalla exclude_polygons so routing actively avoids them. pragmatic/emergency
keep Layer-1 annotate-only behavior; foot is never excluded.
- mvum_exclude.py (new): build_exclude_polygons(start,end,mode,spatial_index,on_date,
boundary_mode) -> GeoJSON Polygon dicts, or None when not applicable (foot / non-strict
/ no index / unmappable mode). Queries the Layer-0 index over a 5km-expanded bbox,
keeps only features closed to the mode (via mvum_annotate._status_for_feature), buffers
each ~15m (lat-corrected), emits one Polygon per part (MultiPolygon split). Caps at 500
with a warning.
- router.py: route() computes self._exclude_polygons once per call (after mode validation;
has boundary_mode), so each Auto candidate probes against its own exclusions. Both
Valhalla /route builders (_route_D_network_only and _valhalla_route) inject
exclude_polygons in array-of-rings form (outer ring per Polygon); omitted when None/empty.
- 6 tests: strict builds polygons, pragmatic/emergency/foot -> None, open not excluded,
1000 closed -> capped at 500 + warning.
No frontend changes (Layer-1 closure warning still fires for residual closures). Wilderness
pathfinder, Layer-0 index, and Layer-1 annotation untouched.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* MVUM Layer 1: per-edge annotation + accurate closure counts
Annotate each network-leg segment with its MVUM access status for the selected mode,
using the Layer-0 MVUMSpatialIndex. No routing-decision or Valhalla changes.
- mvum_annotate.py (new): annotate_network_edges(coords,(lat,lon)), mode, spatial_index,
on_date) -> [EdgeAnnotation{coord_pair_index, matched_features, mvum_status}]. Walks
consecutive pairs, queries the index (10m buffer), applies a parallelism filter (acute
angle to the edge <=45deg, both directions), resolves per-mode access via the existing
check_access/get_mode_field/symbol_to_access (worst/most-restrictive across matches).
Mode map: foot->open (skip), 2w->e_bike_class1, 4w->atv, vehicle->highclearancevehicle;
auto is already resolved to a concrete candidate upstream.
- router.py: _route_D_network_only and _build_response annotate the network leg using the
injected self.spatial_index (getattr-guarded; skipped + debug-logged if None), attach
edge_mvum to the network feature, and add summary.mvum_closed_crossings +
summary.mvum_segments_annotated.
- offroute_route.py: inject app.config[MVUM_SPATIAL_INDEX] onto the router per request.
- DirectionsPanel.jsx: warning row when mvum_closed_crossings>0.
- tests/test_mvum_annotate.py: parallel match, perpendicular reject, seasonal closure,
symbol fallback, summary count.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix seasonal date default + hoist annotation, single annotate for Auto
- Default mvum_on_date to datetime.now() at annotation time so seasonal MVUM
openings/closings actually fire in production (was effectively None -> no seasonal).
- Hoist per-edge annotation out of _route_D_network_only and _build_response into a new
central OffrouteRouter._annotate_network_segments(result, mode), invoked once at the end
of route() (annotate_mvum=True). _route_auto probes with annotate_mvum=False and
annotates only the winning candidate -> Auto runs annotation once instead of up to 4x.
Removed the inline annotation/edge_mvum/summary blocks from both scenario handlers.
Note: the central pass filters network features on properties.segment_type == "network"
(the actual tag) rather than the spec-suggested "kind", which is not a field here.
1 new test: test_route_auto_annotates_only_winner.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* MVUM Layer 0: spatial index foundation
Additive only — no routing logic, no response-format, no Valhalla changes.
- mvum.py: add MVUMSpatialIndex. Loads mvum_roads + mvum_trails from navi.db (read-only),
decodes the pure-WKB shape blobs with shapely, builds a shapely.strtree.STRtree, and
keeps a parallel list of full feature records (all columns minus the blob, plus the
parsed geometry) with a by_id lookup. Exposes query_bbox(min_lat,min_lon,max_lat,max_lon)
and query_buffered_line(coords, tolerance_m) returning candidate records (coarse bbox +
buffer; full parallelism filter is a TODO for PR-B). Reports road_count, trail_count,
bbox, build_time_seconds, memory_estimate_mb (RSS delta).
- app.py: build the index once per process (singleton) at service init; stored on
app.config[MVUM_SPATIAL_INDEX]. Failure is logged and degrades to None.
- admin.py: GET /api/admin/mvum-spatial/info (Authentik-gated, read-only) returning the
counts/bbox/build-time/memory stats.
- tests/test_mvum_spatial.py: index loads, query_bbox returns Boise-area features,
query_buffered_line returns a feature, admin endpoint returns counts.
Diagnostic before coding (read-only): roads_with_shape=150568/null=68,
trails_with_shape=21995/null=6746 (green), shape blobs are pure WKB MultiLineString.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix lat-aware buffer + count WKB parse failures
- query_buffered_line: replace the static _DEG_PER_M with _buffer_degrees_for_meters(),
which scales longitude degrees by cos(lat) and uses the larger lat/lon equivalent;
buffer at the polyline avg latitude. Early-return [] for empty coords.
- _load_table: count WKB parse failures and logger.warning once per table when > 0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_route_auto now probes every eligible candidate (eligibility filter unchanged) and
picks the one with the smallest summary.total_effort_minutes, instead of returning the
first that succeeds. Fixes the case where a wilderness start + on-road end fell through
to foot and computed the entire network leg at foot pace. selected_mode = winning
candidate; ties keep AUTO_MODE_PRIORITY order; all-fail still returns the last error.
Wilderness leg still always foot (unchanged).
Per-leg breakdown: summary now carries wilderness_minutes + network_minutes (mirrors the
existing wilderness_effort_minutes/network_duration_minutes) in _build_response (A/B/C)
and _route_D. Frontend DirectionsPanel shows a transition badge "Auto: Foot Xmin +
<mode> Ymin" when wilderness_minutes>0 and selected_mode!=foot, else the existing
"Auto chose <mode>".
Tests: add min-time pick + per-leg breakdown; update probe-all assertions (no early
return) and make the selected-mode test time-based.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Rename mode constants: mtb->2w, atv->4w
Rename the OFFROUTE travel-mode identifiers mtb->2w and atv->4w across backend and
frontend. MVUM vehicle-access classes/columns (atv, motorcycle) are a separate
vocabulary and are left untouched. UI labels (MTB/ATV) and the cost.py __main__ demo
local variables (cannot be digit-initial identifiers) are unchanged.
Backend: MODE_PROFILES, MODE_TO_COSTING, MODE_TO_VALID_HIGHWAYS, AUTO_MODE_PRIORITY,
_MODES_* sets, CATEGORY_ELIGIBLE_MODES, route()/compute_cost*/_pathfind_wilderness
Literals, VALID_MODES, and tests. offroute_route.py adds a backward-compat shim mapping
legacy mode=mtb->2w and mode=atv->4w before validation so bookmarked URLs still work.
Frontend: store.js routeMode doc, DirectionsPanel TRAVEL_MODES ids + SELECTED_MODE_LABEL
keys, Panel TRAVEL_MODES ids, ManeuverList network_mode->verb map keys, api.js jsdoc.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix user-visible labels: MTB->2W, ATV->4W + grep cleanup
Update the user-facing travel-mode labels to match the renamed ids:
- DirectionsPanel TRAVEL_MODES (2w -> "2W", 4w -> "4W") + SELECTED_MODE_LABEL values.
- Panel TRAVEL_MODES labels.
- Stale test comment: mode=mtb -> mode=2w.
Grep pass over frontend/src + backend/services/navi_offroute found no remaining
mode-identifier string literals to rename. Residual hits are all out-of-scope: MVUM
vehicle-access vocabulary (mvum.py, /api/mvum output, single-quoted test fixtures), the
cost.py __main__ demo locals, the offroute_route.py back-compat shim, and comments
referencing the historical names.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A single noisy DEM cell could fabricate a huge fake slope and make an edge
unconditionally impassable, forcing the pathfinder to route around passable terrain.
Replace the hard cliff (|grade|>max_grade -> skip) with a smooth exponential penalty:
no penalty up to max_grade, then base_time *= exp(overshoot * SLOPE_PENALTY_SCALE);
only grades whose penalty exceeds SLOPE_PENALTY_CAP (true bad data / vertical) are
dropped. Routing can now see through noisy cells while still strongly avoiding real
cliffs. Penalty only raises edge cost, so the heuristic stays admissible.
2 tests: smooth penalty traverses a >max_grade gap (finite, raised cost) yet routes
around / drops a past-cap grade; exactly-at-threshold grade incurs no penalty.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace MCP_Geometric in _pathfind_wilderness with a numba-jit anisotropic A* (new
astar.py): signed-slope speed (climbing != descending; tobler peaks at -0.05), hard
cliff, per-edge avg context multiplier, trail-takes-both via 256-entry lookup, per-edge
barriers (strict/pragmatic/emergency), multi-goal A* (first popped wins) with admissible
distance/base-speed heuristic. New compute_cost_multiplier_grid (slope-free context
multiplier) + exponential inflation (sigma=1.8; inf->HARD=50*p95 for blur, inf re-imposed).
numba>=0.59 added (numba 0.65.1).
fix: wilderness leg is always foot effort; mode parameter reserved for future flexibility.
_pathfind_wilderness keeps the mode param (threaded from _route_A/B/C) but hardcodes
cost_mode=foot for the cost grid, trail friction, speed function, base speed, and max
grade. Off-trail math for MTB/ATV/vehicle is not well-grounded and real-world wilderness
traversal is foot regardless (push the bike, walk past where the vehicle stops). User mode
still drives entry-point eligibility (query_radius highway filter) and Valhalla network
costing. Matches the original pre-#17 design: wilderness ALWAYS uses foot.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The old ST_DWithin(50km) scan returned ~226k candidate points near dense areas before
sorting (~3-10s/call). Replace with PostGIS k-NN ordering: ORDER BY geom::geography <->
point LIMIT k, which the existing GiST index on (geom::geography) walks nearest-first and
stops after K rows. SELECT still computes ST_Distance AS distance_m so callers see real
meters. radius_km is kept as a Python soft cap applied after fetch (drops rows beyond it),
preserving the caller expanded-radius fallback (now effectively a no-op).
2 tests: SQL contains <-> + LIMIT and no ST_DWithin; radius_km soft cap filters beyond-cap rows.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The three wilderness-scenario guards used table_exists() OR get_entry_point_count()==0
to check the index is non-empty — a full SELECT COUNT(*) that scanned the entire
2.94M-row table (~9-73s depending on load/cache). Add EntryPointIndex.has_entry_points()
using SELECT EXISTS (SELECT 1 ... LIMIT 1), which short-circuits at the first row, and
swap it into _route_A/_route_B/_route_C. get_entry_point_count() kept for admin-info/tests.
3 new tests: table missing -> False, empty -> False, rows -> True.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Labeled-feature setSelectedPlace: category from props.kind:kind_detail (or kind:* when
only kind is present), else null.
- Pick-from-map route input (fetchReverse .then): category from reverse-geocode
raw.osm_key:osm_value when present, else null. The .catch fallback has no raw -> null
implicitly (key omitted).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Populate place.category = osm_key:osm_value (else null) where geocoder results
become routable places, so Auto mode can take the type-hint fast path instead of
the spatial /locate probe:
- LocationInput.selectResult (direct setRouteStart/End)
- SearchBar.setSelectedPlace (preview -> PlaceCard -> startDirections) + pending addStop
- DirectionsPanel.handleDragEnd now preserves category through origin/dest reconstruction
Coord-only pins (radial menu, GPS, dropped pins) and basemap-label features carry no
osm_key:value, so category is null and Auto falls back to the spatial check. setRouteStart/
setRouteEnd store places wholesale (category preserved); startDirections already preserves it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Align spatial eligibility with Valhalla's actual /locate vocabulary:
- road grade lives in classification.classification (8-value enum incl service_other,
which was missing from PAVED_HIGHWAY_CLASSES); track/path/footway are NOT grades,
they live in classification.use.
- _locate_on_network now also returns use (defensive .get chain; None in fallback).
- Renamed TRACK/PATH_HIGHWAY_CLASSES -> TRACK_USE_VALUES/PATH_USE_VALUES; atv/mtb now
match on use, vehicle still on the paved grade.
- 4 tests: service_other->vehicle, use=track->atv/mtb/foot, use=footway->mtb/foot,
none->foot.
Live-verified: the Boise end point (43.626,-116.215, service_other) now yields
[atv,mtb,vehicle,foot] instead of [foot]; Auto intersection picks vehicle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
