Add navi-offroute service (extraction #8 — final) (#10)
* Add navi-offroute service (extraction #8 — the last one)
Faithful port of recon's /api/offroute (POST) + /api/mvum (GET) and the
runtime offroute modules into a new :8428 service. Closes the loop: after
this, navi-frontend talks only to navi-backend.
Ported: router.py (OffrouteRouter, EntryPointIndex, 4 route strategies,
in-Python MCP_Geometric least-cost path, Valhalla integration, per-request
osmium extract), mvum.py (MVUMReader over navi.db), cost.py, friction.py,
trails.py, and barriers.py (runtime BarrierReader/WildernessReader only).
NOT ported (per Phase A §3/§15): prototype.py (dead at runtime), barriers.py
build_*_raster (offline GDB→raster prep). DEM imported from shared/dem.py
(PR #9), not duplicated.
Behaviour-faithful changes: hardcoded paths/URLs → env vars; the
profile.offroute.* config (osm_pbf_path/postgis_dsn/densify_interval_m) →
dedicated env vars (router drops deployment_config). Both routes public (no
auth, matching recon). PADUS via libpq peer-auth DSN (dbname=padus) — NO
secret. Owns no DB.
15 hermetic tests (offroute validation + mocked-router shape + close-always;
fixture-SQLite MVUM roads/trails/fallback/null; admin auth + no-secrets +
probe shape). Full suite 119 passed / 1 skipped. Adds scikit-image + rasterio.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* navi-offroute: PR #10 review cleanups (4 faithful-port deviations)
1. trails.py — drop recon-era "Run the Phase B rasterization script"
reference from the not-found error (confusing in navi-offroute context).
2. friction.py — add FileNotFoundError-before-rasterio-open check to
match barriers/trails consistency.
3. mvum.py — remove dead try/except shapely import + warnings.warn at
2 sites (shapely is a hard pyproject dep; the fallback was unreachable).
4. router.py — declare psutil in pyproject, drop the silent fallback;
the MEMORY_LIMIT_GB safety check was silently disabled in prod.
Adds test_friction_reader_raises_file_not_found_when_missing (16 navi-offroute
tests; full suite 120 passed / 1 skipped).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: zvx-echo6 <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 23:30:43 -06:00
|
|
|
"""navi-offroute Flask application factory + gunicorn entry.
|
|
|
|
|
|
|
|
|
|
Gunicorn entry:
|
|
|
|
|
gunicorn 'services.navi_offroute.app:create_app()' --bind 127.0.0.1:8428 --workers 2
|
|
|
|
|
"""
|
2026-05-27 02:43:46 +00:00
|
|
|
import logging
|
Add navi-offroute service (extraction #8 — final) (#10)
* Add navi-offroute service (extraction #8 — the last one)
Faithful port of recon's /api/offroute (POST) + /api/mvum (GET) and the
runtime offroute modules into a new :8428 service. Closes the loop: after
this, navi-frontend talks only to navi-backend.
Ported: router.py (OffrouteRouter, EntryPointIndex, 4 route strategies,
in-Python MCP_Geometric least-cost path, Valhalla integration, per-request
osmium extract), mvum.py (MVUMReader over navi.db), cost.py, friction.py,
trails.py, and barriers.py (runtime BarrierReader/WildernessReader only).
NOT ported (per Phase A §3/§15): prototype.py (dead at runtime), barriers.py
build_*_raster (offline GDB→raster prep). DEM imported from shared/dem.py
(PR #9), not duplicated.
Behaviour-faithful changes: hardcoded paths/URLs → env vars; the
profile.offroute.* config (osm_pbf_path/postgis_dsn/densify_interval_m) →
dedicated env vars (router drops deployment_config). Both routes public (no
auth, matching recon). PADUS via libpq peer-auth DSN (dbname=padus) — NO
secret. Owns no DB.
15 hermetic tests (offroute validation + mocked-router shape + close-always;
fixture-SQLite MVUM roads/trails/fallback/null; admin auth + no-secrets +
probe shape). Full suite 119 passed / 1 skipped. Adds scikit-image + rasterio.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* navi-offroute: PR #10 review cleanups (4 faithful-port deviations)
1. trails.py — drop recon-era "Run the Phase B rasterization script"
reference from the not-found error (confusing in navi-offroute context).
2. friction.py — add FileNotFoundError-before-rasterio-open check to
match barriers/trails consistency.
3. mvum.py — remove dead try/except shapely import + warnings.warn at
2 sites (shapely is a hard pyproject dep; the fallback was unreachable).
4. router.py — declare psutil in pyproject, drop the silent fallback;
the MEMORY_LIMIT_GB safety check was silently disabled in prod.
Adds test_friction_reader_raises_file_not_found_when_missing (16 navi-offroute
tests; full suite 120 passed / 1 skipped).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: zvx-echo6 <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 23:30:43 -06:00
|
|
|
import time
|
|
|
|
|
|
|
|
|
|
from flask import Flask
|
|
|
|
|
|
|
|
|
|
from shared.git_sha import git_short_sha
|
|
|
|
|
|
|
|
|
|
from . import offroute_route, admin
|
MVUM Layer 0: spatial index foundation (#22)
* MVUM Layer 0: spatial index foundation
Additive only — no routing logic, no response-format, no Valhalla changes.
- mvum.py: add MVUMSpatialIndex. Loads mvum_roads + mvum_trails from navi.db (read-only),
decodes the pure-WKB shape blobs with shapely, builds a shapely.strtree.STRtree, and
keeps a parallel list of full feature records (all columns minus the blob, plus the
parsed geometry) with a by_id lookup. Exposes query_bbox(min_lat,min_lon,max_lat,max_lon)
and query_buffered_line(coords, tolerance_m) returning candidate records (coarse bbox +
buffer; full parallelism filter is a TODO for PR-B). Reports road_count, trail_count,
bbox, build_time_seconds, memory_estimate_mb (RSS delta).
- app.py: build the index once per process (singleton) at service init; stored on
app.config[MVUM_SPATIAL_INDEX]. Failure is logged and degrades to None.
- admin.py: GET /api/admin/mvum-spatial/info (Authentik-gated, read-only) returning the
counts/bbox/build-time/memory stats.
- tests/test_mvum_spatial.py: index loads, query_bbox returns Boise-area features,
query_buffered_line returns a feature, admin endpoint returns counts.
Diagnostic before coding (read-only): roads_with_shape=150568/null=68,
trails_with_shape=21995/null=6746 (green), shape blobs are pure WKB MultiLineString.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix lat-aware buffer + count WKB parse failures
- query_buffered_line: replace the static _DEG_PER_M with _buffer_degrees_for_meters(),
which scales longitude degrees by cos(lat) and uses the larger lat/lon equivalent;
buffer at the polyline avg latitude. Early-return [] for empty coords.
- _load_table: count WKB parse failures and logger.warning once per table when > 0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 18:53:58 -06:00
|
|
|
from .mvum import MVUMSpatialIndex
|
MVUM Layer 3a: multi-modal Auto via MVUM trailhead transitions (#26)
Auto now also considers "drive in to a trailhead, switch vehicles, continue
on foot/2w/4w" trips and picks one when it is meaningfully faster than the
single-mode winner. Implicit — no new chip; Auto just returns the fastest plan.
Backend:
- mvum_transitions.py: TrailheadIndex (STRtree over trail_entry_points), built
once per process via load_trailheads() (mirrors the MVUMSpatialIndex singleton).
query_trailheads_near_line(coords, buffer_m=2000) with a precise distance filter.
- router.py: _route_auto, after the single-mode probe and only when the winner is
ok AND total_distance_km >= MIN_HYBRID_DISTANCE_KM (8.0), tries hybrids. For each
candidate trailhead near the winning polyline (closest first, capped at 20) and
each (drive, offroad) pair in HYBRID_PAIRS, it routes both legs (annotate_mvum
off) and sums leg times with NO transition cost. A hybrid wins only if it beats
the single-mode winner by >= HYBRID_MIN_TIME_SAVINGS_MIN (15 min); trivial
offroad detours (< HYBRID_MIN_OFFROAD_KM = 0.8 km) are skipped. The winner is
combined into a new "multi" scenario: leg1 features + a kind=transition marker
+ leg2 features; summary carries total_*, per-leg legs[], summed MVUM counts;
selected_mode="hybrid". Each leg is annotated separately.
- app.py / offroute_route.py: load + inject the trailhead index singleton.
Frontend (additive — no api.js signature change):
- DirectionsPanel: per-leg breakdown row for hybrid/multi ("Drive X mi (Ymin)
-> 4W X mi (Zmin) - total Wmin", lucide Repeat between legs); existing Auto
badge still shows.
- MapView: network polylines colored by network_mode (vehicle/auto blue, 4w
orange, 2w green, foot red); transition points rendered as a circle marker with
the lucide Repeat icon + "Switch to <mode>" tooltip; bounds fit skips Points.
Tests: test_mvum_transitions.py — index load, near-line close-only query, short
trip stays single-mode, big-savings hybrid wins, trivial-detour + below-threshold
+ no-trailheads all fall back. 7 new tests; full offroute suite 71 passed.
Note: the DB column is trail_entry_points.highway_class; surfaced as record
"road_class" per the spec.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 22:49:13 -06:00
|
|
|
from .mvum_transitions import load_trailheads
|
MVUM Layer 3b: OSM parking as multi-modal Auto transition candidates (#31)
* MVUM Layer 3b: OSM parking as multi-modal Auto transition candidates
Adds OSM parking lots as a third multi-modal-Auto transition source alongside
MVUM trailheads (3a) and surface-change points (3c), so Auto can suggest
"drive to a parking lot, switch to foot/2w/4w" trips where no MVUM trailhead
exists -- BLM/state land, urban edges, anywhere OSM has parking but the USFS
trailhead layer does not. Backend-only; consumes the already-ingested
/mnt/nav/osm-parking.db read-only (no data-pipeline change).
- mvum_parking.py: OSMParkingIndex (process-wide singleton via load_parking_index)
over a shapely STRtree of parking points, mirroring MVUMSpatialIndex /
TrailheadIndex. Read-only SQLite. Drops access in (private,no,permit) at load.
query_parking_near_line(coords, buffer_m=2000) with the same coarse-bbox +
precise-distance filter as TrailheadIndex. Records carry
{lat, lon, name, road_class="parking", parking_type, access}.
Perf note: the ingest already stored representative_point() in lat/lon, so the
STRtree is built straight from those columns -- parsing the 1.6M WKB blobs at
boot would add minutes for an identical point.
- router.py: _try_hybrid_auto generalized to gather candidates from each AVAILABLE
source (trailhead index if present + surface-change always + parking index if
present) instead of hard-returning when trailhead_index is None, so parking-only
candidates still work. Combined list keeps the existing closest-first sort +
HYBRID_MAX_TRAILHEADS cap. Signature unchanged; record shape already compatible.
- app.py / offroute_route.py: load + inject the OSM parking singleton, mirroring
MVUM_SPATIAL_INDEX / MVUM_TRAILHEAD_INDEX. Failure logs a warning, degrades None.
- admin.py: GET /api/admin/osm-parking/info -> {count, build_time_seconds,
memory_estimate_mb}, mirroring /api/admin/mvum-spatial/info.
- backend/scripts/ingest_parking.py + README-osm-parking-ingest.md: the
data-pipeline ingest lifted to the repo with argparse (--geojsonseq/--db, no
/tmp) + the download/filter/export/ingest/restart refresh recipe.
Tests: test_mvum_parking.py (loads, near-line close-only, private/no/permit
filtered, null-access kept) + test_offroute.py::test_hybrid_consumes_parking_
candidates (parking-only source probed as a leg-1 destination). Full offroute
suite: 82 passed.
Real-DB sanity (not deployed): index loads 1,489,054 usable parking objects
(182,945 access-blocked dropped) in ~12 s using ~950 MB RSS per worker; a Redfish
Lake/Sawtooth corridor query returns 8 lots. The ~950 MB/worker memory cost is
notable -- flagging for review.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Numpy-pack OSMParkingIndex coords + lazy records to cut RSS (~950->~570 MB/worker)
Store parking coords as packed float64 numpy arrays (_lats/_lons) and the
attribute columns as interned lists (_names/_parking_types/_accesses), and build
candidate record dicts lazily in query_parking_near_line instead of materializing
1.5M dicts + 1.5M shapely Point objects up front. road_class is the constant
"parking" so it is not stored per row.
Measured on the real /mnt/nav/osm-parking.db (1,489,054 usable rows):
RSS/worker ~950 MB -> ~570 MB (~40%), build ~11 s. Across 2 gunicorn workers that
is ~1.9 GB -> ~1.14 GB.
NOTE: this does NOT reach the ~250 MB originally targeted. The remaining cost is
the shapely STRtree itself: it permanently retains the input geometries
(tree.geometries len == row count), so the transient `del points` does not free
them. Attribution on the real DB: columns-only 137 MB, retained Point objects
+230 MB, STRtree index +110 MB. Reaching ~250 MB would require dropping the
shapely STRtree for a coordinate-only structure (e.g. scipy cKDTree over the
lon/lat arrays), which changes the line-buffer query into a per-vertex radius
query -- a behavior change beyond this fix-up's scope. Flagged for a follow-up.
Tests unchanged except one assertion (`len(idx.records) == idx.count`); full
offroute suite 82 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 14:48:35 -06:00
|
|
|
from .mvum_parking import load_parking_index
|
MVUM Layer 0: spatial index foundation (#22)
* MVUM Layer 0: spatial index foundation
Additive only — no routing logic, no response-format, no Valhalla changes.
- mvum.py: add MVUMSpatialIndex. Loads mvum_roads + mvum_trails from navi.db (read-only),
decodes the pure-WKB shape blobs with shapely, builds a shapely.strtree.STRtree, and
keeps a parallel list of full feature records (all columns minus the blob, plus the
parsed geometry) with a by_id lookup. Exposes query_bbox(min_lat,min_lon,max_lat,max_lon)
and query_buffered_line(coords, tolerance_m) returning candidate records (coarse bbox +
buffer; full parallelism filter is a TODO for PR-B). Reports road_count, trail_count,
bbox, build_time_seconds, memory_estimate_mb (RSS delta).
- app.py: build the index once per process (singleton) at service init; stored on
app.config[MVUM_SPATIAL_INDEX]. Failure is logged and degrades to None.
- admin.py: GET /api/admin/mvum-spatial/info (Authentik-gated, read-only) returning the
counts/bbox/build-time/memory stats.
- tests/test_mvum_spatial.py: index loads, query_bbox returns Boise-area features,
query_buffered_line returns a feature, admin endpoint returns counts.
Diagnostic before coding (read-only): roads_with_shape=150568/null=68,
trails_with_shape=21995/null=6746 (green), shape blobs are pure WKB MultiLineString.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix lat-aware buffer + count WKB parse failures
- query_buffered_line: replace the static _DEG_PER_M with _buffer_degrees_for_meters(),
which scales longitude degrees by cos(lat) and uses the larger lat/lon equivalent;
buffer at the polyline avg latitude. Early-return [] for empty coords.
- _load_table: count WKB parse failures and logger.warning once per table when > 0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 18:53:58 -06:00
|
|
|
|
2026-05-27 02:43:46 +00:00
|
|
|
# Emit INFO+ so PR #34 per-stage Auto timing logs (single-mode probing took...,
|
|
|
|
|
# hybrid candidate gathering..., hybrid probing took...) reach stderr/journal;
|
|
|
|
|
# navi-offroute had no logging config, so the last-resort handler dropped INFO.
|
|
|
|
|
logging.basicConfig(level=logging.INFO)
|
|
|
|
|
|
MVUM Layer 0: spatial index foundation (#22)
* MVUM Layer 0: spatial index foundation
Additive only — no routing logic, no response-format, no Valhalla changes.
- mvum.py: add MVUMSpatialIndex. Loads mvum_roads + mvum_trails from navi.db (read-only),
decodes the pure-WKB shape blobs with shapely, builds a shapely.strtree.STRtree, and
keeps a parallel list of full feature records (all columns minus the blob, plus the
parsed geometry) with a by_id lookup. Exposes query_bbox(min_lat,min_lon,max_lat,max_lon)
and query_buffered_line(coords, tolerance_m) returning candidate records (coarse bbox +
buffer; full parallelism filter is a TODO for PR-B). Reports road_count, trail_count,
bbox, build_time_seconds, memory_estimate_mb (RSS delta).
- app.py: build the index once per process (singleton) at service init; stored on
app.config[MVUM_SPATIAL_INDEX]. Failure is logged and degrades to None.
- admin.py: GET /api/admin/mvum-spatial/info (Authentik-gated, read-only) returning the
counts/bbox/build-time/memory stats.
- tests/test_mvum_spatial.py: index loads, query_bbox returns Boise-area features,
query_buffered_line returns a feature, admin endpoint returns counts.
Diagnostic before coding (read-only): roads_with_shape=150568/null=68,
trails_with_shape=21995/null=6746 (green), shape blobs are pure WKB MultiLineString.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix lat-aware buffer + count WKB parse failures
- query_buffered_line: replace the static _DEG_PER_M with _buffer_degrees_for_meters(),
which scales longitude degrees by cos(lat) and uses the larger lat/lon equivalent;
buffer at the polyline avg latitude. Early-return [] for empty coords.
- _load_table: count WKB parse failures and logger.warning once per table when > 0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 18:53:58 -06:00
|
|
|
# Process-wide singleton: build the MVUM spatial index once per process (per gunicorn
|
|
|
|
|
# worker in prod; once across create_app() calls in tests), not once per app instance.
|
|
|
|
|
_MVUM_INDEX = None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _get_mvum_index():
|
|
|
|
|
global _MVUM_INDEX
|
|
|
|
|
if _MVUM_INDEX is None:
|
|
|
|
|
_MVUM_INDEX = MVUMSpatialIndex()
|
|
|
|
|
return _MVUM_INDEX
|
|
|
|
|
|
Add navi-offroute service (extraction #8 — final) (#10)
* Add navi-offroute service (extraction #8 — the last one)
Faithful port of recon's /api/offroute (POST) + /api/mvum (GET) and the
runtime offroute modules into a new :8428 service. Closes the loop: after
this, navi-frontend talks only to navi-backend.
Ported: router.py (OffrouteRouter, EntryPointIndex, 4 route strategies,
in-Python MCP_Geometric least-cost path, Valhalla integration, per-request
osmium extract), mvum.py (MVUMReader over navi.db), cost.py, friction.py,
trails.py, and barriers.py (runtime BarrierReader/WildernessReader only).
NOT ported (per Phase A §3/§15): prototype.py (dead at runtime), barriers.py
build_*_raster (offline GDB→raster prep). DEM imported from shared/dem.py
(PR #9), not duplicated.
Behaviour-faithful changes: hardcoded paths/URLs → env vars; the
profile.offroute.* config (osm_pbf_path/postgis_dsn/densify_interval_m) →
dedicated env vars (router drops deployment_config). Both routes public (no
auth, matching recon). PADUS via libpq peer-auth DSN (dbname=padus) — NO
secret. Owns no DB.
15 hermetic tests (offroute validation + mocked-router shape + close-always;
fixture-SQLite MVUM roads/trails/fallback/null; admin auth + no-secrets +
probe shape). Full suite 119 passed / 1 skipped. Adds scikit-image + rasterio.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* navi-offroute: PR #10 review cleanups (4 faithful-port deviations)
1. trails.py — drop recon-era "Run the Phase B rasterization script"
reference from the not-found error (confusing in navi-offroute context).
2. friction.py — add FileNotFoundError-before-rasterio-open check to
match barriers/trails consistency.
3. mvum.py — remove dead try/except shapely import + warnings.warn at
2 sites (shapely is a hard pyproject dep; the fallback was unreachable).
4. router.py — declare psutil in pyproject, drop the silent fallback;
the MEMORY_LIMIT_GB safety check was silently disabled in prod.
Adds test_friction_reader_raises_file_not_found_when_missing (16 navi-offroute
tests; full suite 120 passed / 1 skipped).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: zvx-echo6 <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 23:30:43 -06:00
|
|
|
|
|
|
|
|
|
|
|
|
|
def create_app():
|
|
|
|
|
app = Flask(__name__)
|
|
|
|
|
|
|
|
|
|
app.config['VERSION'] = git_short_sha()
|
|
|
|
|
app.config['METRICS'] = {
|
|
|
|
|
'start_time': time.time(),
|
|
|
|
|
'request_count': 0,
|
|
|
|
|
'last_error_at': None,
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
@app.before_request
|
|
|
|
|
def _count_request():
|
|
|
|
|
app.config['METRICS']['request_count'] += 1
|
|
|
|
|
|
|
|
|
|
@app.after_request
|
|
|
|
|
def _track_errors(response):
|
|
|
|
|
if response.status_code >= 500:
|
|
|
|
|
app.config['METRICS']['last_error_at'] = time.strftime(
|
|
|
|
|
'%Y-%m-%dT%H:%M:%SZ', time.gmtime()
|
|
|
|
|
)
|
|
|
|
|
return response
|
|
|
|
|
|
MVUM Layer 0: spatial index foundation (#22)
* MVUM Layer 0: spatial index foundation
Additive only — no routing logic, no response-format, no Valhalla changes.
- mvum.py: add MVUMSpatialIndex. Loads mvum_roads + mvum_trails from navi.db (read-only),
decodes the pure-WKB shape blobs with shapely, builds a shapely.strtree.STRtree, and
keeps a parallel list of full feature records (all columns minus the blob, plus the
parsed geometry) with a by_id lookup. Exposes query_bbox(min_lat,min_lon,max_lat,max_lon)
and query_buffered_line(coords, tolerance_m) returning candidate records (coarse bbox +
buffer; full parallelism filter is a TODO for PR-B). Reports road_count, trail_count,
bbox, build_time_seconds, memory_estimate_mb (RSS delta).
- app.py: build the index once per process (singleton) at service init; stored on
app.config[MVUM_SPATIAL_INDEX]. Failure is logged and degrades to None.
- admin.py: GET /api/admin/mvum-spatial/info (Authentik-gated, read-only) returning the
counts/bbox/build-time/memory stats.
- tests/test_mvum_spatial.py: index loads, query_bbox returns Boise-area features,
query_buffered_line returns a feature, admin endpoint returns counts.
Diagnostic before coding (read-only): roads_with_shape=150568/null=68,
trails_with_shape=21995/null=6746 (green), shape blobs are pure WKB MultiLineString.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix lat-aware buffer + count WKB parse failures
- query_buffered_line: replace the static _DEG_PER_M with _buffer_degrees_for_meters(),
which scales longitude degrees by cos(lat) and uses the larger lat/lon equivalent;
buffer at the polyline avg latitude. Early-return [] for empty coords.
- _load_table: count WKB parse failures and logger.warning once per table when > 0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 18:53:58 -06:00
|
|
|
# Load the MVUM spatial index once at service init (logs its own load line).
|
|
|
|
|
try:
|
|
|
|
|
app.config['MVUM_SPATIAL_INDEX'] = _get_mvum_index()
|
|
|
|
|
except Exception as e:
|
|
|
|
|
app.logger.warning("MVUM spatial index failed to load: %s", e)
|
|
|
|
|
app.config['MVUM_SPATIAL_INDEX'] = None
|
|
|
|
|
|
MVUM Layer 3a: multi-modal Auto via MVUM trailhead transitions (#26)
Auto now also considers "drive in to a trailhead, switch vehicles, continue
on foot/2w/4w" trips and picks one when it is meaningfully faster than the
single-mode winner. Implicit — no new chip; Auto just returns the fastest plan.
Backend:
- mvum_transitions.py: TrailheadIndex (STRtree over trail_entry_points), built
once per process via load_trailheads() (mirrors the MVUMSpatialIndex singleton).
query_trailheads_near_line(coords, buffer_m=2000) with a precise distance filter.
- router.py: _route_auto, after the single-mode probe and only when the winner is
ok AND total_distance_km >= MIN_HYBRID_DISTANCE_KM (8.0), tries hybrids. For each
candidate trailhead near the winning polyline (closest first, capped at 20) and
each (drive, offroad) pair in HYBRID_PAIRS, it routes both legs (annotate_mvum
off) and sums leg times with NO transition cost. A hybrid wins only if it beats
the single-mode winner by >= HYBRID_MIN_TIME_SAVINGS_MIN (15 min); trivial
offroad detours (< HYBRID_MIN_OFFROAD_KM = 0.8 km) are skipped. The winner is
combined into a new "multi" scenario: leg1 features + a kind=transition marker
+ leg2 features; summary carries total_*, per-leg legs[], summed MVUM counts;
selected_mode="hybrid". Each leg is annotated separately.
- app.py / offroute_route.py: load + inject the trailhead index singleton.
Frontend (additive — no api.js signature change):
- DirectionsPanel: per-leg breakdown row for hybrid/multi ("Drive X mi (Ymin)
-> 4W X mi (Zmin) - total Wmin", lucide Repeat between legs); existing Auto
badge still shows.
- MapView: network polylines colored by network_mode (vehicle/auto blue, 4w
orange, 2w green, foot red); transition points rendered as a circle marker with
the lucide Repeat icon + "Switch to <mode>" tooltip; bounds fit skips Points.
Tests: test_mvum_transitions.py — index load, near-line close-only query, short
trip stays single-mode, big-savings hybrid wins, trivial-detour + below-threshold
+ no-trailheads all fall back. 7 new tests; full offroute suite 71 passed.
Note: the DB column is trail_entry_points.highway_class; surfaced as record
"road_class" per the spec.
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 22:49:13 -06:00
|
|
|
# Layer 3a: trailhead transition index (process-wide singleton, logs its own line).
|
|
|
|
|
try:
|
|
|
|
|
app.config['MVUM_TRAILHEAD_INDEX'] = load_trailheads()
|
|
|
|
|
except Exception as e:
|
|
|
|
|
app.logger.warning("MVUM trailhead index failed to load: %s", e)
|
|
|
|
|
app.config['MVUM_TRAILHEAD_INDEX'] = None
|
|
|
|
|
|
MVUM Layer 3b: OSM parking as multi-modal Auto transition candidates (#31)
* MVUM Layer 3b: OSM parking as multi-modal Auto transition candidates
Adds OSM parking lots as a third multi-modal-Auto transition source alongside
MVUM trailheads (3a) and surface-change points (3c), so Auto can suggest
"drive to a parking lot, switch to foot/2w/4w" trips where no MVUM trailhead
exists -- BLM/state land, urban edges, anywhere OSM has parking but the USFS
trailhead layer does not. Backend-only; consumes the already-ingested
/mnt/nav/osm-parking.db read-only (no data-pipeline change).
- mvum_parking.py: OSMParkingIndex (process-wide singleton via load_parking_index)
over a shapely STRtree of parking points, mirroring MVUMSpatialIndex /
TrailheadIndex. Read-only SQLite. Drops access in (private,no,permit) at load.
query_parking_near_line(coords, buffer_m=2000) with the same coarse-bbox +
precise-distance filter as TrailheadIndex. Records carry
{lat, lon, name, road_class="parking", parking_type, access}.
Perf note: the ingest already stored representative_point() in lat/lon, so the
STRtree is built straight from those columns -- parsing the 1.6M WKB blobs at
boot would add minutes for an identical point.
- router.py: _try_hybrid_auto generalized to gather candidates from each AVAILABLE
source (trailhead index if present + surface-change always + parking index if
present) instead of hard-returning when trailhead_index is None, so parking-only
candidates still work. Combined list keeps the existing closest-first sort +
HYBRID_MAX_TRAILHEADS cap. Signature unchanged; record shape already compatible.
- app.py / offroute_route.py: load + inject the OSM parking singleton, mirroring
MVUM_SPATIAL_INDEX / MVUM_TRAILHEAD_INDEX. Failure logs a warning, degrades None.
- admin.py: GET /api/admin/osm-parking/info -> {count, build_time_seconds,
memory_estimate_mb}, mirroring /api/admin/mvum-spatial/info.
- backend/scripts/ingest_parking.py + README-osm-parking-ingest.md: the
data-pipeline ingest lifted to the repo with argparse (--geojsonseq/--db, no
/tmp) + the download/filter/export/ingest/restart refresh recipe.
Tests: test_mvum_parking.py (loads, near-line close-only, private/no/permit
filtered, null-access kept) + test_offroute.py::test_hybrid_consumes_parking_
candidates (parking-only source probed as a leg-1 destination). Full offroute
suite: 82 passed.
Real-DB sanity (not deployed): index loads 1,489,054 usable parking objects
(182,945 access-blocked dropped) in ~12 s using ~950 MB RSS per worker; a Redfish
Lake/Sawtooth corridor query returns 8 lots. The ~950 MB/worker memory cost is
notable -- flagging for review.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Numpy-pack OSMParkingIndex coords + lazy records to cut RSS (~950->~570 MB/worker)
Store parking coords as packed float64 numpy arrays (_lats/_lons) and the
attribute columns as interned lists (_names/_parking_types/_accesses), and build
candidate record dicts lazily in query_parking_near_line instead of materializing
1.5M dicts + 1.5M shapely Point objects up front. road_class is the constant
"parking" so it is not stored per row.
Measured on the real /mnt/nav/osm-parking.db (1,489,054 usable rows):
RSS/worker ~950 MB -> ~570 MB (~40%), build ~11 s. Across 2 gunicorn workers that
is ~1.9 GB -> ~1.14 GB.
NOTE: this does NOT reach the ~250 MB originally targeted. The remaining cost is
the shapely STRtree itself: it permanently retains the input geometries
(tree.geometries len == row count), so the transient `del points` does not free
them. Attribution on the real DB: columns-only 137 MB, retained Point objects
+230 MB, STRtree index +110 MB. Reaching ~250 MB would require dropping the
shapely STRtree for a coordinate-only structure (e.g. scipy cKDTree over the
lon/lat arrays), which changes the line-buffer query into a per-vertex radius
query -- a behavior change beyond this fix-up's scope. Flagged for a follow-up.
Tests unchanged except one assertion (`len(idx.records) == idx.count`); full
offroute suite 82 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 14:48:35 -06:00
|
|
|
# Layer 3b: OSM parking index (process-wide singleton, logs its own line).
|
|
|
|
|
try:
|
|
|
|
|
app.config['OSM_PARKING_INDEX'] = load_parking_index()
|
|
|
|
|
except Exception as e:
|
|
|
|
|
app.logger.warning("OSM parking index failed to load: %s", e)
|
|
|
|
|
app.config['OSM_PARKING_INDEX'] = None
|
|
|
|
|
|
Add navi-offroute service (extraction #8 — final) (#10)
* Add navi-offroute service (extraction #8 — the last one)
Faithful port of recon's /api/offroute (POST) + /api/mvum (GET) and the
runtime offroute modules into a new :8428 service. Closes the loop: after
this, navi-frontend talks only to navi-backend.
Ported: router.py (OffrouteRouter, EntryPointIndex, 4 route strategies,
in-Python MCP_Geometric least-cost path, Valhalla integration, per-request
osmium extract), mvum.py (MVUMReader over navi.db), cost.py, friction.py,
trails.py, and barriers.py (runtime BarrierReader/WildernessReader only).
NOT ported (per Phase A §3/§15): prototype.py (dead at runtime), barriers.py
build_*_raster (offline GDB→raster prep). DEM imported from shared/dem.py
(PR #9), not duplicated.
Behaviour-faithful changes: hardcoded paths/URLs → env vars; the
profile.offroute.* config (osm_pbf_path/postgis_dsn/densify_interval_m) →
dedicated env vars (router drops deployment_config). Both routes public (no
auth, matching recon). PADUS via libpq peer-auth DSN (dbname=padus) — NO
secret. Owns no DB.
15 hermetic tests (offroute validation + mocked-router shape + close-always;
fixture-SQLite MVUM roads/trails/fallback/null; admin auth + no-secrets +
probe shape). Full suite 119 passed / 1 skipped. Adds scikit-image + rasterio.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* navi-offroute: PR #10 review cleanups (4 faithful-port deviations)
1. trails.py — drop recon-era "Run the Phase B rasterization script"
reference from the not-found error (confusing in navi-offroute context).
2. friction.py — add FileNotFoundError-before-rasterio-open check to
match barriers/trails consistency.
3. mvum.py — remove dead try/except shapely import + warnings.warn at
2 sites (shapely is a hard pyproject dep; the fallback was unreachable).
4. router.py — declare psutil in pyproject, drop the silent fallback;
the MEMORY_LIMIT_GB safety check was silently disabled in prod.
Adds test_friction_reader_raises_file_not_found_when_missing (16 navi-offroute
tests; full suite 120 passed / 1 skipped).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: zvx-echo6 <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 23:30:43 -06:00
|
|
|
app.register_blueprint(offroute_route.bp)
|
|
|
|
|
app.register_blueprint(admin.bp)
|
|
|
|
|
return app
|