navi/backend/services/navi_offroute/app.py

81 lines
2.6 KiB
Python
Raw Normal View History

Add navi-offroute service (extraction #8 — final) (#10) * Add navi-offroute service (extraction #8 — the last one) Faithful port of recon's /api/offroute (POST) + /api/mvum (GET) and the runtime offroute modules into a new :8428 service. Closes the loop: after this, navi-frontend talks only to navi-backend. Ported: router.py (OffrouteRouter, EntryPointIndex, 4 route strategies, in-Python MCP_Geometric least-cost path, Valhalla integration, per-request osmium extract), mvum.py (MVUMReader over navi.db), cost.py, friction.py, trails.py, and barriers.py (runtime BarrierReader/WildernessReader only). NOT ported (per Phase A §3/§15): prototype.py (dead at runtime), barriers.py build_*_raster (offline GDB→raster prep). DEM imported from shared/dem.py (PR #9), not duplicated. Behaviour-faithful changes: hardcoded paths/URLs → env vars; the profile.offroute.* config (osm_pbf_path/postgis_dsn/densify_interval_m) → dedicated env vars (router drops deployment_config). Both routes public (no auth, matching recon). PADUS via libpq peer-auth DSN (dbname=padus) — NO secret. Owns no DB. 15 hermetic tests (offroute validation + mocked-router shape + close-always; fixture-SQLite MVUM roads/trails/fallback/null; admin auth + no-secrets + probe shape). Full suite 119 passed / 1 skipped. Adds scikit-image + rasterio. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * navi-offroute: PR #10 review cleanups (4 faithful-port deviations) 1. trails.py — drop recon-era "Run the Phase B rasterization script" reference from the not-found error (confusing in navi-offroute context). 2. friction.py — add FileNotFoundError-before-rasterio-open check to match barriers/trails consistency. 3. mvum.py — remove dead try/except shapely import + warnings.warn at 2 sites (shapely is a hard pyproject dep; the fallback was unreachable). 4. router.py — declare psutil in pyproject, drop the silent fallback; the MEMORY_LIMIT_GB safety check was silently disabled in prod. Adds test_friction_reader_raises_file_not_found_when_missing (16 navi-offroute tests; full suite 120 passed / 1 skipped). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: zvx-echo6 <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 23:30:43 -06:00
"""navi-offroute Flask application factory + gunicorn entry.
Gunicorn entry:
gunicorn 'services.navi_offroute.app:create_app()' --bind 127.0.0.1:8428 --workers 2
"""
import logging
Add navi-offroute service (extraction #8 — final) (#10) * Add navi-offroute service (extraction #8 — the last one) Faithful port of recon's /api/offroute (POST) + /api/mvum (GET) and the runtime offroute modules into a new :8428 service. Closes the loop: after this, navi-frontend talks only to navi-backend. Ported: router.py (OffrouteRouter, EntryPointIndex, 4 route strategies, in-Python MCP_Geometric least-cost path, Valhalla integration, per-request osmium extract), mvum.py (MVUMReader over navi.db), cost.py, friction.py, trails.py, and barriers.py (runtime BarrierReader/WildernessReader only). NOT ported (per Phase A §3/§15): prototype.py (dead at runtime), barriers.py build_*_raster (offline GDB→raster prep). DEM imported from shared/dem.py (PR #9), not duplicated. Behaviour-faithful changes: hardcoded paths/URLs → env vars; the profile.offroute.* config (osm_pbf_path/postgis_dsn/densify_interval_m) → dedicated env vars (router drops deployment_config). Both routes public (no auth, matching recon). PADUS via libpq peer-auth DSN (dbname=padus) — NO secret. Owns no DB. 15 hermetic tests (offroute validation + mocked-router shape + close-always; fixture-SQLite MVUM roads/trails/fallback/null; admin auth + no-secrets + probe shape). Full suite 119 passed / 1 skipped. Adds scikit-image + rasterio. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * navi-offroute: PR #10 review cleanups (4 faithful-port deviations) 1. trails.py — drop recon-era "Run the Phase B rasterization script" reference from the not-found error (confusing in navi-offroute context). 2. friction.py — add FileNotFoundError-before-rasterio-open check to match barriers/trails consistency. 3. mvum.py — remove dead try/except shapely import + warnings.warn at 2 sites (shapely is a hard pyproject dep; the fallback was unreachable). 4. router.py — declare psutil in pyproject, drop the silent fallback; the MEMORY_LIMIT_GB safety check was silently disabled in prod. Adds test_friction_reader_raises_file_not_found_when_missing (16 navi-offroute tests; full suite 120 passed / 1 skipped). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: zvx-echo6 <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 23:30:43 -06:00
import time
from flask import Flask
from shared.git_sha import git_short_sha
from . import offroute_route, admin
MVUM Layer 0: spatial index foundation (#22) * MVUM Layer 0: spatial index foundation Additive only — no routing logic, no response-format, no Valhalla changes. - mvum.py: add MVUMSpatialIndex. Loads mvum_roads + mvum_trails from navi.db (read-only), decodes the pure-WKB shape blobs with shapely, builds a shapely.strtree.STRtree, and keeps a parallel list of full feature records (all columns minus the blob, plus the parsed geometry) with a by_id lookup. Exposes query_bbox(min_lat,min_lon,max_lat,max_lon) and query_buffered_line(coords, tolerance_m) returning candidate records (coarse bbox + buffer; full parallelism filter is a TODO for PR-B). Reports road_count, trail_count, bbox, build_time_seconds, memory_estimate_mb (RSS delta). - app.py: build the index once per process (singleton) at service init; stored on app.config[MVUM_SPATIAL_INDEX]. Failure is logged and degrades to None. - admin.py: GET /api/admin/mvum-spatial/info (Authentik-gated, read-only) returning the counts/bbox/build-time/memory stats. - tests/test_mvum_spatial.py: index loads, query_bbox returns Boise-area features, query_buffered_line returns a feature, admin endpoint returns counts. Diagnostic before coding (read-only): roads_with_shape=150568/null=68, trails_with_shape=21995/null=6746 (green), shape blobs are pure WKB MultiLineString. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fix lat-aware buffer + count WKB parse failures - query_buffered_line: replace the static _DEG_PER_M with _buffer_degrees_for_meters(), which scales longitude degrees by cos(lat) and uses the larger lat/lon equivalent; buffer at the polyline avg latitude. Early-return [] for empty coords. - _load_table: count WKB parse failures and logger.warning once per table when > 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 18:53:58 -06:00
from .mvum import MVUMSpatialIndex
MVUM Layer 3a: multi-modal Auto via MVUM trailhead transitions (#26) Auto now also considers "drive in to a trailhead, switch vehicles, continue on foot/2w/4w" trips and picks one when it is meaningfully faster than the single-mode winner. Implicit — no new chip; Auto just returns the fastest plan. Backend: - mvum_transitions.py: TrailheadIndex (STRtree over trail_entry_points), built once per process via load_trailheads() (mirrors the MVUMSpatialIndex singleton). query_trailheads_near_line(coords, buffer_m=2000) with a precise distance filter. - router.py: _route_auto, after the single-mode probe and only when the winner is ok AND total_distance_km >= MIN_HYBRID_DISTANCE_KM (8.0), tries hybrids. For each candidate trailhead near the winning polyline (closest first, capped at 20) and each (drive, offroad) pair in HYBRID_PAIRS, it routes both legs (annotate_mvum off) and sums leg times with NO transition cost. A hybrid wins only if it beats the single-mode winner by >= HYBRID_MIN_TIME_SAVINGS_MIN (15 min); trivial offroad detours (< HYBRID_MIN_OFFROAD_KM = 0.8 km) are skipped. The winner is combined into a new "multi" scenario: leg1 features + a kind=transition marker + leg2 features; summary carries total_*, per-leg legs[], summed MVUM counts; selected_mode="hybrid". Each leg is annotated separately. - app.py / offroute_route.py: load + inject the trailhead index singleton. Frontend (additive — no api.js signature change): - DirectionsPanel: per-leg breakdown row for hybrid/multi ("Drive X mi (Ymin) -> 4W X mi (Zmin) - total Wmin", lucide Repeat between legs); existing Auto badge still shows. - MapView: network polylines colored by network_mode (vehicle/auto blue, 4w orange, 2w green, foot red); transition points rendered as a circle marker with the lucide Repeat icon + "Switch to <mode>" tooltip; bounds fit skips Points. Tests: test_mvum_transitions.py — index load, near-line close-only query, short trip stays single-mode, big-savings hybrid wins, trivial-detour + below-threshold + no-trailheads all fall back. 7 new tests; full offroute suite 71 passed. Note: the DB column is trail_entry_points.highway_class; surfaced as record "road_class" per the spec. Co-authored-by: Matt <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 22:49:13 -06:00
from .mvum_transitions import load_trailheads
MVUM Layer 3b: OSM parking as multi-modal Auto transition candidates (#31) * MVUM Layer 3b: OSM parking as multi-modal Auto transition candidates Adds OSM parking lots as a third multi-modal-Auto transition source alongside MVUM trailheads (3a) and surface-change points (3c), so Auto can suggest "drive to a parking lot, switch to foot/2w/4w" trips where no MVUM trailhead exists -- BLM/state land, urban edges, anywhere OSM has parking but the USFS trailhead layer does not. Backend-only; consumes the already-ingested /mnt/nav/osm-parking.db read-only (no data-pipeline change). - mvum_parking.py: OSMParkingIndex (process-wide singleton via load_parking_index) over a shapely STRtree of parking points, mirroring MVUMSpatialIndex / TrailheadIndex. Read-only SQLite. Drops access in (private,no,permit) at load. query_parking_near_line(coords, buffer_m=2000) with the same coarse-bbox + precise-distance filter as TrailheadIndex. Records carry {lat, lon, name, road_class="parking", parking_type, access}. Perf note: the ingest already stored representative_point() in lat/lon, so the STRtree is built straight from those columns -- parsing the 1.6M WKB blobs at boot would add minutes for an identical point. - router.py: _try_hybrid_auto generalized to gather candidates from each AVAILABLE source (trailhead index if present + surface-change always + parking index if present) instead of hard-returning when trailhead_index is None, so parking-only candidates still work. Combined list keeps the existing closest-first sort + HYBRID_MAX_TRAILHEADS cap. Signature unchanged; record shape already compatible. - app.py / offroute_route.py: load + inject the OSM parking singleton, mirroring MVUM_SPATIAL_INDEX / MVUM_TRAILHEAD_INDEX. Failure logs a warning, degrades None. - admin.py: GET /api/admin/osm-parking/info -> {count, build_time_seconds, memory_estimate_mb}, mirroring /api/admin/mvum-spatial/info. - backend/scripts/ingest_parking.py + README-osm-parking-ingest.md: the data-pipeline ingest lifted to the repo with argparse (--geojsonseq/--db, no /tmp) + the download/filter/export/ingest/restart refresh recipe. Tests: test_mvum_parking.py (loads, near-line close-only, private/no/permit filtered, null-access kept) + test_offroute.py::test_hybrid_consumes_parking_ candidates (parking-only source probed as a leg-1 destination). Full offroute suite: 82 passed. Real-DB sanity (not deployed): index loads 1,489,054 usable parking objects (182,945 access-blocked dropped) in ~12 s using ~950 MB RSS per worker; a Redfish Lake/Sawtooth corridor query returns 8 lots. The ~950 MB/worker memory cost is notable -- flagging for review. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Numpy-pack OSMParkingIndex coords + lazy records to cut RSS (~950->~570 MB/worker) Store parking coords as packed float64 numpy arrays (_lats/_lons) and the attribute columns as interned lists (_names/_parking_types/_accesses), and build candidate record dicts lazily in query_parking_near_line instead of materializing 1.5M dicts + 1.5M shapely Point objects up front. road_class is the constant "parking" so it is not stored per row. Measured on the real /mnt/nav/osm-parking.db (1,489,054 usable rows): RSS/worker ~950 MB -> ~570 MB (~40%), build ~11 s. Across 2 gunicorn workers that is ~1.9 GB -> ~1.14 GB. NOTE: this does NOT reach the ~250 MB originally targeted. The remaining cost is the shapely STRtree itself: it permanently retains the input geometries (tree.geometries len == row count), so the transient `del points` does not free them. Attribution on the real DB: columns-only 137 MB, retained Point objects +230 MB, STRtree index +110 MB. Reaching ~250 MB would require dropping the shapely STRtree for a coordinate-only structure (e.g. scipy cKDTree over the lon/lat arrays), which changes the line-buffer query into a per-vertex radius query -- a behavior change beyond this fix-up's scope. Flagged for a follow-up. Tests unchanged except one assertion (`len(idx.records) == idx.count`); full offroute suite 82 passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 14:48:35 -06:00
from .mvum_parking import load_parking_index
MVUM Layer 0: spatial index foundation (#22) * MVUM Layer 0: spatial index foundation Additive only — no routing logic, no response-format, no Valhalla changes. - mvum.py: add MVUMSpatialIndex. Loads mvum_roads + mvum_trails from navi.db (read-only), decodes the pure-WKB shape blobs with shapely, builds a shapely.strtree.STRtree, and keeps a parallel list of full feature records (all columns minus the blob, plus the parsed geometry) with a by_id lookup. Exposes query_bbox(min_lat,min_lon,max_lat,max_lon) and query_buffered_line(coords, tolerance_m) returning candidate records (coarse bbox + buffer; full parallelism filter is a TODO for PR-B). Reports road_count, trail_count, bbox, build_time_seconds, memory_estimate_mb (RSS delta). - app.py: build the index once per process (singleton) at service init; stored on app.config[MVUM_SPATIAL_INDEX]. Failure is logged and degrades to None. - admin.py: GET /api/admin/mvum-spatial/info (Authentik-gated, read-only) returning the counts/bbox/build-time/memory stats. - tests/test_mvum_spatial.py: index loads, query_bbox returns Boise-area features, query_buffered_line returns a feature, admin endpoint returns counts. Diagnostic before coding (read-only): roads_with_shape=150568/null=68, trails_with_shape=21995/null=6746 (green), shape blobs are pure WKB MultiLineString. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fix lat-aware buffer + count WKB parse failures - query_buffered_line: replace the static _DEG_PER_M with _buffer_degrees_for_meters(), which scales longitude degrees by cos(lat) and uses the larger lat/lon equivalent; buffer at the polyline avg latitude. Early-return [] for empty coords. - _load_table: count WKB parse failures and logger.warning once per table when > 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 18:53:58 -06:00
# Emit INFO+ so PR #34 per-stage Auto timing logs (single-mode probing took...,
# hybrid candidate gathering..., hybrid probing took...) reach stderr/journal;
# navi-offroute had no logging config, so the last-resort handler dropped INFO.
logging.basicConfig(level=logging.INFO)
MVUM Layer 0: spatial index foundation (#22) * MVUM Layer 0: spatial index foundation Additive only — no routing logic, no response-format, no Valhalla changes. - mvum.py: add MVUMSpatialIndex. Loads mvum_roads + mvum_trails from navi.db (read-only), decodes the pure-WKB shape blobs with shapely, builds a shapely.strtree.STRtree, and keeps a parallel list of full feature records (all columns minus the blob, plus the parsed geometry) with a by_id lookup. Exposes query_bbox(min_lat,min_lon,max_lat,max_lon) and query_buffered_line(coords, tolerance_m) returning candidate records (coarse bbox + buffer; full parallelism filter is a TODO for PR-B). Reports road_count, trail_count, bbox, build_time_seconds, memory_estimate_mb (RSS delta). - app.py: build the index once per process (singleton) at service init; stored on app.config[MVUM_SPATIAL_INDEX]. Failure is logged and degrades to None. - admin.py: GET /api/admin/mvum-spatial/info (Authentik-gated, read-only) returning the counts/bbox/build-time/memory stats. - tests/test_mvum_spatial.py: index loads, query_bbox returns Boise-area features, query_buffered_line returns a feature, admin endpoint returns counts. Diagnostic before coding (read-only): roads_with_shape=150568/null=68, trails_with_shape=21995/null=6746 (green), shape blobs are pure WKB MultiLineString. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fix lat-aware buffer + count WKB parse failures - query_buffered_line: replace the static _DEG_PER_M with _buffer_degrees_for_meters(), which scales longitude degrees by cos(lat) and uses the larger lat/lon equivalent; buffer at the polyline avg latitude. Early-return [] for empty coords. - _load_table: count WKB parse failures and logger.warning once per table when > 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 18:53:58 -06:00
# Process-wide singleton: build the MVUM spatial index once per process (per gunicorn
# worker in prod; once across create_app() calls in tests), not once per app instance.
_MVUM_INDEX = None
def _get_mvum_index():
global _MVUM_INDEX
if _MVUM_INDEX is None:
_MVUM_INDEX = MVUMSpatialIndex()
return _MVUM_INDEX
Add navi-offroute service (extraction #8 — final) (#10) * Add navi-offroute service (extraction #8 — the last one) Faithful port of recon's /api/offroute (POST) + /api/mvum (GET) and the runtime offroute modules into a new :8428 service. Closes the loop: after this, navi-frontend talks only to navi-backend. Ported: router.py (OffrouteRouter, EntryPointIndex, 4 route strategies, in-Python MCP_Geometric least-cost path, Valhalla integration, per-request osmium extract), mvum.py (MVUMReader over navi.db), cost.py, friction.py, trails.py, and barriers.py (runtime BarrierReader/WildernessReader only). NOT ported (per Phase A §3/§15): prototype.py (dead at runtime), barriers.py build_*_raster (offline GDB→raster prep). DEM imported from shared/dem.py (PR #9), not duplicated. Behaviour-faithful changes: hardcoded paths/URLs → env vars; the profile.offroute.* config (osm_pbf_path/postgis_dsn/densify_interval_m) → dedicated env vars (router drops deployment_config). Both routes public (no auth, matching recon). PADUS via libpq peer-auth DSN (dbname=padus) — NO secret. Owns no DB. 15 hermetic tests (offroute validation + mocked-router shape + close-always; fixture-SQLite MVUM roads/trails/fallback/null; admin auth + no-secrets + probe shape). Full suite 119 passed / 1 skipped. Adds scikit-image + rasterio. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * navi-offroute: PR #10 review cleanups (4 faithful-port deviations) 1. trails.py — drop recon-era "Run the Phase B rasterization script" reference from the not-found error (confusing in navi-offroute context). 2. friction.py — add FileNotFoundError-before-rasterio-open check to match barriers/trails consistency. 3. mvum.py — remove dead try/except shapely import + warnings.warn at 2 sites (shapely is a hard pyproject dep; the fallback was unreachable). 4. router.py — declare psutil in pyproject, drop the silent fallback; the MEMORY_LIMIT_GB safety check was silently disabled in prod. Adds test_friction_reader_raises_file_not_found_when_missing (16 navi-offroute tests; full suite 120 passed / 1 skipped). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: zvx-echo6 <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 23:30:43 -06:00
def create_app():
app = Flask(__name__)
app.config['VERSION'] = git_short_sha()
app.config['METRICS'] = {
'start_time': time.time(),
'request_count': 0,
'last_error_at': None,
}
@app.before_request
def _count_request():
app.config['METRICS']['request_count'] += 1
@app.after_request
def _track_errors(response):
if response.status_code >= 500:
app.config['METRICS']['last_error_at'] = time.strftime(
'%Y-%m-%dT%H:%M:%SZ', time.gmtime()
)
return response
MVUM Layer 0: spatial index foundation (#22) * MVUM Layer 0: spatial index foundation Additive only — no routing logic, no response-format, no Valhalla changes. - mvum.py: add MVUMSpatialIndex. Loads mvum_roads + mvum_trails from navi.db (read-only), decodes the pure-WKB shape blobs with shapely, builds a shapely.strtree.STRtree, and keeps a parallel list of full feature records (all columns minus the blob, plus the parsed geometry) with a by_id lookup. Exposes query_bbox(min_lat,min_lon,max_lat,max_lon) and query_buffered_line(coords, tolerance_m) returning candidate records (coarse bbox + buffer; full parallelism filter is a TODO for PR-B). Reports road_count, trail_count, bbox, build_time_seconds, memory_estimate_mb (RSS delta). - app.py: build the index once per process (singleton) at service init; stored on app.config[MVUM_SPATIAL_INDEX]. Failure is logged and degrades to None. - admin.py: GET /api/admin/mvum-spatial/info (Authentik-gated, read-only) returning the counts/bbox/build-time/memory stats. - tests/test_mvum_spatial.py: index loads, query_bbox returns Boise-area features, query_buffered_line returns a feature, admin endpoint returns counts. Diagnostic before coding (read-only): roads_with_shape=150568/null=68, trails_with_shape=21995/null=6746 (green), shape blobs are pure WKB MultiLineString. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fix lat-aware buffer + count WKB parse failures - query_buffered_line: replace the static _DEG_PER_M with _buffer_degrees_for_meters(), which scales longitude degrees by cos(lat) and uses the larger lat/lon equivalent; buffer at the polyline avg latitude. Early-return [] for empty coords. - _load_table: count WKB parse failures and logger.warning once per table when > 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 18:53:58 -06:00
# Load the MVUM spatial index once at service init (logs its own load line).
try:
app.config['MVUM_SPATIAL_INDEX'] = _get_mvum_index()
except Exception as e:
app.logger.warning("MVUM spatial index failed to load: %s", e)
app.config['MVUM_SPATIAL_INDEX'] = None
MVUM Layer 3a: multi-modal Auto via MVUM trailhead transitions (#26) Auto now also considers "drive in to a trailhead, switch vehicles, continue on foot/2w/4w" trips and picks one when it is meaningfully faster than the single-mode winner. Implicit — no new chip; Auto just returns the fastest plan. Backend: - mvum_transitions.py: TrailheadIndex (STRtree over trail_entry_points), built once per process via load_trailheads() (mirrors the MVUMSpatialIndex singleton). query_trailheads_near_line(coords, buffer_m=2000) with a precise distance filter. - router.py: _route_auto, after the single-mode probe and only when the winner is ok AND total_distance_km >= MIN_HYBRID_DISTANCE_KM (8.0), tries hybrids. For each candidate trailhead near the winning polyline (closest first, capped at 20) and each (drive, offroad) pair in HYBRID_PAIRS, it routes both legs (annotate_mvum off) and sums leg times with NO transition cost. A hybrid wins only if it beats the single-mode winner by >= HYBRID_MIN_TIME_SAVINGS_MIN (15 min); trivial offroad detours (< HYBRID_MIN_OFFROAD_KM = 0.8 km) are skipped. The winner is combined into a new "multi" scenario: leg1 features + a kind=transition marker + leg2 features; summary carries total_*, per-leg legs[], summed MVUM counts; selected_mode="hybrid". Each leg is annotated separately. - app.py / offroute_route.py: load + inject the trailhead index singleton. Frontend (additive — no api.js signature change): - DirectionsPanel: per-leg breakdown row for hybrid/multi ("Drive X mi (Ymin) -> 4W X mi (Zmin) - total Wmin", lucide Repeat between legs); existing Auto badge still shows. - MapView: network polylines colored by network_mode (vehicle/auto blue, 4w orange, 2w green, foot red); transition points rendered as a circle marker with the lucide Repeat icon + "Switch to <mode>" tooltip; bounds fit skips Points. Tests: test_mvum_transitions.py — index load, near-line close-only query, short trip stays single-mode, big-savings hybrid wins, trivial-detour + below-threshold + no-trailheads all fall back. 7 new tests; full offroute suite 71 passed. Note: the DB column is trail_entry_points.highway_class; surfaced as record "road_class" per the spec. Co-authored-by: Matt <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 22:49:13 -06:00
# Layer 3a: trailhead transition index (process-wide singleton, logs its own line).
try:
app.config['MVUM_TRAILHEAD_INDEX'] = load_trailheads()
except Exception as e:
app.logger.warning("MVUM trailhead index failed to load: %s", e)
app.config['MVUM_TRAILHEAD_INDEX'] = None
MVUM Layer 3b: OSM parking as multi-modal Auto transition candidates (#31) * MVUM Layer 3b: OSM parking as multi-modal Auto transition candidates Adds OSM parking lots as a third multi-modal-Auto transition source alongside MVUM trailheads (3a) and surface-change points (3c), so Auto can suggest "drive to a parking lot, switch to foot/2w/4w" trips where no MVUM trailhead exists -- BLM/state land, urban edges, anywhere OSM has parking but the USFS trailhead layer does not. Backend-only; consumes the already-ingested /mnt/nav/osm-parking.db read-only (no data-pipeline change). - mvum_parking.py: OSMParkingIndex (process-wide singleton via load_parking_index) over a shapely STRtree of parking points, mirroring MVUMSpatialIndex / TrailheadIndex. Read-only SQLite. Drops access in (private,no,permit) at load. query_parking_near_line(coords, buffer_m=2000) with the same coarse-bbox + precise-distance filter as TrailheadIndex. Records carry {lat, lon, name, road_class="parking", parking_type, access}. Perf note: the ingest already stored representative_point() in lat/lon, so the STRtree is built straight from those columns -- parsing the 1.6M WKB blobs at boot would add minutes for an identical point. - router.py: _try_hybrid_auto generalized to gather candidates from each AVAILABLE source (trailhead index if present + surface-change always + parking index if present) instead of hard-returning when trailhead_index is None, so parking-only candidates still work. Combined list keeps the existing closest-first sort + HYBRID_MAX_TRAILHEADS cap. Signature unchanged; record shape already compatible. - app.py / offroute_route.py: load + inject the OSM parking singleton, mirroring MVUM_SPATIAL_INDEX / MVUM_TRAILHEAD_INDEX. Failure logs a warning, degrades None. - admin.py: GET /api/admin/osm-parking/info -> {count, build_time_seconds, memory_estimate_mb}, mirroring /api/admin/mvum-spatial/info. - backend/scripts/ingest_parking.py + README-osm-parking-ingest.md: the data-pipeline ingest lifted to the repo with argparse (--geojsonseq/--db, no /tmp) + the download/filter/export/ingest/restart refresh recipe. Tests: test_mvum_parking.py (loads, near-line close-only, private/no/permit filtered, null-access kept) + test_offroute.py::test_hybrid_consumes_parking_ candidates (parking-only source probed as a leg-1 destination). Full offroute suite: 82 passed. Real-DB sanity (not deployed): index loads 1,489,054 usable parking objects (182,945 access-blocked dropped) in ~12 s using ~950 MB RSS per worker; a Redfish Lake/Sawtooth corridor query returns 8 lots. The ~950 MB/worker memory cost is notable -- flagging for review. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Numpy-pack OSMParkingIndex coords + lazy records to cut RSS (~950->~570 MB/worker) Store parking coords as packed float64 numpy arrays (_lats/_lons) and the attribute columns as interned lists (_names/_parking_types/_accesses), and build candidate record dicts lazily in query_parking_near_line instead of materializing 1.5M dicts + 1.5M shapely Point objects up front. road_class is the constant "parking" so it is not stored per row. Measured on the real /mnt/nav/osm-parking.db (1,489,054 usable rows): RSS/worker ~950 MB -> ~570 MB (~40%), build ~11 s. Across 2 gunicorn workers that is ~1.9 GB -> ~1.14 GB. NOTE: this does NOT reach the ~250 MB originally targeted. The remaining cost is the shapely STRtree itself: it permanently retains the input geometries (tree.geometries len == row count), so the transient `del points` does not free them. Attribution on the real DB: columns-only 137 MB, retained Point objects +230 MB, STRtree index +110 MB. Reaching ~250 MB would require dropping the shapely STRtree for a coordinate-only structure (e.g. scipy cKDTree over the lon/lat arrays), which changes the line-buffer query into a per-vertex radius query -- a behavior change beyond this fix-up's scope. Flagged for a follow-up. Tests unchanged except one assertion (`len(idx.records) == idx.count`); full offroute suite 82 passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 14:48:35 -06:00
# Layer 3b: OSM parking index (process-wide singleton, logs its own line).
try:
app.config['OSM_PARKING_INDEX'] = load_parking_index()
except Exception as e:
app.logger.warning("OSM parking index failed to load: %s", e)
app.config['OSM_PARKING_INDEX'] = None
Add navi-offroute service (extraction #8 — final) (#10) * Add navi-offroute service (extraction #8 — the last one) Faithful port of recon's /api/offroute (POST) + /api/mvum (GET) and the runtime offroute modules into a new :8428 service. Closes the loop: after this, navi-frontend talks only to navi-backend. Ported: router.py (OffrouteRouter, EntryPointIndex, 4 route strategies, in-Python MCP_Geometric least-cost path, Valhalla integration, per-request osmium extract), mvum.py (MVUMReader over navi.db), cost.py, friction.py, trails.py, and barriers.py (runtime BarrierReader/WildernessReader only). NOT ported (per Phase A §3/§15): prototype.py (dead at runtime), barriers.py build_*_raster (offline GDB→raster prep). DEM imported from shared/dem.py (PR #9), not duplicated. Behaviour-faithful changes: hardcoded paths/URLs → env vars; the profile.offroute.* config (osm_pbf_path/postgis_dsn/densify_interval_m) → dedicated env vars (router drops deployment_config). Both routes public (no auth, matching recon). PADUS via libpq peer-auth DSN (dbname=padus) — NO secret. Owns no DB. 15 hermetic tests (offroute validation + mocked-router shape + close-always; fixture-SQLite MVUM roads/trails/fallback/null; admin auth + no-secrets + probe shape). Full suite 119 passed / 1 skipped. Adds scikit-image + rasterio. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * navi-offroute: PR #10 review cleanups (4 faithful-port deviations) 1. trails.py — drop recon-era "Run the Phase B rasterization script" reference from the not-found error (confusing in navi-offroute context). 2. friction.py — add FileNotFoundError-before-rasterio-open check to match barriers/trails consistency. 3. mvum.py — remove dead try/except shapely import + warnings.warn at 2 sites (shapely is a hard pyproject dep; the fallback was unreachable). 4. router.py — declare psutil in pyproject, drop the silent fallback; the MEMORY_LIMIT_GB safety check was silently disabled in prod. Adds test_friction_reader_raises_file_not_found_when_missing (16 navi-offroute tests; full suite 120 passed / 1 skipped). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: zvx-echo6 <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 23:30:43 -06:00
app.register_blueprint(offroute_route.bp)
app.register_blueprint(admin.bp)
return app