MVUM Layer 3b: OSM parking as multi-modal Auto transition candidates (#31)

* MVUM Layer 3b: OSM parking as multi-modal Auto transition candidates

Adds OSM parking lots as a third multi-modal-Auto transition source alongside
MVUM trailheads (3a) and surface-change points (3c), so Auto can suggest
"drive to a parking lot, switch to foot/2w/4w" trips where no MVUM trailhead
exists -- BLM/state land, urban edges, anywhere OSM has parking but the USFS
trailhead layer does not. Backend-only; consumes the already-ingested
/mnt/nav/osm-parking.db read-only (no data-pipeline change).

- mvum_parking.py: OSMParkingIndex (process-wide singleton via load_parking_index)
  over a shapely STRtree of parking points, mirroring MVUMSpatialIndex /
  TrailheadIndex. Read-only SQLite. Drops access in (private,no,permit) at load.
  query_parking_near_line(coords, buffer_m=2000) with the same coarse-bbox +
  precise-distance filter as TrailheadIndex. Records carry
  {lat, lon, name, road_class="parking", parking_type, access}.
  Perf note: the ingest already stored representative_point() in lat/lon, so the
  STRtree is built straight from those columns -- parsing the 1.6M WKB blobs at
  boot would add minutes for an identical point.
- router.py: _try_hybrid_auto generalized to gather candidates from each AVAILABLE
  source (trailhead index if present + surface-change always + parking index if
  present) instead of hard-returning when trailhead_index is None, so parking-only
  candidates still work. Combined list keeps the existing closest-first sort +
  HYBRID_MAX_TRAILHEADS cap. Signature unchanged; record shape already compatible.
- app.py / offroute_route.py: load + inject the OSM parking singleton, mirroring
  MVUM_SPATIAL_INDEX / MVUM_TRAILHEAD_INDEX. Failure logs a warning, degrades None.
- admin.py: GET /api/admin/osm-parking/info -> {count, build_time_seconds,
  memory_estimate_mb}, mirroring /api/admin/mvum-spatial/info.
- backend/scripts/ingest_parking.py + README-osm-parking-ingest.md: the
  data-pipeline ingest lifted to the repo with argparse (--geojsonseq/--db, no
  /tmp) + the download/filter/export/ingest/restart refresh recipe.

Tests: test_mvum_parking.py (loads, near-line close-only, private/no/permit
filtered, null-access kept) + test_offroute.py::test_hybrid_consumes_parking_
candidates (parking-only source probed as a leg-1 destination). Full offroute
suite: 82 passed.

Real-DB sanity (not deployed): index loads 1,489,054 usable parking objects
(182,945 access-blocked dropped) in ~12 s using ~950 MB RSS per worker; a Redfish
Lake/Sawtooth corridor query returns 8 lots. The ~950 MB/worker memory cost is
notable -- flagging for review.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Numpy-pack OSMParkingIndex coords + lazy records to cut RSS (~950->~570 MB/worker)

Store parking coords as packed float64 numpy arrays (_lats/_lons) and the
attribute columns as interned lists (_names/_parking_types/_accesses), and build
candidate record dicts lazily in query_parking_near_line instead of materializing
1.5M dicts + 1.5M shapely Point objects up front. road_class is the constant
"parking" so it is not stored per row.

Measured on the real /mnt/nav/osm-parking.db (1,489,054 usable rows):
RSS/worker ~950 MB -> ~570 MB (~40%), build ~11 s. Across 2 gunicorn workers that
is ~1.9 GB -> ~1.14 GB.

NOTE: this does NOT reach the ~250 MB originally targeted. The remaining cost is
the shapely STRtree itself: it permanently retains the input geometries
(tree.geometries len == row count), so the transient `del points` does not free
them. Attribution on the real DB: columns-only 137 MB, retained Point objects
+230 MB, STRtree index +110 MB. Reaching ~250 MB would require dropping the
shapely STRtree for a coordinate-only structure (e.g. scipy cKDTree over the
lon/lat arrays), which changes the line-buffer query into a per-vertex radius
query -- a behavior change beyond this fix-up's scope. Flagged for a follow-up.

Tests unchanged except one assertion (`len(idx.records) == idx.count`); full
offroute suite 82 passed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

backend/services/navi_offroute/admin.py

@@ -149,3 +149,17 @@ def mvum_spatial_info():
         'build_time_seconds': round(idx.build_time_seconds, 3),
         'memory_estimate_mb': round(idx.memory_estimate_mb, 1),
     })
+
+
+@bp.route('/api/admin/osm-parking/info')
+@require_auth
+def osm_parking_info():
+    """Read-only stats for the in-memory OSM parking index (Layer 3b)."""
+    idx = current_app.config.get('OSM_PARKING_INDEX')
+    if idx is None:
+        return jsonify({'status': 'error', 'message': 'OSM parking index not loaded'}), 503
+    return jsonify({
+        'count': idx.count,
+        'build_time_seconds': round(idx.build_time_seconds, 3),
+        'memory_estimate_mb': round(idx.memory_estimate_mb, 1),
+    })

backend/services/navi_offroute/app.py

@@ -12,6 +12,7 @@ from shared.git_sha import git_short_sha
 from . import offroute_route, admin
 from .mvum import MVUMSpatialIndex
 from .mvum_transitions import load_trailheads
+from .mvum_parking import load_parking_index
 
 # Process-wide singleton: build the MVUM spatial index once per process (per gunicorn
 # worker in prod; once across create_app() calls in tests), not once per app instance.
@@ -62,6 +63,13 @@ def create_app():
         app.logger.warning("MVUM trailhead index failed to load: %s", e)
         app.config['MVUM_TRAILHEAD_INDEX'] = None
 
+    # Layer 3b: OSM parking index (process-wide singleton, logs its own line).
+    try:
+        app.config['OSM_PARKING_INDEX'] = load_parking_index()
+    except Exception as e:
+        app.logger.warning("OSM parking index failed to load: %s", e)
+        app.config['OSM_PARKING_INDEX'] = None
+
     app.register_blueprint(offroute_route.bp)
     app.register_blueprint(admin.bp)
     return app

backend/services/navi_offroute/mvum_parking.py

@@ -0,0 +1,154 @@
+"""
+MVUM Layer 3b: OSM parking as multi-modal Auto transition candidates.
+
+Loads ``/mnt/nav/osm-parking.db`` (amenity=parking objects ingested from the
+geofabrik North America extract) into a shapely STRtree of parking points, so Auto
+can suggest "drive to a parking lot, switch to foot/2w/4w" trips where no MVUM
+trailhead exists — BLM/state land, urban edges, anywhere OSM has parking but the
+USFS trailhead layer does not. Read-only, pure spatial lookup; mirrors the
+MVUMSpatialIndex (Layer 0) / TrailheadIndex (Layer 3a) singleton pattern.
+
+Coordinates are stored in packed numpy arrays and the per-feature attribute columns
+as plain lists; the shapely Point objects exist only long enough to build the
+STRtree and are then released. Candidate record dicts are constructed lazily in
+query_parking_near_line. This keeps RSS to a few hundred MB for ~1.5M rows instead
+of ~1 GB of per-row dicts + Point objects.
+"""
+import logging
+import os
+import sqlite3
+import sys
+import time as _time
+from pathlib import Path
+
+import numpy as np
+import psutil
+from shapely.geometry import Point, LineString
+from shapely.strtree import STRtree
+
+from .mvum import _buffer_degrees_for_meters
+
+logger = logging.getLogger("navi_offroute.mvum_parking")
+
+DEFAULT_PARKING_DB = Path("/mnt/nav/osm-parking.db")
+
+# Parking that is off-limits as a public transition point.
+_BLOCKED_ACCESS = frozenset({"private", "no", "permit"})
+
+
+def parking_db_path() -> Path:
+    """osm-parking.db path, env-overridable via NAVI_OFFROUTE_PARKING_DB."""
+    return Path(os.environ.get("NAVI_OFFROUTE_PARKING_DB", str(DEFAULT_PARKING_DB)))
+
+
+class OSMParkingIndex:
+    """In-memory STRtree over OSM parking points from osm-parking.db.
+
+    Storage is columnar: ``_lats``/``_lons`` (float64 numpy arrays) plus
+    ``_names``/``_parking_types``/``_accesses`` (lists, aligned by index).
+    ``road_class`` is the constant ``"parking"`` so it is not stored per row.
+    query_parking_near_line() builds the ``{lat, lon, name, road_class,
+    parking_type, access}`` record dicts lazily from these columns.
+    """
+
+    def __init__(self, db_path=None):
+        t0 = _time.perf_counter()
+        proc = psutil.Process()
+        rss_before = proc.memory_info().rss
+
+        self.db_path = Path(db_path) if db_path else parking_db_path()
+        lats, lons = [], []
+        self._names, self._parking_types, self._accesses = [], [], []
+        skipped_access = 0
+
+        conn = sqlite3.connect(f"file:{self.db_path}?mode=ro", uri=True)
+        conn.row_factory = sqlite3.Row
+        try:
+            cur = conn.execute(
+                "SELECT name, capacity, access, parking_type, lat, lon FROM parking")
+            for row in cur:
+                access = row["access"]
+                if access in _BLOCKED_ACCESS:
+                    skipped_access += 1
+                    continue
+                lat, lon = row["lat"], row["lon"]
+                if lat is None or lon is None:
+                    continue
+                # The ingest already stored representative_point() (an interior point
+                # of each parking polygon) in the lat/lon columns, so the STRtree is
+                # built straight from them -- parsing the 1.5M WKB shape blobs here
+                # would add minutes to every worker boot for an identical point.
+                lats.append(float(lat))
+                lons.append(float(lon))
+                self._names.append(row["name"] or "")
+                # intern the small-cardinality attribute strings so duplicate values
+                # share one object instead of 1.5M separate ones.
+                pt = row["parking_type"]
+                self._parking_types.append(sys.intern(pt) if isinstance(pt, str) else pt)
+                self._accesses.append(sys.intern(access) if isinstance(access, str) else access)
+        finally:
+            conn.close()
+
+        self._lats = np.asarray(lats, dtype=np.float64)
+        self._lons = np.asarray(lons, dtype=np.float64)
+
+        # Build the STRtree from transient Point objects, then release them; the tree
+        # internalizes its own geometry storage and we reconstruct points on demand.
+        points = [Point(lon, lat) for lon, lat in zip(lons, lats)]
+        self._tree = STRtree(points) if points else None
+        del points
+
+        self.count = len(self._lats)
+        self.skipped_access = skipped_access
+        self.build_time_seconds = _time.perf_counter() - t0
+        self.memory_estimate_mb = max(
+            0.0, (proc.memory_info().rss - rss_before) / (1024 * 1024))
+        logger.info(
+            "OSM parking index loaded: %d parking objects (%d access-blocked skipped) "
+            "in %.2f seconds", self.count, skipped_access, self.build_time_seconds)
+
+    def _record(self, i):
+        """Construct a candidate record dict for column index ``i``."""
+        return {
+            "lat": float(self._lats[i]),
+            "lon": float(self._lons[i]),
+            "name": self._names[i],
+            "road_class": "parking",
+            "parking_type": self._parking_types[i],
+            "access": self._accesses[i],
+        }
+
+    @property
+    def records(self):
+        """All records, built lazily (used by tests / introspection — not the hot path)."""
+        return [self._record(i) for i in range(self.count)]
+
+    def query_parking_near_line(self, coords, buffer_m=2000):
+        """Parking records within ~``buffer_m`` of a (lat, lon) polyline.
+
+        Coarse STRtree bbox prefilter then a precise degree-distance check, matching
+        TrailheadIndex.query_trailheads_near_line.
+        """
+        if not coords or self._tree is None:
+            return []
+        pts = [(lon, lat) for (lat, lon) in coords]
+        geom = LineString(pts) if len(pts) >= 2 else Point(pts[0])
+        avg_lat = sum(lat for (lat, lon) in coords) / len(coords)
+        buffer_deg = _buffer_degrees_for_meters(buffer_m, avg_lat)
+        out = []
+        for i in self._tree.query(geom.buffer(buffer_deg)):
+            if geom.distance(Point(self._lons[i], self._lats[i])) <= buffer_deg:
+                out.append(self._record(i))
+        return out
+
+
+# Process-wide singleton, mirroring app.py's _MVUM_INDEX / trailhead handling.
+_PARKING_INDEX = None
+
+
+def load_parking_index(db_path=None):
+    """Return the process-wide OSMParkingIndex singleton, building it on first call."""
+    global _PARKING_INDEX
+    if _PARKING_INDEX is None:
+        _PARKING_INDEX = OSMParkingIndex(db_path)
+    return _PARKING_INDEX

backend/services/navi_offroute/offroute_route.py

@@ -75,6 +75,8 @@ def api_offroute():
         router.spatial_index = current_app.config.get('MVUM_SPATIAL_INDEX')
         # Inject the Layer-3a trailhead index for multi-modal Auto transitions.
         router.trailhead_index = current_app.config.get('MVUM_TRAILHEAD_INDEX')
+        # Inject the Layer-3b OSM parking index for multi-modal Auto transitions.
+        router.parking_index = current_app.config.get('OSM_PARKING_INDEX')
         try:
             result = router.route(
                 start_lat=start_lat, start_lon=start_lon,

backend/services/navi_offroute/router.py

@@ -35,6 +35,7 @@ import psycopg2.extras
 from shapely.geometry import LineString, Point
 from .astar import astar_multigoal, inflate_cost_multiplier
 from .mvum_surface_change import get_surface_change_candidates
+from .mvum_parking import load_parking_index  # noqa: F401 (singleton injected by handler)
 
 from shared.dem import DEMReader, dem_path
 from .cost import compute_cost_grid, compute_cost_multiplier_grid, MODE_PROFILES
@@ -550,6 +551,7 @@ class OffrouteRouter:
         self.mvum_on_date = None    # optional datetime for seasonal MVUM checks
         self._exclude_polygons = None  # MVUM Layer 2c, set per route() call
         self.trailhead_index = None    # TrailheadIndex (Layer 3a), injected by the handler
+        self.parking_index = None      # OSMParkingIndex (Layer 3b), injected by the handler
 
     def _init_readers(self):
         """Lazy init readers."""
@@ -921,9 +923,6 @@ class OffrouteRouter:
         single-mode winner by HYBRID_MIN_TIME_SAVINGS_MIN, else None (caller keeps
         the single-mode winner). Leg times are summed with no transition penalty.
         """
-        idx = getattr(self, "trailhead_index", None)
-        if idx is None:
-            return None
         best_summary = best_result.get("summary") or {}
         if best_summary.get("total_distance_km", 0.0) < MIN_HYBRID_DISTANCE_KM:
             return None
@@ -931,12 +930,24 @@ class OffrouteRouter:
         coords = self._route_coords_latlon(best_result)
         if len(coords) < 2:
             return None
-        candidates = idx.query_trailheads_near_line(
-            coords, buffer_m=HYBRID_TRAILHEAD_BUFFER_M)
-        # Layer 3c: also treat surface-category boundaries along the winning polyline
-        # (e.g. pavement -> dirt) as transition candidates. Same record shape, so they
-        # mix freely with trailheads below.
-        candidates = candidates + get_surface_change_candidates(coords, VALHALLA_URL)
+
+        # Gather transition candidates from every available source; each yields the
+        # same {lat, lon, name, road_class, ...} record shape, so they mix freely and
+        # share the closest-first sort + cap below.
+        candidates = []
+        # Layer 3a: MVUM/USFS trailheads near the winning polyline.
+        th_idx = getattr(self, "trailhead_index", None)
+        if th_idx is not None:
+            candidates += th_idx.query_trailheads_near_line(
+                coords, buffer_m=HYBRID_TRAILHEAD_BUFFER_M)
+        # Layer 3c: surface-category boundaries along the polyline (e.g. pavement -> dirt).
+        candidates += get_surface_change_candidates(coords, VALHALLA_URL)
+        # Layer 3b: OSM parking -- covers BLM/state/private land + urban areas where
+        # MVUM trailheads don't exist.
+        pk_idx = getattr(self, "parking_index", None)
+        if pk_idx is not None:
+            candidates += pk_idx.query_parking_near_line(
+                coords, buffer_m=HYBRID_TRAILHEAD_BUFFER_M)
         if not candidates:
             return None
         # Closest-to-route first, then cap the combined list.

backend/services/navi_offroute/tests/test_mvum_parking.py

@@ -0,0 +1,75 @@
+"""MVUM Layer 3b tests: OSMParkingIndex over a synthetic osm-parking.db."""
+import sqlite3
+
+import pytest
+
+from services.navi_offroute.mvum_parking import OSMParkingIndex
+
+
+def _parking_db(tmp_path, rows):
+    """rows: list of (osm_id, osm_type, name, capacity, access, parking_type, lat, lon)."""
+    db = tmp_path / "osm-parking.db"
+    conn = sqlite3.connect(db)
+    conn.execute(
+        "CREATE TABLE parking (id INTEGER PRIMARY KEY, osm_id TEXT, osm_type TEXT, "
+        "name TEXT, capacity INTEGER, access TEXT, parking_type TEXT, "
+        "lat REAL, lon REAL, shape BLOB)")
+    conn.executemany(
+        "INSERT INTO parking (osm_id,osm_type,name,capacity,access,parking_type,lat,lon) "
+        "VALUES (?,?,?,?,?,?,?,?)", rows)
+    conn.commit()
+    conn.close()
+    return db
+
+
+def test_parking_index_loads(tmp_path):
+    db = _parking_db(tmp_path, [
+        ("1", "node", "Lot A", 20, None, "surface", 44.00, -114.00),
+        ("2", "way", "Lot B", None, "yes", "surface", 44.01, -114.02),
+        ("3", "way", "", None, "customers", None, 44.02, -114.03),
+    ])
+    idx = OSMParkingIndex(db_path=db)
+    assert idx.count == 3
+    assert len(idx.records) == idx.count == 3
+    rec = idx.records[0]
+    assert rec["name"] == "Lot A"
+    assert rec["road_class"] == "parking"
+    assert rec["parking_type"] == "surface"
+    assert rec["lat"] == 44.00 and rec["lon"] == -114.00
+
+
+def test_query_parking_near_line_returns_close_only(tmp_path):
+    db = _parking_db(tmp_path, [
+        ("1", "node", "On Line", None, None, "surface", 44.000, -114.000),
+        ("2", "node", "Far Away", None, None, "surface", 44.300, -114.000),  # ~33 km N
+    ])
+    idx = OSMParkingIndex(db_path=db)
+    line = [(44.000, -114.010), (44.000, -113.990)]  # ~1.6 km segment through the close pt
+    near = idx.query_parking_near_line(line, buffer_m=2000)
+    names = {r["name"] for r in near}
+    assert "On Line" in names
+    assert "Far Away" not in names
+
+
+def test_private_parking_filtered_out(tmp_path):
+    db = _parking_db(tmp_path, [
+        ("1", "way", "Public", None, "yes", "surface", 44.00, -114.00),
+        ("2", "way", "Private", None, "private", "surface", 44.01, -114.01),
+        ("3", "way", "NoAccess", None, "no", "surface", 44.02, -114.02),
+        ("4", "way", "PermitOnly", None, "permit", "surface", 44.03, -114.03),
+    ])
+    idx = OSMParkingIndex(db_path=db)
+    names = {r["name"] for r in idx.records}
+    assert names == {"Public"}
+    assert idx.count == 1
+    assert idx.skipped_access == 3
+
+
+def test_no_access_field_kept(tmp_path):
+    # Most OSM parking rows have NULL access -> must be kept (not treated as blocked).
+    db = _parking_db(tmp_path, [
+        ("1", "way", "Unspecified", None, None, "surface", 44.00, -114.00),
+    ])
+    idx = OSMParkingIndex(db_path=db)
+    assert idx.count == 1
+    assert idx.records[0]["access"] is None

backend/services/navi_offroute/tests/test_offroute.py

@@ -884,3 +884,78 @@ def test_route_auto_annotates_only_winner(monkeypatch):
     out = r._route_auto(42.0, -114.0, 42.5, -114.5, "pragmatic")
     assert out["selected_mode"] == "4w"
     assert annotated == ["4w"]   # annotated once, on the winner only
+
+
+# ── Layer 3b: parking as a hybrid transition candidate source ──
+
+def _hybrid_ok_leg(distance_km, minutes):
+    return {
+        "status": "ok",
+        "route": {"type": "FeatureCollection", "features": [
+            {"type": "Feature",
+             "properties": {"segment_type": "network", "network_mode": "x"},
+             "geometry": {"type": "LineString",
+                          "coordinates": [[-114.0, 44.0], [-114.1, 44.0]]}},
+        ]},
+        "summary": {"total_distance_km": distance_km, "total_effort_minutes": minutes,
+                    "network_distance_km": distance_km, "network_duration_minutes": minutes,
+                    "wilderness_distance_km": 0.0, "wilderness_effort_minutes": 0.0,
+                    "scenario": "D"},
+    }
+
+
+def _hybrid_winning_single_mode(distance_km, minutes):
+    return {
+        "status": "ok",
+        "route": {"type": "FeatureCollection", "features": [
+            {"type": "Feature", "properties": {"segment_type": "combined"},
+             "geometry": {"type": "LineString",
+                          "coordinates": [[-114.0, 44.0], [-114.5, 44.0]]}},
+        ]},
+        "summary": {"total_distance_km": distance_km, "total_effort_minutes": minutes,
+                    "scenario": "D"},
+        "selected_mode": "vehicle",
+    }
+
+
+class _FakeParking:
+    def __init__(self, records):
+        self._records = records
+
+    def query_parking_near_line(self, coords, buffer_m=2000):
+        return list(self._records)
+
+
+def test_hybrid_consumes_parking_candidates(monkeypatch):
+    # Only the parking index supplies candidates (no trailhead index, no surface
+    # changes); the parking lot must be probed as a leg-1 destination and win.
+    parking = {"lat": 44.0, "lon": -114.25, "name": "BLM Trailhead Lot",
+               "road_class": "parking", "parking_type": "surface", "access": None}
+    monkeypatch.setattr("services.navi_offroute.router.get_surface_change_candidates",
+                        lambda coords, url: [])
+
+    seen_dests = []
+
+    def fake_route(self, s_lat, s_lon, e_lat, e_lon, mode="foot",
+                   boundary_mode="pragmatic", annotate_mvum=True, **k):
+        seen_dests.append((round(e_lat, 4), round(e_lon, 4)))
+        if mode == "vehicle":
+            return _hybrid_ok_leg(12.0, 20.0)
+        return _hybrid_ok_leg(4.0, 30.0)
+    monkeypatch.setattr(OffrouteRouter, "route", fake_route)
+
+    r = object.__new__(OffrouteRouter)
+    r.spatial_index = None
+    r.trailhead_index = None          # no trailheads -> parking must still be gathered
+    r.parking_index = _FakeParking([parking])
+
+    best = _hybrid_winning_single_mode(distance_km=20.0, minutes=120.0)
+    out = r._try_hybrid_auto(44.0, -114.0, 44.0, -114.5, "pragmatic",
+                             best, 120.0, frozenset({"vehicle", "4w", "2w", "foot"}))
+    assert out is not None
+    assert out["selected_mode"] == "hybrid"
+    # the parking lot was probed as a drive-to (leg-1) destination
+    assert (round(parking["lat"], 4), round(parking["lon"], 4)) in seen_dests
+    trans = next(f for f in out["route"]["features"]
+                 if f["properties"].get("kind") == "transition")
+    assert trans["properties"]["name"] == "BLM Trailhead Lot"