* MVUM Layer 3b: OSM parking as multi-modal Auto transition candidates
Adds OSM parking lots as a third multi-modal-Auto transition source alongside
MVUM trailheads (3a) and surface-change points (3c), so Auto can suggest
"drive to a parking lot, switch to foot/2w/4w" trips where no MVUM trailhead
exists -- BLM/state land, urban edges, anywhere OSM has parking but the USFS
trailhead layer does not. Backend-only; consumes the already-ingested
/mnt/nav/osm-parking.db read-only (no data-pipeline change).
- mvum_parking.py: OSMParkingIndex (process-wide singleton via load_parking_index)
over a shapely STRtree of parking points, mirroring MVUMSpatialIndex /
TrailheadIndex. Read-only SQLite. Drops access in (private,no,permit) at load.
query_parking_near_line(coords, buffer_m=2000) with the same coarse-bbox +
precise-distance filter as TrailheadIndex. Records carry
{lat, lon, name, road_class="parking", parking_type, access}.
Perf note: the ingest already stored representative_point() in lat/lon, so the
STRtree is built straight from those columns -- parsing the 1.6M WKB blobs at
boot would add minutes for an identical point.
- router.py: _try_hybrid_auto generalized to gather candidates from each AVAILABLE
source (trailhead index if present + surface-change always + parking index if
present) instead of hard-returning when trailhead_index is None, so parking-only
candidates still work. Combined list keeps the existing closest-first sort +
HYBRID_MAX_TRAILHEADS cap. Signature unchanged; record shape already compatible.
- app.py / offroute_route.py: load + inject the OSM parking singleton, mirroring
MVUM_SPATIAL_INDEX / MVUM_TRAILHEAD_INDEX. Failure logs a warning, degrades None.
- admin.py: GET /api/admin/osm-parking/info -> {count, build_time_seconds,
memory_estimate_mb}, mirroring /api/admin/mvum-spatial/info.
- backend/scripts/ingest_parking.py + README-osm-parking-ingest.md: the
data-pipeline ingest lifted to the repo with argparse (--geojsonseq/--db, no
/tmp) + the download/filter/export/ingest/restart refresh recipe.
Tests: test_mvum_parking.py (loads, near-line close-only, private/no/permit
filtered, null-access kept) + test_offroute.py::test_hybrid_consumes_parking_
candidates (parking-only source probed as a leg-1 destination). Full offroute
suite: 82 passed.
Real-DB sanity (not deployed): index loads 1,489,054 usable parking objects
(182,945 access-blocked dropped) in ~12 s using ~950 MB RSS per worker; a Redfish
Lake/Sawtooth corridor query returns 8 lots. The ~950 MB/worker memory cost is
notable -- flagging for review.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Numpy-pack OSMParkingIndex coords + lazy records to cut RSS (~950->~570 MB/worker)
Store parking coords as packed float64 numpy arrays (_lats/_lons) and the
attribute columns as interned lists (_names/_parking_types/_accesses), and build
candidate record dicts lazily in query_parking_near_line instead of materializing
1.5M dicts + 1.5M shapely Point objects up front. road_class is the constant
"parking" so it is not stored per row.
Measured on the real /mnt/nav/osm-parking.db (1,489,054 usable rows):
RSS/worker ~950 MB -> ~570 MB (~40%), build ~11 s. Across 2 gunicorn workers that
is ~1.9 GB -> ~1.14 GB.
NOTE: this does NOT reach the ~250 MB originally targeted. The remaining cost is
the shapely STRtree itself: it permanently retains the input geometries
(tree.geometries len == row count), so the transient `del points` does not free
them. Attribution on the real DB: columns-only 137 MB, retained Point objects
+230 MB, STRtree index +110 MB. Reaching ~250 MB would require dropping the
shapely STRtree for a coordinate-only structure (e.g. scipy cKDTree over the
lon/lat arrays), which changes the line-buffer query into a per-vertex radius
query -- a behavior change beyond this fix-up's scope. Flagged for a follow-up.
Tests unchanged except one assertion (`len(idx.records) == idx.count`); full
offroute suite 82 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| config | ||
| deploy | ||
| scripts | ||
| services | ||
| shared | ||
| .gitignore | ||
| LICENSE | ||
| pyproject.toml | ||
| README.md | ||
navi-backend
Monorepo of small, single-responsibility HTTP services extracted from the recon
codebase as part of the recon ↔ Navi decoupling project. Each service owns a
slice of the /api/* surface that navi.echo6.co depends on, runs behind the
existing Caddy/Authentik edge, and is fronted by the navi.echo6.co nginx vhost.
See HANDOFF-recon-navi-decoupling-v3.md for the full plan. This repo is
extraction #1: navi-traffic.
Layout
navi-backend/
├── shared/ # cross-service helpers, imported by every service
│ ├── auth.py # get_user_id(req), require_auth decorator (Authentik header)
│ └── admin_info.py # build_info_response(), mask_key(), time_dependency()
├── services/
│ └── navi_traffic/ # extraction #1 — TomTom traffic tile proxy (:8421)
│ ├── app.py # Flask factory (create_app) + gunicorn entry
│ ├── traffic.py # /api/traffic/flow/<z>/<x>/<y>.png (ported from recon)
│ ├── admin.py # /api/admin/navi-traffic/info (§4.5 admin convention)
│ └── tests/
└── deploy/
├── systemd/navi-traffic.service
└── nginx/navi-traffic.conf.snippet
Service directories use an underscore (navi_traffic) so they're importable
Python packages; the service name stays navi-traffic (hyphen) in systemd,
nginx, and the admin-info service field.
Setup
Single workspace, single virtualenv:
python -m venv .venv
.venv/bin/pip install -e .
Test
.venv/bin/pytest services/navi_traffic/tests/ -v
Run (local)
TOMTOM_API_KEY=... .venv/bin/gunicorn 'services.navi_traffic.app:create_app()' \
--bind 127.0.0.1:8421 --workers 2
Run (local) — navi-geo (extraction #6)
.venv/bin/pytest services/navi_geo/tests/ -v
# All paths/URLs are env-overridable (see deploy/env/navi-geo.env.example).
# No secrets — landclass is HTTP-delegated to navi-landclass (:8424).
.venv/bin/gunicorn 'services.navi_geo.app:create_app()' \
--bind 127.0.0.1:8426 --workers 2
navi-geo serves /api/geocode, /api/reverse?lat=&lon=, and the reverse
enrichment bundle /api/reverse/<lat>/<lon> (Central's 9-key contract). All
public. The reverse bundle fans out to Photon, the SpatiaLite timezone DB,
navi-landclass (HTTP), and the planet-DEM PMTiles — each degrading to null
independently, never 5xx.
Run (local) — navi-admin (extraction #7)
.venv/bin/pytest services/navi_admin/tests/ -v
# No secrets — read-only HTTP fan-out over localhost (see
# deploy/env/navi-admin.env.example). Owns no DB.
.venv/bin/gunicorn 'services.navi_admin.app:create_app()' \
--bind 127.0.0.1:8427 --workers 2
navi-admin is the fleet admin front door: /api/admin/fleet fans out to every
navi-* service's localhost /api/admin/<svc>/info + recon's /api/health
(merged, never 5xx — failures land in errors[]); /api/admin/recon/info wraps
recon's health into the uniform shape; /api/admin/navi-admin/info self-describes.
All @require_auth. The per-service admin endpoints stay localhost-only; this is
the single edge-exposed admin surface (needs a Caddy @authed_api edit — see
deploy/caddy/navi-admin.caddy.notes.md).
Run (local) — navi-offroute (extraction #8)
.venv/bin/pytest services/navi_offroute/tests/ -v
# All paths/URLs env-overridable (deploy/env/navi-offroute.env.example).
# No secrets — PADUS via libpq peer-auth (dbname=padus). DEM via shared/dem.py.
# Needs osmium-tool on the host + scikit-image/rasterio in the venv.
.venv/bin/gunicorn 'services.navi_offroute.app:create_app()' \
--bind 127.0.0.1:8428 --workers 2 --timeout 130
navi-offroute serves POST /api/offroute (off-network effort-based routing —
in-Python least-cost path over a DEM/friction/barriers/trails/MVUM cost grid,
stitched to the road network via Valhalla) and GET /api/mvum (Motor Vehicle
Use Map road/trail access lookup). Both public. The ^~ /api/offroute nginx
block needs a long proxy_read_timeout (130s); routes can take ~2 min.
The admin-info convention (§4.5)
Every service exposes GET /api/admin/<service-name>/info, gated by require_auth,
returning a uniform shape: service, version (git SHA), port, config, env
(names + masked values), dependencies (upstream health checks), filesystem,
runtime (uptime / request count / last error). No aggregator — a future admin
panel fans out to each service in parallel.