2026-05-21 22:11:10 -06:00
|
|
|
[build-system]
|
|
|
|
|
requires = ["setuptools>=68"]
|
|
|
|
|
build-backend = "setuptools.build_meta"
|
|
|
|
|
|
|
|
|
|
[project]
|
|
|
|
|
name = "navi-backend"
|
|
|
|
|
version = "0.1.0"
|
|
|
|
|
description = "Echo6 navi-backend monorepo — Navi API services extracted from recon (decoupling project)."
|
|
|
|
|
requires-python = ">=3.10"
|
|
|
|
|
# Single workspace, single .venv. pytest is included so `pip install -e .` is the
|
|
|
|
|
# only setup step needed before running the test suite.
|
|
|
|
|
dependencies = [
|
|
|
|
|
"Flask>=3.0",
|
|
|
|
|
"gunicorn>=21",
|
|
|
|
|
"requests>=2.31",
|
Add navi-config service (extraction #2 PR-B)
New services/navi_config/ on :8422, mirroring recon's /api/config contract:
- config_route.py: GET /api/config -> jsonify(get_deployment_config())
with Cache-Control: public, max-age=300 (byte-for-byte recon's response).
- config_loader.py: faithful port of recon lib/deployment_config.py. Reads
RECON_PROFILE (default "home") and NAVI_CONFIG_PROFILES_DIR (default
/opt/recon/config/profiles, so it serves the SAME files recon does during
cutover). yaml.safe_load, module-level cache. Lazy load (vs recon's eager
import-time load) so the module imports cleanly off-VM and a missing
profile surfaces as HTTP 500 at request time rather than a failed import.
- admin.py: /api/admin/navi-config/info per handoff §4.5, require_auth gated.
env values (NAVI_CONFIG_PROFILES_DIR, RECON_PROFILE) are non-secret paths/
names, shown as-is (no mask_key); dependencies=[]; filesystem reports the
active profile path + exists/readable.
- app.py: create_app() factory mirroring navi_traffic, same metrics wiring;
resets the loader cache per instance so each worker/test reloads fresh.
Deploy artifacts: systemd unit (:8422) and an nginx snippet using
`location ^~ /api/config` (the ^~ convention from extraction #1 so the asset
.png/.css regex can't shadow it). No proxy_cache zone — the response is
already cached in-process and via Cache-Control: max-age=300; emits a literal
X-Cache-Status: BYPASS for parity with navi-traffic.
Adds PyYAML>=6 to deps. Tests (services/navi_config/tests): 200 + parsed dict,
Cache-Control header, RECON_PROFILE override, default=home, missing profile=500.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 08:25:06 -06:00
|
|
|
"PyYAML>=6",
|
Add navi-landclass service (extraction #4) (#4)
New services/navi_landclass/ on :8424 — single blueprint, behavior-identical
port of recon's lib/landclass.py + the /api/landclass handler.
GET /api/landclass?lat=&lon= -> { lat, lon, classifications[], count,
is_public, is_private, summary }; 400 on bad/out-of-range lat/lon.
db.py: faithful port of recon's PostGIS module — lazy module-level
psycopg2.pool.SimpleConnectionPool(minconn=1, maxconn=3) from PADUS_DB_* env;
the ST_Intersects query on pad_units (antimeridian filter, acres-ordered,
limit 10); all PAD-US code->label maps verbatim; graceful degradation
(returns [] when PG is unreachable, never raises/500). Adds reset_pool()
(create_app resets per worker) and probe_db() (SELECT 1) for admin health.
No filesystem state — PostGIS is external. No DB-on-disk migration; only the
5 PADUS_DB_* env vars (PADUS_DB_PASSWORD is a real secret, masked in
admin-info via mask_key; the other 4 shown plain). adds psycopg2-binary>=2.9.
Decision — DROPPED the recon `has_landclass` profile-flag gate: the frontend
already gates on its own has_landclass feature flag, and removing the
cross-service config dependency keeps navi-landclass self-contained per the
"only API" rule (the service's existence is the feature being available).
navi-geo coupling (reverse-bundle needs landclass) — per Phase A, recommend
Option B: navi-geo HTTP-calls /api/landclass and reads `.summary` (the
endpoint already returns it); no shared module. Decided when #6 lands.
Tests (8; recon had 2): point-with-coverage -> classification + decoded
labels, ocean point -> empty, bad/missing/out-of-range lat/lon -> 400, PG
down -> graceful 200 empty (not 500), format_summary unit. Full suite 46.
Deploy: systemd unit (:8424) + nginx snippet (one ^~ /api/landclass block, no
proxy_cache; /api/landclass is public so no Caddy edit — TIER 2 already
routes through nginx since extraction #2).
See ../recon_refactor/extraction-4-phase-a.md (which also corrects the handoff:
/mnt/nav/padus/ is source GIS files, NOT a runtime path — this service has no
/mnt/nav dependency, only PADUS_DB_* + PG network access).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 12:08:58 -06:00
|
|
|
"psycopg2-binary>=2.9",
|
2026-05-21 22:11:10 -06:00
|
|
|
"pytest>=8",
|
Add navi-geo service (extraction #6) (#6)
* Add navi-geo service (extraction #6)
Faithful port of recon's geocode/reverse family to a new :8426 service:
GET /api/geocode?q=&limit=&lat=&lon=&zoom= Photon-first ranked search
GET /api/reverse?lat=&lon= reverse geocode (Photon)
GET /api/reverse/<lat>/<lon> reverse enrichment bundle (Central)
Ported modules: geocode.py (engine), netsyms.py (address SQLite), dem.py
(planet-DEM reader), address_book.py (reader copy), and the three handlers +
four bundle helpers from netsyms_api.py. All three routes public, behaviour-
identical to recon.
Behaviour-changing edges (both pre-decided in Phase A/B, called out in the PR):
- landclass: in-process call replaced with HTTP GET to navi-landclass :8424,
reading .summary (the same most-specific unit-name string). First navi→navi
edge after landclass itself.
- hardcoded paths/URLs → env vars (PHOTON_URL, NAVI_NETSYMS_DB,
NAVI_TIMEZONE_DB, NAVI_DEM_PMTILES, NAVI_ADDRESS_BOOK_YAML,
NAVI_LANDCLASS_URL); rerank trace log opt-in (NAVI_GEO_RERANK_TRACE_LOG,
default off — recon always wrote /tmp).
No secrets in this service: PADUS_DB_* disappears because landclass is HTTP-
delegated (Phase A §10). Address book uses Option B (shared-file read), the
same pattern navi-contacts already uses.
Bundle 9-key contract preserved exactly (name/city/county/state/country/
postal_code/timezone/landclass/elevation_m), same null-on-component-failure
semantics, same in-memory TTLCache(10_000, 86_400) per worker.
Tests: 28 passing, 1 skipped (real timezone DB, off-box). Ported the 9 recon
reverse-bundle tests + added the HTTP-landclass coupling tests + hermetic
geocode reranker/intent-classifier tests (recon's geocode_test.py was a live
smoke test). Adds usaddress/rapidfuzz/cachetools/shapely/numpy/Pillow/pmtiles
to deps.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR #6 review fixes
1. Rename geocode._setup_trace_logger → setup_trace_logger (public hook)
2. Hoist `import requests as http_requests` to module level in geo_route.py
3. Wire netsyms.health() into admin.py (enriches the netsyms filesystem entry
with row_count/file_size_bytes/indexed_countries; no shared-builder change)
4. Fix misleading LANDCLASS_TIMEOUT_S comment (recon had no timeout)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: zvx-echo6 <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 20:29:22 -06:00
|
|
|
# navi-geo (extraction #6): geocode engine + reverse bundle.
|
|
|
|
|
"usaddress>=0.5", # address parsing / intent classification
|
|
|
|
|
"rapidfuzz>=3", # reranker fuzzy string scoring
|
|
|
|
|
"cachetools>=5", # reverse-bundle TTLCache
|
|
|
|
|
"shapely>=2", # timezone point-in-polygon
|
|
|
|
|
"numpy>=1.24", # planet-DEM tile decode
|
|
|
|
|
"Pillow>=10", # planet-DEM Terrarium WebP decode
|
|
|
|
|
"pmtiles>=3", # planet-DEM PMTiles reader
|
Add navi-offroute service (extraction #8 — final) (#10)
* Add navi-offroute service (extraction #8 — the last one)
Faithful port of recon's /api/offroute (POST) + /api/mvum (GET) and the
runtime offroute modules into a new :8428 service. Closes the loop: after
this, navi-frontend talks only to navi-backend.
Ported: router.py (OffrouteRouter, EntryPointIndex, 4 route strategies,
in-Python MCP_Geometric least-cost path, Valhalla integration, per-request
osmium extract), mvum.py (MVUMReader over navi.db), cost.py, friction.py,
trails.py, and barriers.py (runtime BarrierReader/WildernessReader only).
NOT ported (per Phase A §3/§15): prototype.py (dead at runtime), barriers.py
build_*_raster (offline GDB→raster prep). DEM imported from shared/dem.py
(PR #9), not duplicated.
Behaviour-faithful changes: hardcoded paths/URLs → env vars; the
profile.offroute.* config (osm_pbf_path/postgis_dsn/densify_interval_m) →
dedicated env vars (router drops deployment_config). Both routes public (no
auth, matching recon). PADUS via libpq peer-auth DSN (dbname=padus) — NO
secret. Owns no DB.
15 hermetic tests (offroute validation + mocked-router shape + close-always;
fixture-SQLite MVUM roads/trails/fallback/null; admin auth + no-secrets +
probe shape). Full suite 119 passed / 1 skipped. Adds scikit-image + rasterio.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* navi-offroute: PR #10 review cleanups (4 faithful-port deviations)
1. trails.py — drop recon-era "Run the Phase B rasterization script"
reference from the not-found error (confusing in navi-offroute context).
2. friction.py — add FileNotFoundError-before-rasterio-open check to
match barriers/trails consistency.
3. mvum.py — remove dead try/except shapely import + warnings.warn at
2 sites (shapely is a hard pyproject dep; the fallback was unreachable).
4. router.py — declare psutil in pyproject, drop the silent fallback;
the MEMORY_LIMIT_GB safety check was silently disabled in prod.
Adds test_friction_reader_raises_file_not_found_when_missing (16 navi-offroute
tests; full suite 120 passed / 1 skipped).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: zvx-echo6 <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 23:30:43 -06:00
|
|
|
# navi-offroute (extraction #8): off-network router + readers.
|
|
|
|
|
"scikit-image>=0.22", # MCP_Geometric least-cost pathfinding (router.py)
|
feat(offroute): numba A* with anisotropic Tobler, exponentially-inflated cost grid (combined #17+#18)
Replace MCP_Geometric in _pathfind_wilderness with a numba-jit anisotropic A* (new
astar.py): signed-slope speed (climbing != descending; tobler peaks at -0.05), hard
cliff, per-edge avg context multiplier, trail-takes-both via 256-entry lookup, per-edge
barriers (strict/pragmatic/emergency), multi-goal A* (first popped wins) with admissible
distance/base-speed heuristic. New compute_cost_multiplier_grid (slope-free context
multiplier) + exponential inflation (sigma=1.8; inf->HARD=50*p95 for blur, inf re-imposed).
numba>=0.59 added (numba 0.65.1).
fix: wilderness leg is always foot effort; mode parameter reserved for future flexibility.
_pathfind_wilderness keeps the mode param (threaded from _route_A/B/C) but hardcodes
cost_mode=foot for the cost grid, trail friction, speed function, base speed, and max
grade. Off-trail math for MTB/ATV/vehicle is not well-grounded and real-world wilderness
traversal is foot regardless (push the bike, walk past where the vehicle stops). User mode
still drives entry-point eligibility (query_radius highway filter) and Valhalla network
costing. Matches the original pre-#17 design: wilderness ALWAYS uses foot.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 08:13:30 +00:00
|
|
|
"numba>=0.59", # navi-offroute anisotropic A* JIT pathfinder (astar.py)
|
Add navi-offroute service (extraction #8 — final) (#10)
* Add navi-offroute service (extraction #8 — the last one)
Faithful port of recon's /api/offroute (POST) + /api/mvum (GET) and the
runtime offroute modules into a new :8428 service. Closes the loop: after
this, navi-frontend talks only to navi-backend.
Ported: router.py (OffrouteRouter, EntryPointIndex, 4 route strategies,
in-Python MCP_Geometric least-cost path, Valhalla integration, per-request
osmium extract), mvum.py (MVUMReader over navi.db), cost.py, friction.py,
trails.py, and barriers.py (runtime BarrierReader/WildernessReader only).
NOT ported (per Phase A §3/§15): prototype.py (dead at runtime), barriers.py
build_*_raster (offline GDB→raster prep). DEM imported from shared/dem.py
(PR #9), not duplicated.
Behaviour-faithful changes: hardcoded paths/URLs → env vars; the
profile.offroute.* config (osm_pbf_path/postgis_dsn/densify_interval_m) →
dedicated env vars (router drops deployment_config). Both routes public (no
auth, matching recon). PADUS via libpq peer-auth DSN (dbname=padus) — NO
secret. Owns no DB.
15 hermetic tests (offroute validation + mocked-router shape + close-always;
fixture-SQLite MVUM roads/trails/fallback/null; admin auth + no-secrets +
probe shape). Full suite 119 passed / 1 skipped. Adds scikit-image + rasterio.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* navi-offroute: PR #10 review cleanups (4 faithful-port deviations)
1. trails.py — drop recon-era "Run the Phase B rasterization script"
reference from the not-found error (confusing in navi-offroute context).
2. friction.py — add FileNotFoundError-before-rasterio-open check to
match barriers/trails consistency.
3. mvum.py — remove dead try/except shapely import + warnings.warn at
2 sites (shapely is a hard pyproject dep; the fallback was unreachable).
4. router.py — declare psutil in pyproject, drop the silent fallback;
the MEMORY_LIMIT_GB safety check was silently disabled in prod.
Adds test_friction_reader_raises_file_not_found_when_missing (16 navi-offroute
tests; full suite 120 passed / 1 skipped).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: zvx-echo6 <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 23:30:43 -06:00
|
|
|
"rasterio>=1.3", # barriers/wilderness/trails/friction raster readers
|
|
|
|
|
"psutil>=5.9", # MEMORY_LIMIT_GB enforcement in router.py
|
2026-05-23 13:49:32 -06:00
|
|
|
# scripts/overture_import.py: Overture Places ETL (S3 Parquet -> overture PG).
|
|
|
|
|
"duckdb>=1.5", # S3 Parquet read for the overture import script
|
2026-05-21 22:11:10 -06:00
|
|
|
]
|
|
|
|
|
|
|
|
|
|
[tool.setuptools.packages.find]
|
|
|
|
|
where = ["."]
|
|
|
|
|
include = ["shared*", "services*"]
|
|
|
|
|
namespaces = true
|
|
|
|
|
|
|
|
|
|
[tool.pytest.ini_options]
|
Add navi-admin service (extraction #7) (#7)
* Add navi-admin service (extraction #7)
Net-new fleet admin aggregator on :8427 — no port from recon (recon has no
/api/admin route; Phase A §3). Three @require_auth routes:
GET /api/admin/fleet fan-out to all 6 navi-* /api/admin/<svc>/info
+ recon /api/health, merged; never 5xx
(failures land in errors[])
GET /api/admin/recon/info recon /api/health wrapped in the info shape
GET /api/admin/navi-admin/info self-describe
Fan-out forwards the caller's X-Authentik-Username so the @require_auth
upstreams accept it; per-service admin endpoints stay localhost-only (this is
the single edge-exposed admin surface). Service discovery: hardcoded list in
fleet.py (Option B). No secrets, no DB.
Deploy artifacts (NOT applied here): navi-admin.env.example, systemd unit,
nginx ^~ /api/admin snippet, and deploy/caddy notes for the @authed_api edit
(first Caddy change since #2).
12 hermetic tests (fleet happy-path, per-service timeout/500 → errors[],
auth-header forwarding, recon-down degraded-not-5xx, self-info no-secrets,
auth-required). Full monorepo suite green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR #7 review fixes
1. Symmetric degraded-entry handling in fleet.build_fleet — every
probed service now appears in `services` with a uniform degraded
dict on failure (matches recon's existing pattern), AND in errors[].
Operators see "everything I tried + which broke" consistently.
2. Catch ValueError specifically in _get_json — non-JSON 200 responses
now surface as `error: 'invalid JSON'` instead of opaque 'ValueError'.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR #7 review fixes (round 2)
1. Unified degraded shape: wrap_recon_health calls _degraded_entry on
failure — no more runtime.status vs runtime.recon_status asymmetry.
Every probed service has the same shape on failure
(runtime.status == 'unreachable'). recon-specific runtime fields
(recon_status/recon_uptime/pipeline) remain only on the success path.
2. DRY'd git short-SHA helper into shared/git_sha.py — was duplicated in
7 service app.py files + fleet.recon_git_sha. One implementation,
one place to fix when behavior changes. Adds shared/tests (testpaths
now includes "shared").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: zvx-echo6 <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 21:21:00 -06:00
|
|
|
testpaths = ["services", "shared"]
|