navi/backend/pyproject.toml

42 lines
1.7 KiB
TOML
Raw Normal View History

[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[project]
name = "navi-backend"
version = "0.1.0"
description = "Echo6 navi-backend monorepo — Navi API services extracted from recon (decoupling project)."
requires-python = ">=3.10"
# Single workspace, single .venv. pytest is included so `pip install -e .` is the
# only setup step needed before running the test suite.
dependencies = [
"Flask>=3.0",
"gunicorn>=21",
"requests>=2.31",
Add navi-config service (extraction #2 PR-B) New services/navi_config/ on :8422, mirroring recon's /api/config contract: - config_route.py: GET /api/config -> jsonify(get_deployment_config()) with Cache-Control: public, max-age=300 (byte-for-byte recon's response). - config_loader.py: faithful port of recon lib/deployment_config.py. Reads RECON_PROFILE (default "home") and NAVI_CONFIG_PROFILES_DIR (default /opt/recon/config/profiles, so it serves the SAME files recon does during cutover). yaml.safe_load, module-level cache. Lazy load (vs recon's eager import-time load) so the module imports cleanly off-VM and a missing profile surfaces as HTTP 500 at request time rather than a failed import. - admin.py: /api/admin/navi-config/info per handoff §4.5, require_auth gated. env values (NAVI_CONFIG_PROFILES_DIR, RECON_PROFILE) are non-secret paths/ names, shown as-is (no mask_key); dependencies=[]; filesystem reports the active profile path + exists/readable. - app.py: create_app() factory mirroring navi_traffic, same metrics wiring; resets the loader cache per instance so each worker/test reloads fresh. Deploy artifacts: systemd unit (:8422) and an nginx snippet using `location ^~ /api/config` (the ^~ convention from extraction #1 so the asset .png/.css regex can't shadow it). No proxy_cache zone — the response is already cached in-process and via Cache-Control: max-age=300; emits a literal X-Cache-Status: BYPASS for parity with navi-traffic. Adds PyYAML>=6 to deps. Tests (services/navi_config/tests): 200 + parsed dict, Cache-Control header, RECON_PROFILE override, default=home, missing profile=500. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 08:25:06 -06:00
"PyYAML>=6",
Add navi-landclass service (extraction #4) (#4) New services/navi_landclass/ on :8424 — single blueprint, behavior-identical port of recon's lib/landclass.py + the /api/landclass handler. GET /api/landclass?lat=&lon= -> { lat, lon, classifications[], count, is_public, is_private, summary }; 400 on bad/out-of-range lat/lon. db.py: faithful port of recon's PostGIS module — lazy module-level psycopg2.pool.SimpleConnectionPool(minconn=1, maxconn=3) from PADUS_DB_* env; the ST_Intersects query on pad_units (antimeridian filter, acres-ordered, limit 10); all PAD-US code->label maps verbatim; graceful degradation (returns [] when PG is unreachable, never raises/500). Adds reset_pool() (create_app resets per worker) and probe_db() (SELECT 1) for admin health. No filesystem state — PostGIS is external. No DB-on-disk migration; only the 5 PADUS_DB_* env vars (PADUS_DB_PASSWORD is a real secret, masked in admin-info via mask_key; the other 4 shown plain). adds psycopg2-binary>=2.9. Decision — DROPPED the recon `has_landclass` profile-flag gate: the frontend already gates on its own has_landclass feature flag, and removing the cross-service config dependency keeps navi-landclass self-contained per the "only API" rule (the service's existence is the feature being available). navi-geo coupling (reverse-bundle needs landclass) — per Phase A, recommend Option B: navi-geo HTTP-calls /api/landclass and reads `.summary` (the endpoint already returns it); no shared module. Decided when #6 lands. Tests (8; recon had 2): point-with-coverage -> classification + decoded labels, ocean point -> empty, bad/missing/out-of-range lat/lon -> 400, PG down -> graceful 200 empty (not 500), format_summary unit. Full suite 46. Deploy: systemd unit (:8424) + nginx snippet (one ^~ /api/landclass block, no proxy_cache; /api/landclass is public so no Caddy edit — TIER 2 already routes through nginx since extraction #2). See ../recon_refactor/extraction-4-phase-a.md (which also corrects the handoff: /mnt/nav/padus/ is source GIS files, NOT a runtime path — this service has no /mnt/nav dependency, only PADUS_DB_* + PG network access). Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 12:08:58 -06:00
"psycopg2-binary>=2.9",
"pytest>=8",
Add navi-geo service (extraction #6) (#6) * Add navi-geo service (extraction #6) Faithful port of recon's geocode/reverse family to a new :8426 service: GET /api/geocode?q=&limit=&lat=&lon=&zoom= Photon-first ranked search GET /api/reverse?lat=&lon= reverse geocode (Photon) GET /api/reverse/<lat>/<lon> reverse enrichment bundle (Central) Ported modules: geocode.py (engine), netsyms.py (address SQLite), dem.py (planet-DEM reader), address_book.py (reader copy), and the three handlers + four bundle helpers from netsyms_api.py. All three routes public, behaviour- identical to recon. Behaviour-changing edges (both pre-decided in Phase A/B, called out in the PR): - landclass: in-process call replaced with HTTP GET to navi-landclass :8424, reading .summary (the same most-specific unit-name string). First navi→navi edge after landclass itself. - hardcoded paths/URLs → env vars (PHOTON_URL, NAVI_NETSYMS_DB, NAVI_TIMEZONE_DB, NAVI_DEM_PMTILES, NAVI_ADDRESS_BOOK_YAML, NAVI_LANDCLASS_URL); rerank trace log opt-in (NAVI_GEO_RERANK_TRACE_LOG, default off — recon always wrote /tmp). No secrets in this service: PADUS_DB_* disappears because landclass is HTTP- delegated (Phase A §10). Address book uses Option B (shared-file read), the same pattern navi-contacts already uses. Bundle 9-key contract preserved exactly (name/city/county/state/country/ postal_code/timezone/landclass/elevation_m), same null-on-component-failure semantics, same in-memory TTLCache(10_000, 86_400) per worker. Tests: 28 passing, 1 skipped (real timezone DB, off-box). Ported the 9 recon reverse-bundle tests + added the HTTP-landclass coupling tests + hermetic geocode reranker/intent-classifier tests (recon's geocode_test.py was a live smoke test). Adds usaddress/rapidfuzz/cachetools/shapely/numpy/Pillow/pmtiles to deps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * PR #6 review fixes 1. Rename geocode._setup_trace_logger → setup_trace_logger (public hook) 2. Hoist `import requests as http_requests` to module level in geo_route.py 3. Wire netsyms.health() into admin.py (enriches the netsyms filesystem entry with row_count/file_size_bytes/indexed_countries; no shared-builder change) 4. Fix misleading LANDCLASS_TIMEOUT_S comment (recon had no timeout) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: zvx-echo6 <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 20:29:22 -06:00
# navi-geo (extraction #6): geocode engine + reverse bundle.
"usaddress>=0.5", # address parsing / intent classification
"rapidfuzz>=3", # reranker fuzzy string scoring
"cachetools>=5", # reverse-bundle TTLCache
"shapely>=2", # timezone point-in-polygon
"numpy>=1.24", # planet-DEM tile decode
"Pillow>=10", # planet-DEM Terrarium WebP decode
"pmtiles>=3", # planet-DEM PMTiles reader
Add navi-offroute service (extraction #8 — final) (#10) * Add navi-offroute service (extraction #8 — the last one) Faithful port of recon's /api/offroute (POST) + /api/mvum (GET) and the runtime offroute modules into a new :8428 service. Closes the loop: after this, navi-frontend talks only to navi-backend. Ported: router.py (OffrouteRouter, EntryPointIndex, 4 route strategies, in-Python MCP_Geometric least-cost path, Valhalla integration, per-request osmium extract), mvum.py (MVUMReader over navi.db), cost.py, friction.py, trails.py, and barriers.py (runtime BarrierReader/WildernessReader only). NOT ported (per Phase A §3/§15): prototype.py (dead at runtime), barriers.py build_*_raster (offline GDB→raster prep). DEM imported from shared/dem.py (PR #9), not duplicated. Behaviour-faithful changes: hardcoded paths/URLs → env vars; the profile.offroute.* config (osm_pbf_path/postgis_dsn/densify_interval_m) → dedicated env vars (router drops deployment_config). Both routes public (no auth, matching recon). PADUS via libpq peer-auth DSN (dbname=padus) — NO secret. Owns no DB. 15 hermetic tests (offroute validation + mocked-router shape + close-always; fixture-SQLite MVUM roads/trails/fallback/null; admin auth + no-secrets + probe shape). Full suite 119 passed / 1 skipped. Adds scikit-image + rasterio. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * navi-offroute: PR #10 review cleanups (4 faithful-port deviations) 1. trails.py — drop recon-era "Run the Phase B rasterization script" reference from the not-found error (confusing in navi-offroute context). 2. friction.py — add FileNotFoundError-before-rasterio-open check to match barriers/trails consistency. 3. mvum.py — remove dead try/except shapely import + warnings.warn at 2 sites (shapely is a hard pyproject dep; the fallback was unreachable). 4. router.py — declare psutil in pyproject, drop the silent fallback; the MEMORY_LIMIT_GB safety check was silently disabled in prod. Adds test_friction_reader_raises_file_not_found_when_missing (16 navi-offroute tests; full suite 120 passed / 1 skipped). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: zvx-echo6 <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 23:30:43 -06:00
# navi-offroute (extraction #8): off-network router + readers.
"scikit-image>=0.22", # MCP_Geometric least-cost pathfinding (router.py)
"numba>=0.59", # navi-offroute anisotropic A* JIT pathfinder (astar.py)
Add navi-offroute service (extraction #8 — final) (#10) * Add navi-offroute service (extraction #8 — the last one) Faithful port of recon's /api/offroute (POST) + /api/mvum (GET) and the runtime offroute modules into a new :8428 service. Closes the loop: after this, navi-frontend talks only to navi-backend. Ported: router.py (OffrouteRouter, EntryPointIndex, 4 route strategies, in-Python MCP_Geometric least-cost path, Valhalla integration, per-request osmium extract), mvum.py (MVUMReader over navi.db), cost.py, friction.py, trails.py, and barriers.py (runtime BarrierReader/WildernessReader only). NOT ported (per Phase A §3/§15): prototype.py (dead at runtime), barriers.py build_*_raster (offline GDB→raster prep). DEM imported from shared/dem.py (PR #9), not duplicated. Behaviour-faithful changes: hardcoded paths/URLs → env vars; the profile.offroute.* config (osm_pbf_path/postgis_dsn/densify_interval_m) → dedicated env vars (router drops deployment_config). Both routes public (no auth, matching recon). PADUS via libpq peer-auth DSN (dbname=padus) — NO secret. Owns no DB. 15 hermetic tests (offroute validation + mocked-router shape + close-always; fixture-SQLite MVUM roads/trails/fallback/null; admin auth + no-secrets + probe shape). Full suite 119 passed / 1 skipped. Adds scikit-image + rasterio. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * navi-offroute: PR #10 review cleanups (4 faithful-port deviations) 1. trails.py — drop recon-era "Run the Phase B rasterization script" reference from the not-found error (confusing in navi-offroute context). 2. friction.py — add FileNotFoundError-before-rasterio-open check to match barriers/trails consistency. 3. mvum.py — remove dead try/except shapely import + warnings.warn at 2 sites (shapely is a hard pyproject dep; the fallback was unreachable). 4. router.py — declare psutil in pyproject, drop the silent fallback; the MEMORY_LIMIT_GB safety check was silently disabled in prod. Adds test_friction_reader_raises_file_not_found_when_missing (16 navi-offroute tests; full suite 120 passed / 1 skipped). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: zvx-echo6 <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 23:30:43 -06:00
"rasterio>=1.3", # barriers/wilderness/trails/friction raster readers
"psutil>=5.9", # MEMORY_LIMIT_GB enforcement in router.py
# scripts/overture_import.py: Overture Places ETL (S3 Parquet -> overture PG).
"duckdb>=1.5", # S3 Parquet read for the overture import script
]
[tool.setuptools.packages.find]
where = ["."]
include = ["shared*", "services*"]
namespaces = true
[tool.pytest.ini_options]
Add navi-admin service (extraction #7) (#7) * Add navi-admin service (extraction #7) Net-new fleet admin aggregator on :8427 — no port from recon (recon has no /api/admin route; Phase A §3). Three @require_auth routes: GET /api/admin/fleet fan-out to all 6 navi-* /api/admin/<svc>/info + recon /api/health, merged; never 5xx (failures land in errors[]) GET /api/admin/recon/info recon /api/health wrapped in the info shape GET /api/admin/navi-admin/info self-describe Fan-out forwards the caller's X-Authentik-Username so the @require_auth upstreams accept it; per-service admin endpoints stay localhost-only (this is the single edge-exposed admin surface). Service discovery: hardcoded list in fleet.py (Option B). No secrets, no DB. Deploy artifacts (NOT applied here): navi-admin.env.example, systemd unit, nginx ^~ /api/admin snippet, and deploy/caddy notes for the @authed_api edit (first Caddy change since #2). 12 hermetic tests (fleet happy-path, per-service timeout/500 → errors[], auth-header forwarding, recon-down degraded-not-5xx, self-info no-secrets, auth-required). Full monorepo suite green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * PR #7 review fixes 1. Symmetric degraded-entry handling in fleet.build_fleet — every probed service now appears in `services` with a uniform degraded dict on failure (matches recon's existing pattern), AND in errors[]. Operators see "everything I tried + which broke" consistently. 2. Catch ValueError specifically in _get_json — non-JSON 200 responses now surface as `error: 'invalid JSON'` instead of opaque 'ValueError'. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * PR #7 review fixes (round 2) 1. Unified degraded shape: wrap_recon_health calls _degraded_entry on failure — no more runtime.status vs runtime.recon_status asymmetry. Every probed service has the same shape on failure (runtime.status == 'unreachable'). recon-specific runtime fields (recon_status/recon_uptime/pipeline) remain only on the success path. 2. DRY'd git short-SHA helper into shared/git_sha.py — was duplicated in 7 service app.py files + fleet.recon_git_sha. One implementation, one place to fix when behavior changes. Adds shared/tests (testpaths now includes "shared"). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: zvx-echo6 <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 21:21:00 -06:00
testpaths = ["services", "shared"]