mirror of
https://github.com/zvx-echo6/navi.git
synced 2026-08-26 17:31:37 +00:00
* Add navi-geo service (extraction #6) Faithful port of recon's geocode/reverse family to a new :8426 service: GET /api/geocode?q=&limit=&lat=&lon=&zoom= Photon-first ranked search GET /api/reverse?lat=&lon= reverse geocode (Photon) GET /api/reverse/<lat>/<lon> reverse enrichment bundle (Central) Ported modules: geocode.py (engine), netsyms.py (address SQLite), dem.py (planet-DEM reader), address_book.py (reader copy), and the three handlers + four bundle helpers from netsyms_api.py. All three routes public, behaviour- identical to recon. Behaviour-changing edges (both pre-decided in Phase A/B, called out in the PR): - landclass: in-process call replaced with HTTP GET to navi-landclass :8424, reading .summary (the same most-specific unit-name string). First navi→navi edge after landclass itself. - hardcoded paths/URLs → env vars (PHOTON_URL, NAVI_NETSYMS_DB, NAVI_TIMEZONE_DB, NAVI_DEM_PMTILES, NAVI_ADDRESS_BOOK_YAML, NAVI_LANDCLASS_URL); rerank trace log opt-in (NAVI_GEO_RERANK_TRACE_LOG, default off — recon always wrote /tmp). No secrets in this service: PADUS_DB_* disappears because landclass is HTTP- delegated (Phase A §10). Address book uses Option B (shared-file read), the same pattern navi-contacts already uses. Bundle 9-key contract preserved exactly (name/city/county/state/country/ postal_code/timezone/landclass/elevation_m), same null-on-component-failure semantics, same in-memory TTLCache(10_000, 86_400) per worker. Tests: 28 passing, 1 skipped (real timezone DB, off-box). Ported the 9 recon reverse-bundle tests + added the HTTP-landclass coupling tests + hermetic geocode reranker/intent-classifier tests (recon's geocode_test.py was a live smoke test). Adds usaddress/rapidfuzz/cachetools/shapely/numpy/Pillow/pmtiles to deps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * PR #6 review fixes 1. Rename geocode._setup_trace_logger → setup_trace_logger (public hook) 2. Hoist `import requests as http_requests` to module level in geo_route.py 3. Wire netsyms.health() into admin.py (enriches the netsyms filesystem entry with row_count/file_size_bytes/indexed_countries; no shared-builder change) 4. Fix misleading LANDCLASS_TIMEOUT_S comment (recon had no timeout) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: zvx-echo6 <mj@k7zvx.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
GitHub
parent
34df49a9cd
commit
15cad0abf4
17 changed files with 2531 additions and 0 deletions
|
|
@ -52,6 +52,23 @@ TOMTOM_API_KEY=... .venv/bin/gunicorn 'services.navi_traffic.app:create_app()' \
|
|||
--bind 127.0.0.1:8421 --workers 2
|
||||
```
|
||||
|
||||
## Run (local) — navi-geo (extraction #6)
|
||||
|
||||
```bash
|
||||
.venv/bin/pytest services/navi_geo/tests/ -v
|
||||
|
||||
# All paths/URLs are env-overridable (see deploy/env/navi-geo.env.example).
|
||||
# No secrets — landclass is HTTP-delegated to navi-landclass (:8424).
|
||||
.venv/bin/gunicorn 'services.navi_geo.app:create_app()' \
|
||||
--bind 127.0.0.1:8426 --workers 2
|
||||
```
|
||||
|
||||
`navi-geo` serves `/api/geocode`, `/api/reverse?lat=&lon=`, and the reverse
|
||||
enrichment bundle `/api/reverse/<lat>/<lon>` (Central's 9-key contract). All
|
||||
public. The reverse bundle fans out to Photon, the SpatiaLite timezone DB,
|
||||
navi-landclass (HTTP), and the planet-DEM PMTiles — each degrading to `null`
|
||||
independently, never 5xx.
|
||||
|
||||
## The admin-info convention (§4.5)
|
||||
|
||||
Every service exposes `GET /api/admin/<service-name>/info`, gated by `require_auth`,
|
||||
|
|
|
|||
29
backend/deploy/env/navi-geo.env.example
vendored
Normal file
29
backend/deploy/env/navi-geo.env.example
vendored
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
# navi-geo — /etc/navi-backend/navi-geo.env
|
||||
# Geocode + reverse + reverse-bundle API (extraction #6). NO SECRETS in this
|
||||
# service: landclass is HTTP-delegated to navi-landclass, so PADUS_DB_* — the
|
||||
# only secret in recon's geocode/reverse path — disappears entirely (Phase A §10).
|
||||
|
||||
# Photon geocoder (local). Single source of truth for the default lives in
|
||||
# services/navi_geo/geocode.py (DEFAULT_PHOTON_URL).
|
||||
PHOTON_URL=http://localhost:2322
|
||||
|
||||
# navi-landclass HTTP coupling (the reverse bundle's `landclass` field).
|
||||
NAVI_LANDCLASS_URL=http://127.0.0.1:8424
|
||||
|
||||
# Big external read-only data files (all stay external per the data-ownership
|
||||
# rule — Phase A §11: ~35 GB / ~123 MB / ~657 GB respectively).
|
||||
NAVI_NETSYMS_DB=/mnt/nav/addresses/AddressDatabase2025.sqlite
|
||||
NAVI_TIMEZONE_DB=/mnt/nav/sources/timezones.sqlite
|
||||
NAVI_DEM_PMTILES=/mnt/nas/nav/planet-dem.pmtiles
|
||||
|
||||
# Address book (Phase B Option B: shared-file read; same file/var navi-contacts
|
||||
# uses — navi-contacts owns writes/UI, navi-geo only reads).
|
||||
NAVI_ADDRESS_BOOK_YAML=/home/zvx/projects/repos/navi-backend/config/address_book.yaml
|
||||
|
||||
# OPTIONAL: rerank audit trace (DEBUG). UNSET = no FileHandler, no file written
|
||||
# (Phase B locked decision #3). Set to a writable path to enable, recon-style.
|
||||
# NAVI_GEO_RERANK_TRACE_LOG=/var/log/navi-backend/navi-geo-rerank.log
|
||||
|
||||
# NOTE: recon's deployment_config (RECON_PROFILE / profiles dir) is intentionally
|
||||
# NOT wired here. Phase A §6 confirmed the geocode/reverse/bundle paths read NO
|
||||
# profile feature flags, so navi-geo carries no profile config (unlike navi-places).
|
||||
41
backend/deploy/nginx/navi-geo.conf.snippet
Normal file
41
backend/deploy/nginx/navi-geo.conf.snippet
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
# =============================================================================
|
||||
# navi-geo — nginx integration for the navi.echo6.co vhost
|
||||
#
|
||||
# TWO blocks. Add INSIDE the existing
|
||||
# server { server_name navi.echo6.co; ... }
|
||||
# block, BEFORE the existing `location /api/ { ... }` block.
|
||||
#
|
||||
# `^~` (lesson from extraction #1) makes nginx skip regex evaluation so the
|
||||
# asset-regex can't shadow these. The no-trailing-slash prefixes are deliberate:
|
||||
#
|
||||
# ^~ /api/geocode → /api/geocode?...
|
||||
# ^~ /api/reverse → BOTH /api/reverse?lat=&lon= AND /api/reverse/<lat>/<lon>
|
||||
#
|
||||
# The single `^~ /api/reverse` prefix catching both reverse forms is the whole
|
||||
# point — the query-string reverse (navi frontend) and the path-param reverse
|
||||
# bundle (Central) are different routes on the same service.
|
||||
#
|
||||
# Both endpoints are PUBLIC (no forward_auth) — TIER 2 @public_api in Caddy,
|
||||
# already routed through nginx :8440 since extraction #2. So NO Caddy edit.
|
||||
#
|
||||
# No proxy_cache: geocode/reverse vary per query and the reverse bundle has its
|
||||
# own in-memory TTLCache. `X-Cache-Status: BYPASS` for parity with the other
|
||||
# navi-* blocks.
|
||||
# -----------------------------------------------------------------------------
|
||||
location ^~ /api/geocode {
|
||||
proxy_pass http://127.0.0.1:8426;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Authentik-Username $http_x_authentik_username;
|
||||
proxy_read_timeout 15s;
|
||||
add_header X-Cache-Status BYPASS;
|
||||
}
|
||||
|
||||
location ^~ /api/reverse {
|
||||
proxy_pass http://127.0.0.1:8426;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Authentik-Username $http_x_authentik_username;
|
||||
proxy_read_timeout 15s;
|
||||
add_header X-Cache-Status BYPASS;
|
||||
}
|
||||
15
backend/deploy/systemd/navi-geo.service
Normal file
15
backend/deploy/systemd/navi-geo.service
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
[Unit]
|
||||
Description=navi-geo — geocode + reverse + reverse-bundle API (Echo6 navi-backend, extraction #6)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
User=zvx
|
||||
WorkingDirectory=/home/zvx/projects/repos/navi-backend
|
||||
EnvironmentFile=/etc/navi-backend/navi-geo.env
|
||||
ExecStart=/home/zvx/projects/repos/navi-backend/.venv/bin/gunicorn 'services.navi_geo.app:create_app()' --bind 127.0.0.1:8426 --workers 2
|
||||
Restart=on-failure
|
||||
RestartSec=2
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
@ -16,6 +16,14 @@ dependencies = [
|
|||
"PyYAML>=6",
|
||||
"psycopg2-binary>=2.9",
|
||||
"pytest>=8",
|
||||
# navi-geo (extraction #6): geocode engine + reverse bundle.
|
||||
"usaddress>=0.5", # address parsing / intent classification
|
||||
"rapidfuzz>=3", # reranker fuzzy string scoring
|
||||
"cachetools>=5", # reverse-bundle TTLCache
|
||||
"shapely>=2", # timezone point-in-polygon
|
||||
"numpy>=1.24", # planet-DEM tile decode
|
||||
"Pillow>=10", # planet-DEM Terrarium WebP decode
|
||||
"pmtiles>=3", # planet-DEM PMTiles reader
|
||||
]
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
|
|
|
|||
0
backend/services/navi_geo/__init__.py
Normal file
0
backend/services/navi_geo/__init__.py
Normal file
176
backend/services/navi_geo/address_book.py
Normal file
176
backend/services/navi_geo/address_book.py
Normal file
|
|
@ -0,0 +1,176 @@
|
|||
"""Address Book — YAML-backed saved-location lookup (navi-geo's reader copy).
|
||||
|
||||
Behavior-identical port of recon's ``lib/address_book.py`` — the same copy
|
||||
navi-contacts ships. navi-geo only *reads* this file (for the geocode nickname
|
||||
short-circuit + the 75 m result annotation); navi-contacts owns the writes/UI.
|
||||
Both read the shared YAML via ``NAVI_ADDRESS_BOOK_YAML`` (Phase-B address-book
|
||||
decision: Option B / shared-file read — see the PR description).
|
||||
|
||||
Named locations (home, work, etc.) with fuzzy matching over name + aliases +
|
||||
partial address. Hot-reloads when the YAML's mtime changes.
|
||||
"""
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
|
||||
import yaml
|
||||
|
||||
logger = logging.getLogger('navi_geo.address_book')
|
||||
|
||||
DEFAULT_CONFIG_PATH = '/home/zvx/projects/repos/navi-backend/config/address_book.yaml'
|
||||
|
||||
_lock = threading.Lock()
|
||||
_entries: list[dict] = []
|
||||
_mtime: float = 0.0
|
||||
_loaded_path: str | None = None
|
||||
|
||||
|
||||
def _config_path():
|
||||
return os.environ.get('NAVI_ADDRESS_BOOK_YAML', DEFAULT_CONFIG_PATH)
|
||||
|
||||
|
||||
def reset_cache():
|
||||
"""Drop cached entries so the next access reloads (env/path may have changed)."""
|
||||
global _entries, _mtime, _loaded_path
|
||||
with _lock:
|
||||
_entries = []
|
||||
_mtime = 0.0
|
||||
_loaded_path = None
|
||||
|
||||
|
||||
def _reload_if_changed():
|
||||
"""Reload the YAML file if its mtime (or path) has changed."""
|
||||
global _entries, _mtime, _loaded_path
|
||||
path = _config_path()
|
||||
try:
|
||||
st = os.stat(path)
|
||||
except FileNotFoundError:
|
||||
logger.warning("Address book not found: %s", path)
|
||||
_entries = []
|
||||
_mtime = 0.0
|
||||
_loaded_path = path
|
||||
return
|
||||
|
||||
if st.st_mtime == _mtime and path == _loaded_path:
|
||||
return
|
||||
|
||||
with _lock:
|
||||
# Double-check after acquiring lock
|
||||
try:
|
||||
st = os.stat(path)
|
||||
except FileNotFoundError:
|
||||
_entries = []
|
||||
_mtime = 0.0
|
||||
_loaded_path = path
|
||||
return
|
||||
if st.st_mtime == _mtime and path == _loaded_path:
|
||||
return
|
||||
|
||||
with open(path, 'r') as f:
|
||||
data = yaml.safe_load(f) or {}
|
||||
|
||||
raw = data.get('entries', [])
|
||||
loaded = []
|
||||
for entry in raw:
|
||||
# Normalise aliases to lowercase for matching
|
||||
aliases = [a.lower() for a in entry.get('aliases', [])]
|
||||
loaded.append({
|
||||
'id': entry.get('id', ''),
|
||||
'name': entry.get('name', ''),
|
||||
'aliases': aliases,
|
||||
'address': entry.get('address', ''),
|
||||
'lat': entry.get('lat'),
|
||||
'lon': entry.get('lon'),
|
||||
'tags': entry.get('tags', []),
|
||||
})
|
||||
_entries = loaded
|
||||
_mtime = st.st_mtime
|
||||
_loaded_path = path
|
||||
logger.info("Address book loaded: %d entries from %s", len(_entries), path)
|
||||
|
||||
|
||||
def load():
|
||||
"""Ensure the address book is loaded (and refreshed if the file changed)."""
|
||||
_reload_if_changed()
|
||||
return _entries
|
||||
|
||||
|
||||
def _normalize(text: str) -> str:
|
||||
"""Lowercase, strip, remove commas, collapse whitespace."""
|
||||
t = text.strip().lower()
|
||||
t = t.replace(',', ' ')
|
||||
return ' '.join(t.split())
|
||||
|
||||
|
||||
def lookup(query: str):
|
||||
"""
|
||||
Look up a query against name and aliases.
|
||||
|
||||
Returns dict with the matching entry plus a 'confidence' field:
|
||||
- "exact": full name/alias match, OR query starts with alias + word boundary
|
||||
- "partial": alias starts with query + word boundary, or alias appears
|
||||
as a contiguous token sequence inside the query
|
||||
- None if no match
|
||||
|
||||
Matching order (first exact wins, else first partial):
|
||||
1. normalized(query) == normalized(name or alias) → exact
|
||||
2. normalized(query) starts with normalized(alias) + " " → exact
|
||||
3. normalized(alias) starts with normalized(query) + " " → partial
|
||||
4. normalized(alias) is a contiguous token sub-sequence → partial
|
||||
"""
|
||||
_reload_if_changed()
|
||||
q = _normalize(query)
|
||||
if not q:
|
||||
return None
|
||||
|
||||
first_exact = None
|
||||
first_partial = None
|
||||
|
||||
for entry in _entries:
|
||||
norm_name = _normalize(entry['name'])
|
||||
check_aliases = [_normalize(a) for a in entry.get('aliases', [])]
|
||||
all_forms = [norm_name] + check_aliases
|
||||
|
||||
for form in all_forms:
|
||||
if not form:
|
||||
continue
|
||||
|
||||
# Rule 1: exact match
|
||||
if q == form:
|
||||
return {**entry, 'confidence': 'exact'}
|
||||
|
||||
# Rule 2: query starts with alias + word boundary
|
||||
if q.startswith(form + ' '):
|
||||
if first_exact is None:
|
||||
first_exact = entry
|
||||
continue
|
||||
|
||||
# Rule 3: alias starts with query (user still typing)
|
||||
if form.startswith(q) and len(q) < len(form):
|
||||
if first_partial is None:
|
||||
first_partial = entry
|
||||
continue
|
||||
|
||||
# Rule 4: alias is contiguous token sub-sequence in query
|
||||
# Build regex: token1\s+token2\s+...tokenN
|
||||
tokens = form.split()
|
||||
if len(tokens) >= 1:
|
||||
pattern = r'(?:^|\s)' + r'\s+'.join(re.escape(t) for t in tokens) + r'(?:\s|$)'
|
||||
if re.search(pattern, q):
|
||||
if first_partial is None:
|
||||
first_partial = entry
|
||||
|
||||
if first_exact is not None:
|
||||
return {**first_exact, 'confidence': 'exact'}
|
||||
|
||||
if first_partial is not None:
|
||||
return {**first_partial, 'confidence': 'partial'}
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def list_all():
|
||||
"""Return all address book entries."""
|
||||
_reload_if_changed()
|
||||
return list(_entries)
|
||||
137
backend/services/navi_geo/admin.py
Normal file
137
backend/services/navi_geo/admin.py
Normal file
|
|
@ -0,0 +1,137 @@
|
|||
"""navi-geo admin-info endpoint (handoff §4.5).
|
||||
|
||||
``GET /api/admin/navi-geo/info`` — Authentik-gated, read-only.
|
||||
|
||||
Per Phase A §10 this service has NO secrets: landclass is HTTP-delegated to
|
||||
navi-landclass, so ``PADUS_DB_*`` disappears entirely. The env block below
|
||||
contains only non-secret URLs/paths, and ``build_info_response`` is given no
|
||||
masked values.
|
||||
"""
|
||||
import os
|
||||
import time
|
||||
|
||||
import requests
|
||||
from flask import Blueprint, jsonify, current_app
|
||||
|
||||
from shared.auth import require_auth
|
||||
from shared.admin_info import build_info_response
|
||||
|
||||
from . import netsyms
|
||||
from .geocode import photon_url
|
||||
from .landclass_client import landclass_url
|
||||
from .netsyms import db_path as netsyms_db_path
|
||||
from .dem import dem_path
|
||||
from .geo_route import tz_db_path
|
||||
from .address_book import _config_path as address_book_path
|
||||
|
||||
bp = Blueprint('geo_admin', __name__)
|
||||
|
||||
PORT = 8426
|
||||
|
||||
|
||||
def _photon_dependency():
|
||||
"""Photon up iff GET /api?q=test&limit=1 returns 200."""
|
||||
start = time.monotonic()
|
||||
try:
|
||||
resp = requests.get(
|
||||
f"{photon_url()}/api", params={'q': 'test', 'limit': 1}, timeout=3
|
||||
)
|
||||
latency_ms = round((time.monotonic() - start) * 1000, 1)
|
||||
ok = resp.status_code == 200
|
||||
r = {'name': 'photon', 'status': 'ok' if ok else 'error', 'latency_ms': latency_ms}
|
||||
if not ok:
|
||||
r['error'] = f'HTTP {resp.status_code}'
|
||||
return r
|
||||
except Exception as e:
|
||||
latency_ms = round((time.monotonic() - start) * 1000, 1)
|
||||
return {'name': 'photon', 'status': 'error', 'latency_ms': latency_ms, 'error': type(e).__name__}
|
||||
|
||||
|
||||
def _landclass_dependency():
|
||||
"""navi-landclass up iff /api/landclass?lat=0&lon=0 returns 200 (ocean point
|
||||
→ summary:null, the confirmed 'no coverage' shape)."""
|
||||
start = time.monotonic()
|
||||
try:
|
||||
resp = requests.get(
|
||||
f"{landclass_url()}/api/landclass", params={'lat': 0, 'lon': 0}, timeout=3
|
||||
)
|
||||
latency_ms = round((time.monotonic() - start) * 1000, 1)
|
||||
ok = resp.status_code == 200
|
||||
r = {'name': 'navi-landclass', 'status': 'ok' if ok else 'error', 'latency_ms': latency_ms}
|
||||
if not ok:
|
||||
r['error'] = f'HTTP {resp.status_code}'
|
||||
return r
|
||||
except Exception as e:
|
||||
latency_ms = round((time.monotonic() - start) * 1000, 1)
|
||||
return {'name': 'navi-landclass', 'status': 'error', 'latency_ms': latency_ms, 'error': type(e).__name__}
|
||||
|
||||
|
||||
def _file_entry(path):
|
||||
"""Cheap present/absent + readable report for an external data file. Never
|
||||
errors on missing — just reports (Phase A §11 files stay external)."""
|
||||
return {
|
||||
'path': path,
|
||||
'exists': os.path.exists(path),
|
||||
'readable': os.access(path, os.R_OK),
|
||||
}
|
||||
|
||||
|
||||
def _netsyms_fs_entry(path):
|
||||
"""netsyms filesystem entry enriched with netsyms.health() — row count, file
|
||||
size, and indexed countries on top of the standard path/exists/readable.
|
||||
|
||||
health() degrades gracefully (ok:False, zeros) when the DB is absent, so this
|
||||
never raises. row_count is cached after the first call inside netsyms; the
|
||||
DISTINCT-country query runs per call but admin-info is auth-gated + rare."""
|
||||
entry = _file_entry(path)
|
||||
h = netsyms.health()
|
||||
entry.update({
|
||||
'ok': h['ok'],
|
||||
'row_count': h['row_count'],
|
||||
'file_size_bytes': h['file_size_bytes'],
|
||||
'indexed_countries': h['indexed_countries'],
|
||||
})
|
||||
return entry
|
||||
|
||||
|
||||
@bp.route('/api/admin/navi-geo/info')
|
||||
@require_auth
|
||||
def navi_geo_info():
|
||||
metrics = current_app.config['METRICS']
|
||||
netsyms_db = netsyms_db_path()
|
||||
timezone_db = tz_db_path()
|
||||
dem_file = str(dem_path())
|
||||
trace_log = os.environ.get('NAVI_GEO_RERANK_TRACE_LOG', '(unset — trace off)')
|
||||
|
||||
info = build_info_response(
|
||||
service='navi-geo',
|
||||
version=current_app.config.get('VERSION', 'unknown'),
|
||||
port=PORT,
|
||||
config={},
|
||||
# No secrets in this service (Phase A §10) — all non-secret URLs/paths.
|
||||
env=[
|
||||
{'name': 'PHOTON_URL', 'value': photon_url()},
|
||||
{'name': 'NAVI_LANDCLASS_URL', 'value': landclass_url()},
|
||||
{'name': 'NAVI_NETSYMS_DB', 'value': netsyms_db},
|
||||
{'name': 'NAVI_TIMEZONE_DB', 'value': timezone_db},
|
||||
{'name': 'NAVI_DEM_PMTILES', 'value': dem_file},
|
||||
{'name': 'NAVI_ADDRESS_BOOK_YAML', 'value': address_book_path()},
|
||||
{'name': 'NAVI_GEO_RERANK_TRACE_LOG', 'value': trace_log},
|
||||
],
|
||||
dependencies=[
|
||||
_photon_dependency(),
|
||||
_landclass_dependency(),
|
||||
],
|
||||
filesystem=[
|
||||
_netsyms_fs_entry(netsyms_db),
|
||||
_file_entry(timezone_db),
|
||||
_file_entry(dem_file),
|
||||
_file_entry(address_book_path()),
|
||||
],
|
||||
runtime={
|
||||
'uptime_s': round(time.time() - metrics['start_time'], 1),
|
||||
'request_count': metrics['request_count'],
|
||||
'last_error_at': metrics['last_error_at'],
|
||||
},
|
||||
)
|
||||
return jsonify(info)
|
||||
58
backend/services/navi_geo/app.py
Normal file
58
backend/services/navi_geo/app.py
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
"""navi-geo Flask application factory + gunicorn entry.
|
||||
|
||||
Gunicorn entry:
|
||||
gunicorn 'services.navi_geo.app:create_app()' --bind 127.0.0.1:8426 --workers 2
|
||||
"""
|
||||
import subprocess
|
||||
import time
|
||||
|
||||
from flask import Flask
|
||||
|
||||
from . import geo_route, admin
|
||||
from . import geocode as geocode_mod
|
||||
from . import netsyms
|
||||
from . import address_book
|
||||
|
||||
|
||||
def _git_sha():
|
||||
try:
|
||||
sha = subprocess.check_output(
|
||||
['git', 'rev-parse', '--short', 'HEAD'],
|
||||
stderr=subprocess.DEVNULL, text=True,
|
||||
).strip()
|
||||
return sha or 'unknown'
|
||||
except Exception:
|
||||
return 'unknown'
|
||||
|
||||
|
||||
def create_app():
|
||||
app = Flask(__name__)
|
||||
|
||||
app.config['VERSION'] = _git_sha()
|
||||
app.config['METRICS'] = {
|
||||
'start_time': time.time(),
|
||||
'request_count': 0,
|
||||
'last_error_at': None,
|
||||
}
|
||||
|
||||
# Fresh per-worker (and per-test) state, so each picks up current env.
|
||||
geo_route.reset_cache()
|
||||
netsyms.reset_conn()
|
||||
address_book.reset_cache()
|
||||
geocode_mod.setup_trace_logger() # re-read NAVI_GEO_RERANK_TRACE_LOG
|
||||
|
||||
@app.before_request
|
||||
def _count_request():
|
||||
app.config['METRICS']['request_count'] += 1
|
||||
|
||||
@app.after_request
|
||||
def _track_errors(response):
|
||||
if response.status_code >= 500:
|
||||
app.config['METRICS']['last_error_at'] = time.strftime(
|
||||
'%Y-%m-%dT%H:%M:%SZ', time.gmtime()
|
||||
)
|
||||
return response
|
||||
|
||||
app.register_blueprint(geo_route.bp)
|
||||
app.register_blueprint(admin.bp)
|
||||
return app
|
||||
218
backend/services/navi_geo/dem.py
Executable file
218
backend/services/navi_geo/dem.py
Executable file
|
|
@ -0,0 +1,218 @@
|
|||
"""
|
||||
DEM tile reader (port of recon's offroute/dem.py).
|
||||
|
||||
Reads elevation tiles from planet-dem.pmtiles (Terrarium-encoded WebP),
|
||||
decodes them into numpy arrays, and provides a stitched elevation grid
|
||||
for a given bounding box. navi-geo uses ``sample_point`` for the reverse
|
||||
bundle's ``elevation_m``. Faithful port; only change is the env-override path.
|
||||
"""
|
||||
import math
|
||||
import os
|
||||
from functools import lru_cache
|
||||
from io import BytesIO
|
||||
from pathlib import Path
|
||||
from typing import Tuple, Optional
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
from pmtiles.reader import MmapSource, Reader as PMTilesReader
|
||||
|
||||
# Default path to the planet DEM PMTiles file (~657 GB, stays external).
|
||||
DEFAULT_DEM_PATH = Path("/mnt/nas/nav/planet-dem.pmtiles")
|
||||
|
||||
|
||||
def dem_path():
|
||||
"""planet-DEM PMTiles path, env-overridable via NAVI_DEM_PMTILES."""
|
||||
return Path(os.environ.get('NAVI_DEM_PMTILES', str(DEFAULT_DEM_PATH)))
|
||||
|
||||
# Tile size in pixels (z12 tiles are 512x512 in this tileset)
|
||||
TILE_SIZE = 512
|
||||
|
||||
# Zoom level to use for elevation data
|
||||
ZOOM_LEVEL = 12
|
||||
|
||||
|
||||
def terrarium_decode(rgb_array: np.ndarray) -> np.ndarray:
|
||||
"""
|
||||
Decode Terrarium-encoded RGB values to elevation in meters.
|
||||
|
||||
Formula: elevation = (R * 256 + G + B/256) - 32768
|
||||
"""
|
||||
r = rgb_array[:, :, 0].astype(np.float32)
|
||||
g = rgb_array[:, :, 1].astype(np.float32)
|
||||
b = rgb_array[:, :, 2].astype(np.float32)
|
||||
|
||||
elevation = (r * 256.0 + g + b / 256.0) - 32768.0
|
||||
return elevation
|
||||
|
||||
|
||||
def lat_lon_to_tile(lat: float, lon: float, zoom: int) -> Tuple[int, int]:
|
||||
"""Convert lat/lon to tile coordinates at given zoom level."""
|
||||
n = 2 ** zoom
|
||||
x = int((lon + 180.0) / 360.0 * n)
|
||||
lat_rad = math.radians(lat)
|
||||
y = int((1.0 - math.asinh(math.tan(lat_rad)) / math.pi) / 2.0 * n)
|
||||
return x, y
|
||||
|
||||
|
||||
def tile_to_lat_lon(x: int, y: int, zoom: int) -> Tuple[float, float, float, float]:
|
||||
"""Convert tile coordinates to bounding box (north, south, west, east)."""
|
||||
n = 2 ** zoom
|
||||
lon_west = x / n * 360.0 - 180.0
|
||||
lon_east = (x + 1) / n * 360.0 - 180.0
|
||||
lat_north = math.degrees(math.atan(math.sinh(math.pi * (1 - 2 * y / n))))
|
||||
lat_south = math.degrees(math.atan(math.sinh(math.pi * (1 - 2 * (y + 1) / n))))
|
||||
return lat_north, lat_south, lon_west, lon_east
|
||||
|
||||
|
||||
class DEMReader:
|
||||
"""Reader for Terrarium-encoded DEM tiles from PMTiles."""
|
||||
|
||||
def __init__(self, pmtiles_path: Path = DEFAULT_DEM_PATH, tile_cache_size: int = 128):
|
||||
self.pmtiles_path = pmtiles_path
|
||||
self._source = MmapSource(open(pmtiles_path, "rb"))
|
||||
self._reader = PMTilesReader(self._source)
|
||||
self._header = self._reader.header()
|
||||
self._decode_tile = lru_cache(maxsize=tile_cache_size)(self._decode_tile_impl)
|
||||
|
||||
def _decode_tile_impl(self, z: int, x: int, y: int) -> Optional[np.ndarray]:
|
||||
"""Fetch and decode a single tile."""
|
||||
tile_data = self._reader.get(z, x, y)
|
||||
if tile_data is None:
|
||||
return None
|
||||
|
||||
img = Image.open(BytesIO(tile_data))
|
||||
rgb_array = np.array(img)
|
||||
|
||||
if rgb_array.shape[2] == 4:
|
||||
rgb_array = rgb_array[:, :, :3]
|
||||
|
||||
elevation = terrarium_decode(rgb_array)
|
||||
return elevation
|
||||
|
||||
def get_elevation_grid(
|
||||
self,
|
||||
south: float,
|
||||
north: float,
|
||||
west: float,
|
||||
east: float,
|
||||
zoom: int = ZOOM_LEVEL
|
||||
) -> Tuple[np.ndarray, dict]:
|
||||
"""Get a stitched elevation grid for the given bounding box."""
|
||||
x_min, y_max = lat_lon_to_tile(south, west, zoom)
|
||||
x_max, y_min = lat_lon_to_tile(north, east, zoom)
|
||||
|
||||
n = 2 ** zoom
|
||||
x_min = max(0, x_min)
|
||||
x_max = min(n - 1, x_max)
|
||||
y_min = max(0, y_min)
|
||||
y_max = min(n - 1, y_max)
|
||||
|
||||
n_tiles_x = x_max - x_min + 1
|
||||
n_tiles_y = y_max - y_min + 1
|
||||
out_height = n_tiles_y * TILE_SIZE
|
||||
out_width = n_tiles_x * TILE_SIZE
|
||||
|
||||
elevation = np.full((out_height, out_width), np.nan, dtype=np.float32)
|
||||
|
||||
for ty in range(y_min, y_max + 1):
|
||||
for tx in range(x_min, x_max + 1):
|
||||
tile_elev = self._decode_tile(zoom, tx, ty)
|
||||
if tile_elev is not None:
|
||||
out_y = (ty - y_min) * TILE_SIZE
|
||||
out_x = (tx - x_min) * TILE_SIZE
|
||||
elevation[out_y:out_y + TILE_SIZE, out_x:out_x + TILE_SIZE] = tile_elev
|
||||
|
||||
grid_north, _, grid_west, _ = tile_to_lat_lon(x_min, y_min, zoom)
|
||||
_, grid_south, _, grid_east = tile_to_lat_lon(x_max, y_max, zoom)
|
||||
|
||||
pixel_size_lat = (grid_north - grid_south) / out_height
|
||||
pixel_size_lon = (grid_east - grid_west) / out_width
|
||||
|
||||
origin_lat = grid_north - pixel_size_lat / 2
|
||||
origin_lon = grid_west + pixel_size_lon / 2
|
||||
|
||||
center_lat = (south + north) / 2
|
||||
lat_m = 111320.0
|
||||
lon_m = 111320.0 * math.cos(math.radians(center_lat))
|
||||
cell_size_lat_m = abs(pixel_size_lat) * lat_m
|
||||
cell_size_lon_m = abs(pixel_size_lon) * lon_m
|
||||
cell_size_m = (cell_size_lat_m + cell_size_lon_m) / 2
|
||||
|
||||
row_start = int((grid_north - north) / abs(pixel_size_lat))
|
||||
row_end = int((grid_north - south) / abs(pixel_size_lat))
|
||||
col_start = int((west - grid_west) / pixel_size_lon)
|
||||
col_end = int((east - grid_west) / pixel_size_lon)
|
||||
|
||||
row_start = max(0, row_start)
|
||||
row_end = min(out_height, row_end)
|
||||
col_start = max(0, col_start)
|
||||
col_end = min(out_width, col_end)
|
||||
|
||||
elevation = elevation[row_start:row_end, col_start:col_end]
|
||||
|
||||
origin_lat = grid_north - (row_start + 0.5) * abs(pixel_size_lat)
|
||||
origin_lon = grid_west + (col_start + 0.5) * pixel_size_lon
|
||||
|
||||
metadata = {
|
||||
"bounds": (south, north, west, east),
|
||||
"pixel_size_lat": -abs(pixel_size_lat),
|
||||
"pixel_size_lon": pixel_size_lon,
|
||||
"origin_lat": origin_lat,
|
||||
"origin_lon": origin_lon,
|
||||
"cell_size_m": cell_size_m,
|
||||
"shape": elevation.shape,
|
||||
}
|
||||
|
||||
return elevation, metadata
|
||||
|
||||
def sample_point(self, lat: float, lon: float) -> Optional[float]:
|
||||
"""Return elevation in meters at a single point, or None if untiled.
|
||||
|
||||
Reads one z12 Terrarium tile (LRU-cached) and indexes the matching
|
||||
pixel. Sub-ms warm, ~15 ms cold per tile via NFS. Returns None when the
|
||||
tile is absent (e.g. true ocean nodata) or lat is outside the
|
||||
Web-Mercator pole cap (~+/-85.05 deg).
|
||||
"""
|
||||
if not -85.05112878 <= lat <= 85.05112878:
|
||||
return None
|
||||
n = 2 ** ZOOM_LEVEL
|
||||
fx = (lon + 180.0) / 360.0 * n
|
||||
fy = (1.0 - math.asinh(math.tan(math.radians(lat))) / math.pi) / 2.0 * n
|
||||
tx, ty = int(fx), int(fy)
|
||||
tile = self._decode_tile(ZOOM_LEVEL, tx, ty)
|
||||
if tile is None:
|
||||
return None
|
||||
row = min(TILE_SIZE - 1, int((fy - ty) * TILE_SIZE))
|
||||
col = min(TILE_SIZE - 1, int((fx - tx) * TILE_SIZE))
|
||||
return float(tile[row, col])
|
||||
|
||||
def pixel_to_latlon(self, row: int, col: int, metadata: dict) -> Tuple[float, float]:
|
||||
"""Convert pixel coordinates to lat/lon."""
|
||||
lat = metadata["origin_lat"] + row * metadata["pixel_size_lat"]
|
||||
lon = metadata["origin_lon"] + col * metadata["pixel_size_lon"]
|
||||
return lat, lon
|
||||
|
||||
def latlon_to_pixel(self, lat: float, lon: float, metadata: dict) -> Tuple[int, int]:
|
||||
"""Convert lat/lon to pixel coordinates."""
|
||||
row = int((metadata["origin_lat"] - lat) / abs(metadata["pixel_size_lat"]))
|
||||
col = int((lon - metadata["origin_lon"]) / metadata["pixel_size_lon"])
|
||||
return row, col
|
||||
|
||||
def close(self):
|
||||
"""Close the PMTiles file."""
|
||||
pass # MmapSource handles cleanup
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
reader = DEMReader()
|
||||
elevation, meta = reader.get_elevation_grid(
|
||||
south=42.4, north=42.6, west=-114.5, east=-114.3
|
||||
)
|
||||
print(f"Elevation grid shape: {elevation.shape}")
|
||||
print(f"Cell size: {meta['cell_size_m']:.1f} m")
|
||||
print(f"Elevation range: {np.nanmin(elevation):.1f} - {np.nanmax(elevation):.1f} m")
|
||||
center_row, center_col = elevation.shape[0] // 2, elevation.shape[1] // 2
|
||||
lat, lon = reader.pixel_to_latlon(center_row, center_col, meta)
|
||||
print(f"Center pixel lat/lon: {lat:.4f}, {lon:.4f}")
|
||||
reader.close()
|
||||
295
backend/services/navi_geo/geo_route.py
Normal file
295
backend/services/navi_geo/geo_route.py
Normal file
|
|
@ -0,0 +1,295 @@
|
|||
"""navi-geo API blueprint — faithful port of recon's geocode/reverse routes.
|
||||
|
||||
GET /api/geocode?q=&limit=&lat=&lon=&zoom= Photon-first ranked search
|
||||
GET /api/reverse?lat=&lon= reverse geocode (Photon)
|
||||
GET /api/reverse/<lat>/<lon> reverse enrichment bundle (Central)
|
||||
|
||||
Ported from recon's ``lib/netsyms_api.py`` (the ``geocode_bp`` half). All three
|
||||
routes are public (no auth), matching recon. Behaviour-identical except:
|
||||
- landclass is fetched over HTTP from navi-landclass (Phase A §5), not in-process
|
||||
- Photon URL / timezone DB / DEM path come from env vars, not hardcoded constants
|
||||
|
||||
The unrelated ``/api/netsyms/*`` debug routes are NOT ported (they stay in recon).
|
||||
"""
|
||||
import logging
|
||||
import os
|
||||
import sqlite3
|
||||
import threading
|
||||
|
||||
import requests as http_requests
|
||||
from cachetools import TTLCache
|
||||
from flask import Blueprint, request, jsonify
|
||||
|
||||
from . import geocode as geocode_mod
|
||||
from . import landclass_client
|
||||
from .geocode import photon_url, _parse_photon_features
|
||||
from .dem import DEMReader, dem_path
|
||||
|
||||
logger = logging.getLogger('navi_geo.geo_route')
|
||||
|
||||
bp = Blueprint('geo', __name__)
|
||||
|
||||
# ── Timezone DB (single source of truth for the default) ──
|
||||
DEFAULT_TZ_DB_PATH = '/mnt/nav/sources/timezones.sqlite'
|
||||
|
||||
|
||||
def tz_db_path():
|
||||
"""SpatiaLite timezone DB path, env-overridable via NAVI_TIMEZONE_DB."""
|
||||
return os.environ.get('NAVI_TIMEZONE_DB', DEFAULT_TZ_DB_PATH)
|
||||
|
||||
|
||||
# ── Reverse-bundle cache: key=(round(lat,4), round(lon,4)) -> dict. ──
|
||||
# ~10k entries, 24h TTL, per gunicorn worker (in-memory; not shared/persisted).
|
||||
_REVERSE_BUNDLE_CACHE = TTLCache(maxsize=10_000, ttl=86_400)
|
||||
_REVERSE_BUNDLE_LOCK = threading.Lock()
|
||||
|
||||
# Exact key set the bundle always returns (Central consumes this contract).
|
||||
_BUNDLE_KEYS = ('name', 'city', 'county', 'state', 'country',
|
||||
'postal_code', 'timezone', 'landclass', 'elevation_m')
|
||||
|
||||
# planet-DEM elevation source (single PMTiles). Instantiated once at import; the
|
||||
# underlying mmap is lazy. None if unavailable — the startup log tells us if the
|
||||
# mount is missing (Phase B locked decision #3, Option A).
|
||||
try:
|
||||
_DEM = DEMReader(dem_path())
|
||||
except Exception as e: # pragma: no cover - depends on PMTiles availability
|
||||
logger.warning("DEMReader unavailable, elevation will be null: %s", e)
|
||||
_DEM = None
|
||||
|
||||
|
||||
def reset_cache():
|
||||
"""Clear the reverse-bundle cache (per app instance / per test)."""
|
||||
with _REVERSE_BUNDLE_LOCK:
|
||||
_REVERSE_BUNDLE_CACHE.clear()
|
||||
|
||||
|
||||
def _safe_float(val, lo, hi):
|
||||
"""Parse val as float; return None if missing, non-numeric, or out of [lo, hi]."""
|
||||
if val is None:
|
||||
return None
|
||||
try:
|
||||
f = float(val)
|
||||
if lo <= f <= hi:
|
||||
return f
|
||||
except (ValueError, TypeError):
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
@bp.route('/api/geocode')
|
||||
def api_geocode():
|
||||
"""
|
||||
Photon-first geocoding with ranked candidates.
|
||||
|
||||
GET /api/geocode?q=<query>&limit=<N>
|
||||
|
||||
Always returns 200 OK with:
|
||||
{query, results: [{name, lat, lon, source, confidence, type, raw, ...}], count}
|
||||
|
||||
- source: "address_book" | "coordinates" | "photon"
|
||||
- confidence: "exact" | "high" | "medium" | "low"
|
||||
- type: "nickname" | "coordinates" | "street_address" | "poi" | "locality"
|
||||
- labeled_as: present when result is within 75m of an address book entry
|
||||
- Empty results array is valid (no match). No 404s.
|
||||
"""
|
||||
q = request.args.get('q', '').strip()
|
||||
limit = request.args.get('limit', '10')
|
||||
try:
|
||||
limit = max(1, min(int(limit), 20))
|
||||
except (ValueError, TypeError):
|
||||
limit = 10
|
||||
|
||||
# Viewport bias parameters (optional)
|
||||
lat = _safe_float(request.args.get("lat"), -90, 90)
|
||||
lon = _safe_float(request.args.get("lon"), -180, 180)
|
||||
zoom = _safe_float(request.args.get("zoom"), 0, 22)
|
||||
|
||||
result = geocode_mod.geocode(q, limit=limit, lat=lat, lon=lon, zoom=zoom)
|
||||
return jsonify(result)
|
||||
|
||||
|
||||
@bp.route('/api/reverse')
|
||||
def api_reverse():
|
||||
"""
|
||||
Reverse geocode coordinates via Photon.
|
||||
|
||||
GET /api/reverse?lat=X&lon=Y
|
||||
|
||||
Returns same shape as /api/geocode:
|
||||
{query: "lat,lon", results: [{name, lat, lon, source, type, raw, ...}], count}
|
||||
|
||||
Returns 200 OK with empty results on no match. 400 on invalid coords.
|
||||
"""
|
||||
try:
|
||||
lat = float(request.args.get('lat', ''))
|
||||
lon = float(request.args.get('lon', ''))
|
||||
except (ValueError, TypeError):
|
||||
return jsonify({'error': 'Missing or invalid lat/lon parameters'}), 400
|
||||
|
||||
if not (-90 <= lat <= 90) or not (-180 <= lon <= 180):
|
||||
return jsonify({'error': 'Coordinates out of range'}), 400
|
||||
|
||||
query_str = f"{lat},{lon}"
|
||||
|
||||
try:
|
||||
resp = http_requests.get(
|
||||
f"{photon_url()}/reverse",
|
||||
params={"lat": lat, "lon": lon, "limit": 1},
|
||||
timeout=10,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
features = data.get("features", [])
|
||||
except Exception:
|
||||
logger.warning("Photon reverse geocode failed for %s", query_str)
|
||||
return jsonify({'query': query_str, 'results': [], 'count': 0})
|
||||
|
||||
if not features:
|
||||
return jsonify({'query': query_str, 'results': [], 'count': 0})
|
||||
|
||||
results = _parse_photon_features(features, source='photon_reverse')
|
||||
|
||||
return jsonify({'query': query_str, 'results': results, 'count': len(results)})
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────
|
||||
# /api/reverse/<lat>/<lon> — localhost-sourced enrichment bundle (Central)
|
||||
#
|
||||
# Sibling to the query-string /api/reverse above; that route is unchanged.
|
||||
# Every component is sourced locally (Photon, timezones.sqlite, navi-landclass
|
||||
# over HTTP, planet-DEM PMTiles). Each lookup is independent: a component
|
||||
# failure logs a warning and yields null — never 5xx.
|
||||
# ─────────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _spatialite_blob_to_wkb(blob):
|
||||
"""Recover standard WKB from a SpatiaLite geometry BLOB.
|
||||
|
||||
Layout: [00][endian][srid:4][mbr:32][7C][WKB body][FE]. The body omits the
|
||||
leading byte-order marker, so we re-prepend it and drop the trailing 0xFE.
|
||||
"""
|
||||
return bytes([blob[1]]) + blob[39:-1]
|
||||
|
||||
|
||||
def _reverse_photon(lat, lon):
|
||||
"""Nearest-feature admin fields from local Photon. Returns the six address
|
||||
fields (any value may be None). Mirrors the existing /api/reverse call."""
|
||||
resp = http_requests.get(
|
||||
f"{photon_url()}/reverse",
|
||||
params={"lat": lat, "lon": lon, "limit": 1},
|
||||
timeout=10,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
features = resp.json().get("features", [])
|
||||
if not features:
|
||||
return {}
|
||||
props = features[0].get("properties", {})
|
||||
return {
|
||||
"name": props.get("name"),
|
||||
"city": props.get("city"),
|
||||
"county": props.get("county"),
|
||||
"state": props.get("state"),
|
||||
"country": props.get("country"),
|
||||
"postal_code": props.get("postcode"),
|
||||
}
|
||||
|
||||
|
||||
def _reverse_timezone(lat, lon):
|
||||
"""IANA tzid for the point from local timezones.sqlite (SpatiaLite tz_world).
|
||||
|
||||
Uses the table's R-tree index for an MBR prefilter, then shapely
|
||||
point-in-polygon on the few candidates. Returns None if unresolved.
|
||||
"""
|
||||
from shapely import wkb
|
||||
from shapely.geometry import Point
|
||||
con = sqlite3.connect(f"file:{tz_db_path()}?mode=ro", uri=True)
|
||||
try:
|
||||
cur = con.cursor()
|
||||
cur.execute(
|
||||
"SELECT pkid FROM idx_tz_world_geom "
|
||||
"WHERE xmin<=? AND xmax>=? AND ymin<=? AND ymax>=?",
|
||||
(lon, lon, lat, lat),
|
||||
)
|
||||
candidates = [r[0] for r in cur.fetchall()]
|
||||
if not candidates:
|
||||
return None
|
||||
pt = Point(lon, lat)
|
||||
for pk in candidates:
|
||||
row = cur.execute(
|
||||
"SELECT tzid, geom FROM tz_world WHERE pk_uid=?", (pk,)
|
||||
).fetchone()
|
||||
if row and wkb.loads(_spatialite_blob_to_wkb(row[1])).contains(pt):
|
||||
return row[0]
|
||||
return None
|
||||
finally:
|
||||
con.close()
|
||||
|
||||
|
||||
def _reverse_landclass(lat, lon):
|
||||
"""Most-specific PAD-US land class for the point, via navi-landclass HTTP.
|
||||
|
||||
Phase A §5: recon called landclass in-process and returned the most-specific
|
||||
unit name (a string). Here we GET navi-landclass /api/landclass and read its
|
||||
``summary`` field — the same string. Returns None on no coverage/unavailable.
|
||||
"""
|
||||
return landclass_client.reverse_landclass_summary(lat, lon)
|
||||
|
||||
|
||||
def _reverse_elevation(lat, lon):
|
||||
"""Elevation in metres from the planet-DEM PMTiles — the single elevation
|
||||
source. None on failure, on untiled points (e.g. true ocean), or if
|
||||
DEMReader could not be initialized at startup."""
|
||||
if _DEM is None:
|
||||
return None
|
||||
return _DEM.sample_point(lat, lon)
|
||||
|
||||
|
||||
@bp.route('/api/reverse/<lat>/<lon>')
|
||||
def api_reverse_bundle(lat, lon):
|
||||
"""Localhost-sourced reverse-geocode enrichment bundle for Central.
|
||||
|
||||
GET /api/reverse/<lat>/<lon>
|
||||
|
||||
Always returns 200 with EXACTLY these keys (any may be null):
|
||||
name, city, county, state, country, postal_code, timezone, landclass, elevation_m
|
||||
|
||||
lat/lon are parsed manually (not via Flask's <float:> converter, which
|
||||
rejects negative and integer coordinates) so out-of-range or unparseable
|
||||
input yields 400 per contract; 503 is reserved for catastrophic failure.
|
||||
"""
|
||||
try:
|
||||
lat = float(lat)
|
||||
lon = float(lon)
|
||||
except (ValueError, TypeError):
|
||||
return jsonify({'error': 'lat and lon must be numbers'}), 400
|
||||
if not (-90 <= lat <= 90) or not (-180 <= lon <= 180):
|
||||
return jsonify({'error': 'lat must be -90..90, lon must be -180..180'}), 400
|
||||
|
||||
key = (round(lat, 4), round(lon, 4))
|
||||
with _REVERSE_BUNDLE_LOCK:
|
||||
cached = _REVERSE_BUNDLE_CACHE.get(key)
|
||||
if cached is not None:
|
||||
return jsonify(cached)
|
||||
|
||||
bundle = {k: None for k in _BUNDLE_KEYS}
|
||||
|
||||
try:
|
||||
bundle.update(_reverse_photon(lat, lon))
|
||||
except Exception:
|
||||
logger.warning("reverse-bundle: Photon lookup failed for %s,%s", lat, lon)
|
||||
try:
|
||||
bundle['timezone'] = _reverse_timezone(lat, lon)
|
||||
except Exception:
|
||||
logger.warning("reverse-bundle: timezone lookup failed for %s,%s", lat, lon)
|
||||
try:
|
||||
bundle['landclass'] = _reverse_landclass(lat, lon)
|
||||
except Exception:
|
||||
logger.warning("reverse-bundle: landclass lookup failed for %s,%s", lat, lon)
|
||||
try:
|
||||
bundle['elevation_m'] = _reverse_elevation(lat, lon)
|
||||
except Exception:
|
||||
logger.warning("reverse-bundle: elevation lookup failed for %s,%s", lat, lon)
|
||||
|
||||
with _REVERSE_BUNDLE_LOCK:
|
||||
_REVERSE_BUNDLE_CACHE[key] = bundle
|
||||
return jsonify(bundle)
|
||||
806
backend/services/navi_geo/geocode.py
Executable file
806
backend/services/navi_geo/geocode.py
Executable file
|
|
@ -0,0 +1,806 @@
|
|||
"""
|
||||
navi-geo geocode — structured preprocessing, multi-source retrieval, reranking.
|
||||
|
||||
Faithful port of recon's ``lib/geocode.py``. Behaviour-identical to recon; the
|
||||
only changes are (a) the Photon base URL is read from ``PHOTON_URL`` instead of
|
||||
a hardcoded constant, and (b) the reranking-audit trace log is opt-in via
|
||||
``NAVI_GEO_RERANK_TRACE_LOG`` (recon always wrote ``/tmp/geocode_rerank_trace.log``
|
||||
at import; here no FileHandler is attached unless the env var is set).
|
||||
|
||||
Replaces the naive Photon-only search with:
|
||||
1. usaddress parsing + intent classification (ADDRESS / POI / LOCALITY / COORD / POSTCODE)
|
||||
2. Multi-source retrieval: ADDRESS → Netsyms + Photon; POI/LOCALITY → Photon /api
|
||||
3. Python reranker with weighted signals
|
||||
|
||||
Public entry point: geocode(query, limit) → {query, results, count}
|
||||
"""
|
||||
|
||||
import math
|
||||
import os
|
||||
import re
|
||||
import logging
|
||||
|
||||
import requests
|
||||
import usaddress
|
||||
from rapidfuzz import fuzz
|
||||
|
||||
logger = logging.getLogger('navi_geo.geocode')
|
||||
|
||||
# ── Photon base URL (single source of truth for the default) ──
|
||||
DEFAULT_PHOTON_URL = "http://localhost:2322"
|
||||
|
||||
|
||||
def photon_url():
|
||||
"""Photon base URL, env-overridable. Read per-call so tests/env apply."""
|
||||
return os.environ.get('PHOTON_URL', DEFAULT_PHOTON_URL)
|
||||
|
||||
|
||||
# ── Trace logger for reranking audit (opt-in) ──
|
||||
# recon attached a DEBUG FileHandler to /tmp at import unconditionally. Here the
|
||||
# handler is attached only when NAVI_GEO_RERANK_TRACE_LOG names a path; otherwise
|
||||
# the logger stays silent and cheap (level above DEBUG short-circuits formatting).
|
||||
_trace_logger = logging.getLogger('navi_geo.geocode.trace')
|
||||
_trace_logger.propagate = False
|
||||
|
||||
|
||||
def setup_trace_logger():
|
||||
"""(Re)configure the rerank trace logger from NAVI_GEO_RERANK_TRACE_LOG."""
|
||||
for h in list(_trace_logger.handlers):
|
||||
_trace_logger.removeHandler(h)
|
||||
path = os.environ.get('NAVI_GEO_RERANK_TRACE_LOG')
|
||||
if path:
|
||||
handler = logging.FileHandler(path)
|
||||
handler.setFormatter(logging.Formatter('%(asctime)s %(message)s'))
|
||||
_trace_logger.addHandler(handler)
|
||||
_trace_logger.setLevel(logging.DEBUG)
|
||||
else:
|
||||
# No sink: NullHandler + level above DEBUG so .debug() calls are no-ops.
|
||||
_trace_logger.addHandler(logging.NullHandler())
|
||||
_trace_logger.setLevel(logging.WARNING)
|
||||
|
||||
|
||||
setup_trace_logger()
|
||||
|
||||
# ── Config constants ──
|
||||
GEOCODE_BIAS_LAT = 42.5736
|
||||
GEOCODE_BIAS_LON = -114.6066
|
||||
GEOCODE_BIAS_ZOOM = 10
|
||||
ADDRESS_BOOK_ANNOTATION_RADIUS_M = 75
|
||||
|
||||
# ── Reranker weights ──
|
||||
# Derived from research analysis of failure modes:
|
||||
# housenumber_exact is the strongest signal because Photon's soft-boost
|
||||
# lets wrong-number results bubble up. street_name_fuzz and locality_fuzz
|
||||
# handle abbreviation/case variation. source_authority gives Netsyms a
|
||||
# boost for US addresses since it has USPS-verified data.
|
||||
W_HOUSENUMBER_EXACT = 6.0 # exact housenumber match
|
||||
W_HOUSENUMBER_MISMATCH = -5.0 # housenumber present but wrong
|
||||
W_STREET_NAME_FUZZ = 3.0 # fuzzy street name similarity [0..1] * weight
|
||||
W_TOKEN_COVERAGE = 2.0 # fraction of query tokens found in result
|
||||
W_STREET_TYPE_MATCH = 1.5 # "st" matches "street", etc.
|
||||
W_LOCALITY_FUZZ = 2.0 # city/state fuzzy match
|
||||
W_SOURCE_AUTHORITY = 2.0 # Netsyms for US addresses
|
||||
W_LAYER_RANK = 1.0 # type-appropriate results ranked higher
|
||||
W_PHOTON_POSITION_NORM = 1.0 # Photon's native ranking (normalized by position)
|
||||
W_STATE_EXACT = 1.0 # exact state code match
|
||||
W_POI_CLASS_BOOST = 3.0 # amenity/shop/etc boost for business-name queries
|
||||
W_HIGHWAY_CLASS_PENALTY = -4.0 # highway/route penalty for business-name queries
|
||||
|
||||
# ── US abbreviation expansions ──
|
||||
# Applied ONLY to parsed StreetName/StreetNamePostType tokens, NOT to ordinals.
|
||||
_STREET_TYPE_ABBREVS = {
|
||||
'st': 'street', 'ave': 'avenue', 'blvd': 'boulevard', 'dr': 'drive',
|
||||
'rd': 'road', 'ln': 'lane', 'ct': 'court', 'cir': 'circle',
|
||||
'pl': 'place', 'way': 'way', 'pkwy': 'parkway', 'hwy': 'highway',
|
||||
'trl': 'trail', 'ter': 'terrace', 'sq': 'square',
|
||||
}
|
||||
_DIRECTIONAL_ABBREVS = {
|
||||
'n': 'north', 's': 'south', 'e': 'east', 'w': 'west',
|
||||
'ne': 'northeast', 'nw': 'northwest', 'se': 'southeast', 'sw': 'southwest',
|
||||
}
|
||||
_ORDINAL_RE = re.compile(r'^\d+(st|nd|rd|th)$', re.IGNORECASE)
|
||||
|
||||
# ── Road keywords (for detecting when query is about a road vs a business) ──
|
||||
_ROAD_KEYWORDS = (
|
||||
set(_STREET_TYPE_ABBREVS.keys())
|
||||
| set(_STREET_TYPE_ABBREVS.values())
|
||||
| {'route', 'rte', 'pass'}
|
||||
)
|
||||
|
||||
# ── US state codes ──
|
||||
_STATE_CODES = {
|
||||
'AL', 'AK', 'AZ', 'AR', 'CA', 'CO', 'CT', 'DE', 'FL', 'GA',
|
||||
'HI', 'ID', 'IL', 'IN', 'IA', 'KS', 'KY', 'LA', 'ME', 'MD',
|
||||
'MA', 'MI', 'MN', 'MS', 'MO', 'MT', 'NE', 'NV', 'NH', 'NJ',
|
||||
'NM', 'NY', 'NC', 'ND', 'OH', 'OK', 'OR', 'PA', 'RI', 'SC',
|
||||
'SD', 'TN', 'TX', 'UT', 'VT', 'VA', 'WA', 'WV', 'WI', 'WY', 'DC',
|
||||
}
|
||||
|
||||
# ── Full state name → code (for intent classifier) ──
|
||||
_STATE_NAME_TO_CODE = {
|
||||
'alabama': 'AL', 'alaska': 'AK', 'arizona': 'AZ', 'arkansas': 'AR',
|
||||
'california': 'CA', 'colorado': 'CO', 'connecticut': 'CT', 'delaware': 'DE',
|
||||
'florida': 'FL', 'georgia': 'GA', 'hawaii': 'HI', 'idaho': 'ID',
|
||||
'illinois': 'IL', 'indiana': 'IN', 'iowa': 'IA', 'kansas': 'KS',
|
||||
'kentucky': 'KY', 'louisiana': 'LA', 'maine': 'ME', 'maryland': 'MD',
|
||||
'massachusetts': 'MA', 'michigan': 'MI', 'minnesota': 'MN',
|
||||
'mississippi': 'MS', 'missouri': 'MO', 'montana': 'MT', 'nebraska': 'NE',
|
||||
'nevada': 'NV', 'new hampshire': 'NH', 'new jersey': 'NJ',
|
||||
'new mexico': 'NM', 'new york': 'NY', 'north carolina': 'NC',
|
||||
'north dakota': 'ND', 'ohio': 'OH', 'oklahoma': 'OK', 'oregon': 'OR',
|
||||
'pennsylvania': 'PA', 'rhode island': 'RI', 'south carolina': 'SC',
|
||||
'south dakota': 'SD', 'tennessee': 'TN', 'texas': 'TX', 'utah': 'UT',
|
||||
'vermont': 'VT', 'virginia': 'VA', 'washington': 'WA',
|
||||
'west virginia': 'WV', 'wisconsin': 'WI', 'wyoming': 'WY',
|
||||
}
|
||||
|
||||
# Coordinate regex
|
||||
_COORD_RE = re.compile(r'^\s*(-?\d+\.?\d*)\s*[,\s]\s*(-?\d+\.?\d*)\s*$')
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════════
|
||||
# STEP 1: PREPROCESSING
|
||||
# ═══════════════════════════════════════════════════════════════════
|
||||
|
||||
def _parse_coords(text):
|
||||
"""Return (lat, lon) if text looks like coordinates with valid bounds, else None."""
|
||||
m = _COORD_RE.match(text.strip())
|
||||
if not m:
|
||||
return None
|
||||
lat, lon = float(m.group(1)), float(m.group(2))
|
||||
if -90 <= lat <= 90 and -180 <= lon <= 180:
|
||||
return lat, lon
|
||||
return None
|
||||
|
||||
|
||||
def _classify_and_parse(query):
|
||||
"""
|
||||
Parse query with usaddress, classify intent, expand abbreviations.
|
||||
|
||||
Returns (intent, parsed_dict) where:
|
||||
intent: 'ADDRESS' | 'POI' | 'LOCALITY' | 'POSTCODE' | 'COORD' | 'UNKNOWN'
|
||||
parsed_dict: {number, street, city, state, zipcode, raw_query, expanded_query}
|
||||
"""
|
||||
q = query.strip()
|
||||
parsed = {
|
||||
'number': None, 'street': None, 'street_raw': None,
|
||||
'city': None, 'state': None,
|
||||
'zipcode': None, 'raw_query': q, 'expanded_query': q,
|
||||
}
|
||||
|
||||
# Coordinate check first
|
||||
if _parse_coords(q):
|
||||
return 'COORD', parsed
|
||||
|
||||
# Try usaddress
|
||||
try:
|
||||
tagged, addr_type = usaddress.tag(q)
|
||||
except usaddress.RepeatedLabelError:
|
||||
# Ambiguous input — fall back to free-text Photon
|
||||
return 'UNKNOWN', parsed
|
||||
|
||||
# Extract components
|
||||
number = tagged.get('AddressNumber', '').strip()
|
||||
street_name = tagged.get('StreetName', '').strip()
|
||||
street_pre_dir = tagged.get('StreetNamePreDirectional', '').strip()
|
||||
street_post_type = tagged.get('StreetNamePostType', '').strip()
|
||||
place = tagged.get('PlaceName', '').strip()
|
||||
state = tagged.get('StateName', '').strip()
|
||||
zipcode = tagged.get('ZipCode', '').strip()
|
||||
|
||||
# ── Fix usaddress edge case: "214 N St Filer" ──
|
||||
# usaddress reads single-letter directional + "St" as PreDirectional + empty,
|
||||
# mashing "St Filer" into StreetName. Detect: PreDirectional is single letter,
|
||||
# StreetName has 2+ tokens where the first is a street type.
|
||||
if (street_pre_dir and len(street_pre_dir) <= 2
|
||||
and not street_name.strip().startswith(street_pre_dir)
|
||||
and ' ' in street_name):
|
||||
name_tokens = street_name.split()
|
||||
first_lower = name_tokens[0].lower()
|
||||
if first_lower in _STREET_TYPE_ABBREVS or first_lower in _STREET_TYPE_ABBREVS.values():
|
||||
# "N" is actually the street name, "St" is the post-type
|
||||
street_name = street_pre_dir
|
||||
street_post_type = name_tokens[0]
|
||||
if len(name_tokens) > 1:
|
||||
place = ' '.join(name_tokens[1:])
|
||||
street_pre_dir = ''
|
||||
|
||||
# ── Expand abbreviations (guard ordinals) ──
|
||||
expanded_parts = []
|
||||
|
||||
if number:
|
||||
parsed['number'] = number
|
||||
expanded_parts.append(number)
|
||||
|
||||
if street_pre_dir:
|
||||
exp = _DIRECTIONAL_ABBREVS.get(street_pre_dir.lower(), street_pre_dir)
|
||||
expanded_parts.append(exp)
|
||||
|
||||
if street_name:
|
||||
# Don't expand ordinals: "21st" stays "21st"
|
||||
if _ORDINAL_RE.match(street_name):
|
||||
expanded_parts.append(street_name)
|
||||
else:
|
||||
# Expand directional abbreviation if it IS the street name
|
||||
exp = _DIRECTIONAL_ABBREVS.get(street_name.lower(), street_name)
|
||||
expanded_parts.append(exp)
|
||||
parsed['street'] = street_name
|
||||
|
||||
if street_post_type:
|
||||
if _ORDINAL_RE.match(street_post_type):
|
||||
expanded_parts.append(street_post_type)
|
||||
else:
|
||||
exp = _STREET_TYPE_ABBREVS.get(street_post_type.lower(), street_post_type)
|
||||
expanded_parts.append(exp)
|
||||
|
||||
# Build raw street (original abbreviations, for Netsyms) and expanded (for Photon)
|
||||
raw_street_parts = []
|
||||
if street_pre_dir:
|
||||
raw_street_parts.append(street_pre_dir)
|
||||
if street_name:
|
||||
raw_street_parts.append(street_name)
|
||||
if street_post_type:
|
||||
raw_street_parts.append(street_post_type)
|
||||
parsed['street_raw'] = ' '.join(raw_street_parts)
|
||||
|
||||
# Build the full expanded street
|
||||
if expanded_parts:
|
||||
# The street is everything after the number
|
||||
street_full = ' '.join(expanded_parts[1:] if number else expanded_parts)
|
||||
parsed['street'] = street_full
|
||||
|
||||
if place:
|
||||
parsed['city'] = place
|
||||
expanded_parts.append(place)
|
||||
if state:
|
||||
parsed['state'] = state.upper()
|
||||
expanded_parts.append(state)
|
||||
if zipcode:
|
||||
parsed['zipcode'] = zipcode
|
||||
expanded_parts.append(zipcode)
|
||||
|
||||
parsed['expanded_query'] = ' '.join(expanded_parts)
|
||||
|
||||
# ── Intent classification ──
|
||||
if addr_type == 'Street Address' and number:
|
||||
return 'ADDRESS', parsed
|
||||
elif zipcode and not number and not street_name:
|
||||
return 'POSTCODE', parsed
|
||||
elif addr_type == 'Ambiguous':
|
||||
# Check if it looks like a locality: last token(s) are a state code or name
|
||||
tokens = q.replace(',', ' ').split()
|
||||
if len(tokens) >= 2:
|
||||
last_upper = tokens[-1].upper()
|
||||
if last_upper in _STATE_CODES:
|
||||
parsed['city'] = ' '.join(tokens[:-1])
|
||||
parsed['state'] = last_upper
|
||||
return 'LOCALITY', parsed
|
||||
# Check full state names (single-word like "idaho" or two-word like "new york")
|
||||
last_lower = tokens[-1].lower()
|
||||
if last_lower in _STATE_NAME_TO_CODE:
|
||||
parsed['city'] = ' '.join(tokens[:-1])
|
||||
parsed['state'] = _STATE_NAME_TO_CODE[last_lower]
|
||||
return 'LOCALITY', parsed
|
||||
if len(tokens) >= 3:
|
||||
two_word = f"{tokens[-2].lower()} {last_lower}"
|
||||
if two_word in _STATE_NAME_TO_CODE:
|
||||
parsed['city'] = ' '.join(tokens[:-2])
|
||||
parsed['state'] = _STATE_NAME_TO_CODE[two_word]
|
||||
return 'LOCALITY', parsed
|
||||
return 'UNKNOWN', parsed
|
||||
else:
|
||||
return 'UNKNOWN', parsed
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════════
|
||||
# STEP 2: RETRIEVAL
|
||||
# ═══════════════════════════════════════════════════════════════════
|
||||
|
||||
def _retrieve_netsyms(parsed, limit=10, lat=None, lon=None):
|
||||
"""Query Netsyms for structured address lookup. Returns list of candidate dicts."""
|
||||
try:
|
||||
from . import netsyms
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
results = []
|
||||
number = parsed.get('number', '')
|
||||
street = parsed.get('street_raw') or parsed.get('street', '')
|
||||
city = parsed.get('city', '')
|
||||
state = parsed.get('state', '')
|
||||
zipcode = parsed.get('zipcode', '')
|
||||
|
||||
# When viewport provided, fetch more results to sort from
|
||||
fetch_limit = 200 if (lat is not None and lon is not None) else limit
|
||||
|
||||
if number and street:
|
||||
rows = netsyms.lookup_by_street(
|
||||
number, street, city=city, state=state, zipcode=zipcode, limit=fetch_limit
|
||||
)
|
||||
elif zipcode:
|
||||
rows = netsyms.lookup_by_zipcode(zipcode, limit=fetch_limit)
|
||||
else:
|
||||
return []
|
||||
|
||||
for row in rows:
|
||||
addr_parts = [row['number'], row['street']]
|
||||
if row.get('street2'):
|
||||
addr_parts.append(row['street2'])
|
||||
addr_parts.extend([row['city'], row['state'], row['zipcode']])
|
||||
display = ' '.join(p for p in addr_parts if p)
|
||||
results.append({
|
||||
'name': display,
|
||||
'lat': row['lat'],
|
||||
'lon': row['lon'],
|
||||
'source': 'netsyms',
|
||||
'type': 'street_address',
|
||||
'raw': row,
|
||||
'_number': row.get('number', ''),
|
||||
'_street': row.get('street', ''),
|
||||
'_city': row.get('city', ''),
|
||||
'_state': row.get('state', ''),
|
||||
})
|
||||
# Sort by viewport distance if lat/lon provided, then limit
|
||||
if lat is not None and lon is not None and results:
|
||||
results.sort(key=lambda r: (r["lat"] - lat)**2 + (r["lon"] - lon)**2)
|
||||
results = results[:limit]
|
||||
return results
|
||||
|
||||
|
||||
def _retrieve_photon_structured(parsed, limit=10):
|
||||
"""Query Photon /structured endpoint for address lookup."""
|
||||
params = {'limit': limit, 'countrycode': 'US'}
|
||||
if parsed.get('street'):
|
||||
params['street'] = parsed['street']
|
||||
if parsed.get('number'):
|
||||
params['housenumber'] = parsed['number']
|
||||
if parsed.get('city'):
|
||||
params['city'] = parsed['city']
|
||||
if parsed.get('state'):
|
||||
params['state'] = parsed['state']
|
||||
|
||||
if 'street' not in params:
|
||||
return []
|
||||
|
||||
try:
|
||||
resp = requests.get(f"{photon_url()}/structured", params=params, timeout=5)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
except Exception as e:
|
||||
logger.debug("Photon /structured failed: %s", e)
|
||||
return []
|
||||
|
||||
return _parse_photon_features(data.get('features', []), 'photon')
|
||||
|
||||
|
||||
def _retrieve_photon_freetext(query, limit=10, lat=None, lon=None, zoom=None):
|
||||
"""Query Photon /api for free-text search with location bias."""
|
||||
try:
|
||||
params = {
|
||||
'q': query,
|
||||
'limit': limit,
|
||||
'lat': lat if lat is not None else GEOCODE_BIAS_LAT,
|
||||
'lon': lon if lon is not None else GEOCODE_BIAS_LON,
|
||||
'zoom': int(zoom) if zoom is not None else GEOCODE_BIAS_ZOOM,
|
||||
}
|
||||
resp = requests.get(f"{photon_url()}/api", params=params, timeout=5)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
except Exception as e:
|
||||
return []
|
||||
|
||||
return _parse_photon_features(data.get('features', []), 'photon')
|
||||
|
||||
|
||||
def _parse_photon_features(features, source):
|
||||
"""Convert Photon GeoJSON features to candidate dicts."""
|
||||
results = []
|
||||
for i, feature in enumerate(features):
|
||||
props = feature.get('properties', {})
|
||||
coords = feature.get('geometry', {}).get('coordinates', [0, 0])
|
||||
|
||||
osm_key = props.get('osm_key', '')
|
||||
osm_value = props.get('osm_value', '')
|
||||
feat_type = props.get('type', '')
|
||||
has_hn = bool(props.get('housenumber'))
|
||||
|
||||
if osm_key in ('amenity', 'shop', 'tourism', 'leisure', 'office'):
|
||||
rtype = 'poi'
|
||||
elif has_hn or osm_value in ('house', 'residential'):
|
||||
rtype = 'street_address'
|
||||
elif feat_type in ('city', 'town', 'village', 'hamlet', 'county', 'state', 'country'):
|
||||
rtype = 'locality'
|
||||
else:
|
||||
rtype = 'poi'
|
||||
|
||||
# Build display name
|
||||
parts = []
|
||||
hn = props.get('housenumber')
|
||||
street = props.get('street')
|
||||
name = props.get('name', '')
|
||||
if hn and street:
|
||||
parts.append(f"{hn} {street}")
|
||||
if name and name != street:
|
||||
parts.append(name)
|
||||
elif name:
|
||||
parts.append(name)
|
||||
elif street:
|
||||
parts.append(street)
|
||||
for key in ('city', 'county', 'state', 'country'):
|
||||
v = props.get(key)
|
||||
if v and (not parts or v != parts[-1]):
|
||||
parts.append(v)
|
||||
display = ', '.join(p for p in parts if p) or 'Unknown'
|
||||
|
||||
results.append({
|
||||
'name': display,
|
||||
'lat': coords[1],
|
||||
'lon': coords[0],
|
||||
'source': source,
|
||||
'type': rtype,
|
||||
'raw': props,
|
||||
'_photon_rank': i,
|
||||
'_number': props.get('housenumber', ''),
|
||||
'_street': props.get('street', ''),
|
||||
# For locality results, the name IS the city (Photon omits 'city' on city-type features)
|
||||
'_city': props.get('city', '') or (props.get('name', '') if rtype == 'locality' else ''),
|
||||
'_state': props.get('state', ''),
|
||||
})
|
||||
return results
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════════
|
||||
# STEP 3: RERANKER
|
||||
# ═══════════════════════════════════════════════════════════════════
|
||||
|
||||
def _expand_street_type(s):
|
||||
"""Expand a street type abbreviation for comparison."""
|
||||
return _STREET_TYPE_ABBREVS.get(s.lower(), s.lower())
|
||||
|
||||
|
||||
def _score_candidate(candidate, parsed, intent):
|
||||
"""
|
||||
Score a candidate against the parsed query.
|
||||
Returns (total_score, signal_breakdown_dict).
|
||||
"""
|
||||
signals = {}
|
||||
total = 0.0
|
||||
|
||||
query_number = (parsed.get('number') or '').strip().upper()
|
||||
query_street = (parsed.get('street') or '').strip().upper()
|
||||
query_city = (parsed.get('city') or '').strip().upper()
|
||||
query_state = (parsed.get('state') or '').strip().upper()
|
||||
|
||||
cand_number = (candidate.get('_number') or '').strip().upper()
|
||||
cand_street = (candidate.get('_street') or '').strip().upper()
|
||||
cand_city = (candidate.get('_city') or '').strip().upper()
|
||||
cand_state = (candidate.get('_state') or '').strip().upper()
|
||||
|
||||
# ── Housenumber ──
|
||||
if intent == 'ADDRESS' and query_number:
|
||||
if cand_number == query_number:
|
||||
signals['housenumber_exact'] = W_HOUSENUMBER_EXACT
|
||||
total += W_HOUSENUMBER_EXACT
|
||||
elif cand_number and cand_number != query_number:
|
||||
signals['housenumber_mismatch'] = W_HOUSENUMBER_MISMATCH
|
||||
total += W_HOUSENUMBER_MISMATCH
|
||||
|
||||
# ── Street name fuzz ──
|
||||
if query_street and cand_street:
|
||||
# Expand both for comparison
|
||||
q_expanded = ' '.join(_expand_street_type(t) for t in query_street.split())
|
||||
c_expanded = ' '.join(_expand_street_type(t) for t in cand_street.split())
|
||||
ratio = fuzz.token_sort_ratio(q_expanded, c_expanded) / 100.0
|
||||
score = ratio * W_STREET_NAME_FUZZ
|
||||
signals['street_name_fuzz'] = round(score, 2)
|
||||
total += score
|
||||
|
||||
# ── Street type match ──
|
||||
if query_street and cand_street:
|
||||
q_tokens = set(_expand_street_type(t) for t in query_street.split())
|
||||
c_tokens = set(_expand_street_type(t) for t in cand_street.split())
|
||||
# Check if the street type words overlap
|
||||
street_types = set(_STREET_TYPE_ABBREVS.values())
|
||||
q_types = q_tokens & street_types
|
||||
c_types = c_tokens & street_types
|
||||
if q_types and q_types & c_types:
|
||||
signals['street_type_match'] = W_STREET_TYPE_MATCH
|
||||
total += W_STREET_TYPE_MATCH
|
||||
|
||||
# ── Token coverage ──
|
||||
raw_q = parsed.get('raw_query', '').upper()
|
||||
q_tokens = set(raw_q.replace(',', ' ').split())
|
||||
if q_tokens:
|
||||
cand_text = candidate.get('name', '').upper()
|
||||
matched = sum(1 for t in q_tokens if t in cand_text)
|
||||
coverage = matched / len(q_tokens)
|
||||
score = coverage * W_TOKEN_COVERAGE
|
||||
signals['token_coverage'] = round(score, 2)
|
||||
total += score
|
||||
|
||||
# ── Locality fuzz ──
|
||||
if query_city and cand_city:
|
||||
ratio = fuzz.ratio(query_city, cand_city) / 100.0
|
||||
score = ratio * W_LOCALITY_FUZZ
|
||||
signals['locality_fuzz'] = round(score, 2)
|
||||
total += score
|
||||
|
||||
# ── State exact ──
|
||||
if query_state and cand_state:
|
||||
if cand_state == query_state:
|
||||
signals['state_exact'] = W_STATE_EXACT
|
||||
total += W_STATE_EXACT
|
||||
|
||||
# ── Source authority ──
|
||||
if candidate.get('source') == 'netsyms' and intent == 'ADDRESS':
|
||||
signals['source_authority'] = W_SOURCE_AUTHORITY
|
||||
total += W_SOURCE_AUTHORITY
|
||||
|
||||
# ── Layer rank (type-appropriate bonus) ──
|
||||
cand_type = candidate.get('type', '')
|
||||
if intent == 'ADDRESS' and cand_type == 'street_address':
|
||||
signals['layer_rank'] = W_LAYER_RANK
|
||||
total += W_LAYER_RANK
|
||||
elif intent == 'LOCALITY' and cand_type == 'locality':
|
||||
signals['layer_rank'] = W_LAYER_RANK
|
||||
total += W_LAYER_RANK
|
||||
elif intent == 'POI' and cand_type == 'poi':
|
||||
signals['layer_rank'] = W_LAYER_RANK
|
||||
total += W_LAYER_RANK
|
||||
|
||||
# ── Photon position normalization ──
|
||||
photon_rank = candidate.get('_photon_rank')
|
||||
if photon_rank is not None:
|
||||
# Top result gets full bonus, decays linearly
|
||||
score = max(0, (1.0 - photon_rank / 10.0)) * W_PHOTON_POSITION_NORM
|
||||
signals['photon_position'] = round(score, 2)
|
||||
total += score
|
||||
|
||||
# ── Business intent POI boost ──
|
||||
# When the query has no road keywords (likely a business/POI search),
|
||||
# boost amenity/shop/etc results and penalize highway/route results.
|
||||
# Skipped for LOCALITY, POSTCODE, COORD queries where class is irrelevant.
|
||||
if intent not in ('LOCALITY', 'POSTCODE', 'COORD'):
|
||||
q_tokens_lower = set(parsed.get('raw_query', '').lower().replace(',', ' ').split())
|
||||
if not (q_tokens_lower & _ROAD_KEYWORDS):
|
||||
osm_key = (candidate.get('raw') or {}).get('osm_key', '')
|
||||
if osm_key in ('amenity', 'shop', 'tourism', 'leisure', 'office', 'craft'):
|
||||
signals['poi_class_boost'] = W_POI_CLASS_BOOST
|
||||
total += W_POI_CLASS_BOOST
|
||||
elif osm_key in ('highway', 'route'):
|
||||
signals['highway_class_penalty'] = W_HIGHWAY_CLASS_PENALTY
|
||||
total += W_HIGHWAY_CLASS_PENALTY
|
||||
|
||||
return round(total, 2), signals
|
||||
|
||||
|
||||
def _build_match_code(candidate, parsed, intent):
|
||||
"""Build a match_code dict indicating match quality for each field."""
|
||||
mc = {}
|
||||
if intent == 'ADDRESS':
|
||||
q_num = (parsed.get('number') or '').strip().upper()
|
||||
c_num = (candidate.get('_number') or '').strip().upper()
|
||||
if q_num and c_num == q_num:
|
||||
mc['housenumber'] = 'matched'
|
||||
elif q_num and c_num:
|
||||
mc['housenumber'] = 'unmatched'
|
||||
elif q_num and not c_num:
|
||||
mc['housenumber'] = 'inferred'
|
||||
|
||||
q_street = (parsed.get('street') or '').strip().upper()
|
||||
c_street = (candidate.get('_street') or '').strip().upper()
|
||||
if q_street and c_street:
|
||||
q_exp = ' '.join(_expand_street_type(t) for t in q_street.split())
|
||||
c_exp = ' '.join(_expand_street_type(t) for t in c_street.split())
|
||||
ratio = fuzz.token_sort_ratio(q_exp, c_exp) / 100.0
|
||||
mc['street'] = 'matched' if ratio > 0.8 else 'unmatched'
|
||||
elif q_street:
|
||||
mc['street'] = 'inferred'
|
||||
|
||||
q_city = (parsed.get('city') or '').strip().upper()
|
||||
c_city = (candidate.get('_city') or '').strip().upper()
|
||||
if q_city and c_city:
|
||||
ratio = fuzz.ratio(q_city, c_city) / 100.0
|
||||
mc['city'] = 'matched' if ratio > 0.8 else 'unmatched'
|
||||
elif q_city:
|
||||
mc['city'] = 'inferred'
|
||||
|
||||
return mc
|
||||
|
||||
|
||||
def _rerank(candidates, parsed, intent, query, limit):
|
||||
"""Score, sort, and trim candidates. Trace-log top 3."""
|
||||
scored = []
|
||||
for c in candidates:
|
||||
total, signals = _score_candidate(c, parsed, intent)
|
||||
c['_score'] = total
|
||||
c['_signals'] = signals
|
||||
scored.append(c)
|
||||
|
||||
scored.sort(key=lambda c: c['_score'], reverse=True)
|
||||
|
||||
# Trace log for audit
|
||||
_trace_logger.debug("─── Query: %r intent=%s ───", query, intent)
|
||||
for i, c in enumerate(scored):
|
||||
osm_key = (c.get('raw') or {}).get('osm_key', '—')
|
||||
osm_val = (c.get('raw') or {}).get('osm_value', '—')
|
||||
_trace_logger.debug(
|
||||
" #%d score=%.2f src=%s key=%s/%s name=%s",
|
||||
i, c['_score'], c.get('source', '?'), osm_key, osm_val,
|
||||
c.get('name', '?')[:60]
|
||||
)
|
||||
_trace_logger.debug(" signals=%s", c.get('_signals', {}))
|
||||
|
||||
# Clean internal fields and add match_code
|
||||
result = []
|
||||
for c in scored[:limit]:
|
||||
mc = _build_match_code(c, parsed, intent)
|
||||
|
||||
# Assign confidence from score
|
||||
score = c.get('_score', 0)
|
||||
if score >= 10:
|
||||
confidence = 'exact'
|
||||
elif score >= 5:
|
||||
confidence = 'high'
|
||||
elif score >= 2:
|
||||
confidence = 'medium'
|
||||
else:
|
||||
confidence = 'low'
|
||||
|
||||
entry = {
|
||||
'name': c['name'],
|
||||
'lat': c['lat'],
|
||||
'lon': c['lon'],
|
||||
'source': c['source'],
|
||||
'confidence': confidence,
|
||||
'type': c.get('type', 'poi'),
|
||||
'raw': c.get('raw'),
|
||||
}
|
||||
if mc:
|
||||
entry['match_code'] = mc
|
||||
result.append(entry)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════════
|
||||
# STEP 4: ANNOTATION
|
||||
# ═══════════════════════════════════════════════════════════════════
|
||||
|
||||
def _haversine_m(lat1, lon1, lat2, lon2):
|
||||
"""Haversine distance in meters."""
|
||||
R = 6_371_000
|
||||
rlat1, rlat2 = math.radians(lat1), math.radians(lat2)
|
||||
dlat = math.radians(lat2 - lat1)
|
||||
dlon = math.radians(lon2 - lon1)
|
||||
a = math.sin(dlat / 2) ** 2 + math.cos(rlat1) * math.cos(rlat2) * math.sin(dlon / 2) ** 2
|
||||
return R * 2 * math.atan2(math.sqrt(a), math.sqrt(1 - a))
|
||||
|
||||
|
||||
def _annotate_with_address_book(results):
|
||||
"""Add labeled_as to results within radius of an address book entry."""
|
||||
try:
|
||||
from . import address_book
|
||||
entries = address_book.load()
|
||||
except Exception:
|
||||
return
|
||||
for result in results:
|
||||
rlat, rlon = result.get('lat'), result.get('lon')
|
||||
if rlat is None or rlon is None:
|
||||
continue
|
||||
for entry in entries:
|
||||
elat, elon = entry.get('lat'), entry.get('lon')
|
||||
if elat is None or elon is None:
|
||||
continue
|
||||
if _haversine_m(rlat, rlon, elat, elon) <= ADDRESS_BOOK_ANNOTATION_RADIUS_M:
|
||||
result['labeled_as'] = entry['name']
|
||||
break
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════════
|
||||
# PUBLIC API
|
||||
# ═══════════════════════════════════════════════════════════════════
|
||||
|
||||
def geocode(query, limit=10, lat=None, lon=None, zoom=None):
|
||||
"""
|
||||
Structured geocoding with multi-source retrieval and reranking.
|
||||
|
||||
Returns {query, results: [...], count} — always 200-safe.
|
||||
"""
|
||||
limit = max(1, min(limit, 20))
|
||||
q = (query or '').strip()
|
||||
empty = {'query': q, 'results': [], 'count': 0}
|
||||
|
||||
if not q:
|
||||
return empty
|
||||
|
||||
# ── Coordinate detection ──
|
||||
coords = _parse_coords(q)
|
||||
if coords:
|
||||
return {
|
||||
'query': q,
|
||||
'results': [{
|
||||
'name': q,
|
||||
'lat': coords[0],
|
||||
'lon': coords[1],
|
||||
'source': 'coordinates',
|
||||
'confidence': 'exact',
|
||||
'type': 'coordinates',
|
||||
'raw': None,
|
||||
}],
|
||||
'count': 1,
|
||||
}
|
||||
|
||||
# ── Address book nickname short-circuit ──
|
||||
normalized_q = ' '.join(q.lower().replace(',', ' ').split())
|
||||
is_single_word = ' ' not in normalized_q
|
||||
try:
|
||||
from . import address_book
|
||||
ab_match = address_book.lookup(q)
|
||||
if (ab_match
|
||||
and ab_match['confidence'] == 'exact'
|
||||
and ab_match.get('lat') and ab_match.get('lon')
|
||||
and is_single_word):
|
||||
logger.info("geocode: nickname short-circuit %r → %s", q, ab_match['name'])
|
||||
return {
|
||||
'query': q,
|
||||
'results': [{
|
||||
'name': ab_match.get('address') or ab_match['name'],
|
||||
'lat': ab_match['lat'],
|
||||
'lon': ab_match['lon'],
|
||||
'source': 'address_book',
|
||||
'confidence': 'exact',
|
||||
'type': 'nickname',
|
||||
'raw': ab_match,
|
||||
}],
|
||||
'count': 1,
|
||||
}
|
||||
except Exception as e:
|
||||
logger.debug("geocode: address_book lookup failed: %s", e)
|
||||
|
||||
# ── Classify intent + parse ──
|
||||
intent, parsed = _classify_and_parse(q)
|
||||
logger.debug("geocode: intent=%s parsed=%s", intent, parsed)
|
||||
|
||||
# ── Retrieve candidates ──
|
||||
candidates = []
|
||||
|
||||
if intent == 'ADDRESS':
|
||||
# Parallel: Netsyms (structured) + Photon (freetext with expanded query)
|
||||
netsyms_results = _retrieve_netsyms(parsed, limit=limit, lat=lat, lon=lon)
|
||||
photon_results = _retrieve_photon_freetext(
|
||||
parsed.get('expanded_query', q), limit=limit, lat=lat, lon=lon, zoom=zoom
|
||||
)
|
||||
# Also try Photon /structured for addresses
|
||||
photon_struct = _retrieve_photon_structured(parsed, limit=5)
|
||||
candidates = netsyms_results + photon_results + photon_struct
|
||||
|
||||
elif intent == 'POSTCODE':
|
||||
netsyms_results = _retrieve_netsyms(parsed, limit=limit, lat=lat, lon=lon)
|
||||
photon_results = _retrieve_photon_freetext(q, limit=limit, lat=lat, lon=lon, zoom=zoom)
|
||||
candidates = netsyms_results + photon_results
|
||||
|
||||
elif intent in ('LOCALITY', 'POI', 'UNKNOWN'):
|
||||
candidates = _retrieve_photon_freetext(q, limit=limit, lat=lat, lon=lon, zoom=zoom)
|
||||
|
||||
# ── Deduplicate by (lat, lon) proximity ──
|
||||
deduped = []
|
||||
for c in candidates:
|
||||
is_dup = False
|
||||
for existing in deduped:
|
||||
if (_haversine_m(c['lat'], c['lon'], existing['lat'], existing['lon']) < 50
|
||||
and c.get('source') == existing.get('source')):
|
||||
is_dup = True
|
||||
break
|
||||
if not is_dup:
|
||||
deduped.append(c)
|
||||
candidates = deduped
|
||||
|
||||
# ── Rerank ──
|
||||
results = _rerank(candidates, parsed, intent, q, limit)
|
||||
|
||||
# ── Address book annotation ──
|
||||
_annotate_with_address_book(results)
|
||||
|
||||
logger.info("geocode: %r → intent=%s, %d results", q, intent, len(results))
|
||||
return {'query': q, 'results': results, 'count': len(results)}
|
||||
45
backend/services/navi_geo/landclass_client.py
Normal file
45
backend/services/navi_geo/landclass_client.py
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
"""HTTP client for navi-landclass — the first navi→navi-landclass coupling.
|
||||
|
||||
Phase A §5 locked this edge: recon's reverse bundle called
|
||||
``landclass.lookup_landclass`` + ``format_summary`` in-process and merged the
|
||||
most-specific unit name (a bare string) into ``bundle['landclass']``. navi-geo
|
||||
replaces that in-process call with an HTTP GET to navi-landclass
|
||||
``/api/landclass``, whose ``summary`` field is exactly that same string.
|
||||
|
||||
navi-landclass returns 200 even with PostGIS down or no coverage
|
||||
(``summary: null``), so this client only has to read ``.summary`` and let the
|
||||
caller's try/except turn any transport error into ``None``.
|
||||
"""
|
||||
import os
|
||||
|
||||
import requests
|
||||
|
||||
DEFAULT_LANDCLASS_URL = 'http://127.0.0.1:8424'
|
||||
|
||||
# 5s covers a local in-DC HTTP round-trip plus the underlying PostGIS query.
|
||||
# PAD-US lookups can be slow on points with many overlapping polygons
|
||||
# (Yosemite has ~30 overlapping units), so don't tighten below ~3s without
|
||||
# checking the slow-end p99. Recon's in-process call had no timeout because it
|
||||
# was a direct DB call inside the same process; the HTTP boundary needs one.
|
||||
LANDCLASS_TIMEOUT_S = 5
|
||||
|
||||
|
||||
def landclass_url():
|
||||
"""navi-landclass base URL, env-overridable via NAVI_LANDCLASS_URL."""
|
||||
return os.environ.get('NAVI_LANDCLASS_URL', DEFAULT_LANDCLASS_URL)
|
||||
|
||||
|
||||
def reverse_landclass_summary(lat, lon):
|
||||
"""Return the most-specific PAD-US unit name for a point, or None.
|
||||
|
||||
Mirrors recon's ``_reverse_landclass`` return contract (a string or None).
|
||||
Raises on transport/HTTP error so the bundle's per-component try/except can
|
||||
log a warning and leave ``landclass`` null — never a 5xx.
|
||||
"""
|
||||
resp = requests.get(
|
||||
f"{landclass_url()}/api/landclass",
|
||||
params={'lat': lat, 'lon': lon},
|
||||
timeout=LANDCLASS_TIMEOUT_S,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json().get('summary')
|
||||
250
backend/services/navi_geo/netsyms.py
Executable file
250
backend/services/navi_geo/netsyms.py
Executable file
|
|
@ -0,0 +1,250 @@
|
|||
"""
|
||||
navi-geo Netsyms AddressDatabase2025 — SQLite-backed US+CA address lookup.
|
||||
|
||||
Faithful port of recon's ``lib/netsyms.py``. Only change: the DB path is read
|
||||
from ``NAVI_NETSYMS_DB`` (recon hardcoded it). The ~35 GB file stays external
|
||||
and read-only.
|
||||
|
||||
Provides 159.78M geocoded addresses as tier-2 between address book
|
||||
(exact named locations) and Photon (full-text global geocoding).
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
import sqlite3
|
||||
import threading
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger('navi_geo.netsyms')
|
||||
|
||||
DEFAULT_DB_PATH = '/mnt/nav/addresses/AddressDatabase2025.sqlite'
|
||||
|
||||
|
||||
def db_path():
|
||||
"""Netsyms SQLite path, env-overridable via NAVI_NETSYMS_DB."""
|
||||
return os.environ.get('NAVI_NETSYMS_DB', DEFAULT_DB_PATH)
|
||||
|
||||
|
||||
_conn = None
|
||||
_lock = threading.Lock()
|
||||
_cached_row_count = None
|
||||
|
||||
|
||||
def reset_conn():
|
||||
"""Drop the cached connection + row count so the next call reopens (env may
|
||||
have changed). Used per app instance / per test."""
|
||||
global _conn, _cached_row_count
|
||||
with _lock:
|
||||
if _conn is not None:
|
||||
try:
|
||||
_conn.close()
|
||||
except Exception:
|
||||
pass
|
||||
_conn = None
|
||||
_cached_row_count = None
|
||||
|
||||
# US states + DC + territories, CA provinces, for free-text parsing
|
||||
_STATE_CODES = {
|
||||
'AL', 'AK', 'AZ', 'AR', 'CA', 'CO', 'CT', 'DE', 'FL', 'GA',
|
||||
'HI', 'ID', 'IL', 'IN', 'IA', 'KS', 'KY', 'LA', 'ME', 'MD',
|
||||
'MA', 'MI', 'MN', 'MS', 'MO', 'MT', 'NE', 'NV', 'NH', 'NJ',
|
||||
'NM', 'NY', 'NC', 'ND', 'OH', 'OK', 'OR', 'PA', 'RI', 'SC',
|
||||
'SD', 'TN', 'TX', 'UT', 'VT', 'VA', 'WA', 'WV', 'WI', 'WY',
|
||||
'DC', 'PR', 'VI', 'GU', 'AS', 'MP',
|
||||
# Canadian provinces
|
||||
'AB', 'BC', 'MB', 'NB', 'NL', 'NS', 'NT', 'NU', 'ON', 'PE',
|
||||
'QC', 'SK', 'YT',
|
||||
}
|
||||
|
||||
_NUMBER_RE = re.compile(r'^(\d+[\w-]*)(.*)$')
|
||||
|
||||
|
||||
def _get_conn():
|
||||
"""Lazy-open a read-only SQLite connection."""
|
||||
global _conn
|
||||
if _conn is not None:
|
||||
return _conn
|
||||
with _lock:
|
||||
if _conn is not None:
|
||||
return _conn
|
||||
path = db_path()
|
||||
uri = f'file:{path}?mode=ro'
|
||||
_conn = sqlite3.connect(uri, uri=True, check_same_thread=False)
|
||||
_conn.row_factory = sqlite3.Row
|
||||
logger.info("Netsyms DB opened: %s", path)
|
||||
return _conn
|
||||
|
||||
|
||||
def _row_to_dict(row):
|
||||
"""Convert a sqlite3.Row to a plain dict with lat/lon keys."""
|
||||
return {
|
||||
'zipcode': row['zipcode'],
|
||||
'number': row['number'],
|
||||
'street': row['street'],
|
||||
'street2': row['street2'],
|
||||
'city': row['city'],
|
||||
'state': row['state'],
|
||||
'plus4': row['plus4'],
|
||||
'country': row['country'],
|
||||
'lat': float(row['latitude']),
|
||||
'lon': float(row['longitude']),
|
||||
'source': row['source'],
|
||||
}
|
||||
|
||||
|
||||
def lookup_by_street(number, street, city=None, state=None,
|
||||
zipcode=None, country=None, limit=20):
|
||||
"""Match on number + street, with optional qualifiers."""
|
||||
conn = _get_conn()
|
||||
clauses = ['number = ?', 'street = ?']
|
||||
params = [str(number).strip().upper(), street.strip().upper()]
|
||||
|
||||
if city:
|
||||
clauses.append('city = ?')
|
||||
params.append(city.strip().upper())
|
||||
if state:
|
||||
clauses.append('state = ?')
|
||||
params.append(state.strip().upper())
|
||||
if zipcode:
|
||||
clauses.append('zipcode = ?')
|
||||
params.append(zipcode.strip())
|
||||
if country:
|
||||
clauses.append('country = ?')
|
||||
params.append(country.strip().upper())
|
||||
|
||||
sql = f"SELECT * FROM addresses WHERE {' AND '.join(clauses)} LIMIT ?"
|
||||
params.append(limit)
|
||||
|
||||
with _lock:
|
||||
try:
|
||||
rows = conn.execute(sql, params).fetchall()
|
||||
except sqlite3.Error as e:
|
||||
logger.warning("Netsyms lookup_by_street error: %s", e)
|
||||
return []
|
||||
|
||||
results = [_row_to_dict(r) for r in rows]
|
||||
logger.debug("lookup_by_street(%s, %s, city=%s, state=%s) → %d results",
|
||||
number, street, city, state, len(results))
|
||||
return results
|
||||
|
||||
|
||||
def lookup_free_text(query, country_hint=None):
|
||||
"""Parse a free-text address and look it up."""
|
||||
q = query.strip()
|
||||
if not q:
|
||||
return []
|
||||
|
||||
# Strip trailing zipcode if present
|
||||
zipcode = None
|
||||
zip_match = re.search(r'\b(\d{5})\s*$', q)
|
||||
if zip_match:
|
||||
zipcode = zip_match.group(1)
|
||||
q = q[:zip_match.start()].strip().rstrip(',').strip()
|
||||
|
||||
# Strip trailing state
|
||||
tokens = re.split(r'[,\s]+', q)
|
||||
tokens = [t for t in tokens if t]
|
||||
if not tokens:
|
||||
return []
|
||||
|
||||
state = None
|
||||
if len(tokens) >= 2 and tokens[-1].upper() in _STATE_CODES:
|
||||
state = tokens[-1].upper()
|
||||
tokens = tokens[:-1]
|
||||
|
||||
# Leading digits → number
|
||||
number = None
|
||||
if tokens and re.match(r'^\d', tokens[0]):
|
||||
number = tokens[0]
|
||||
tokens = tokens[1:]
|
||||
|
||||
if not tokens:
|
||||
# Only a number, or empty — try zipcode if we have one
|
||||
if zipcode:
|
||||
return lookup_by_zipcode(zipcode, limit=20)
|
||||
return []
|
||||
|
||||
# If state was found and we have 2+ tokens remaining, last token is city
|
||||
city = None
|
||||
if state and len(tokens) >= 2:
|
||||
city = tokens[-1]
|
||||
tokens = tokens[:-1]
|
||||
|
||||
street = ' '.join(tokens)
|
||||
|
||||
if number:
|
||||
results = lookup_by_street(number, street, city=city, state=state,
|
||||
zipcode=zipcode, country=country_hint)
|
||||
if results:
|
||||
logger.debug("lookup_free_text(%r) → %d results via street match",
|
||||
query, len(results))
|
||||
return results
|
||||
|
||||
# Fallback: try zipcode only if available
|
||||
if zipcode:
|
||||
return lookup_by_zipcode(zipcode, limit=20)
|
||||
|
||||
logger.debug("lookup_free_text(%r) → 0 results", query)
|
||||
return []
|
||||
|
||||
|
||||
def lookup_by_zipcode(zipcode, limit=100):
|
||||
"""Direct zipcode lookup."""
|
||||
conn = _get_conn()
|
||||
sql = "SELECT * FROM addresses WHERE zipcode = ? LIMIT ?"
|
||||
params = [zipcode.strip(), limit]
|
||||
|
||||
with _lock:
|
||||
try:
|
||||
rows = conn.execute(sql, params).fetchall()
|
||||
except sqlite3.Error as e:
|
||||
logger.warning("Netsyms lookup_by_zipcode error: %s", e)
|
||||
return []
|
||||
|
||||
results = [_row_to_dict(r) for r in rows]
|
||||
logger.debug("lookup_by_zipcode(%s) → %d results", zipcode, len(results))
|
||||
return results
|
||||
|
||||
|
||||
def health():
|
||||
"""Health check with cached row count."""
|
||||
global _cached_row_count
|
||||
|
||||
try:
|
||||
file_size = os.path.getsize(db_path())
|
||||
except OSError:
|
||||
return {'ok': False, 'row_count': 0, 'file_size_bytes': 0,
|
||||
'indexed_countries': []}
|
||||
|
||||
try:
|
||||
conn = _get_conn()
|
||||
except Exception:
|
||||
return {'ok': False, 'row_count': 0, 'file_size_bytes': file_size,
|
||||
'indexed_countries': []}
|
||||
|
||||
if _cached_row_count is None:
|
||||
with _lock:
|
||||
if _cached_row_count is None:
|
||||
try:
|
||||
row = conn.execute(
|
||||
"SELECT COUNT(*) AS cnt FROM addresses"
|
||||
).fetchone()
|
||||
_cached_row_count = row['cnt']
|
||||
except sqlite3.Error:
|
||||
_cached_row_count = 0
|
||||
|
||||
with _lock:
|
||||
try:
|
||||
rows = conn.execute(
|
||||
"SELECT DISTINCT country FROM addresses"
|
||||
).fetchall()
|
||||
countries = sorted(r['country'] for r in rows)
|
||||
except sqlite3.Error:
|
||||
countries = []
|
||||
|
||||
return {
|
||||
'ok': True,
|
||||
'row_count': _cached_row_count,
|
||||
'file_size_bytes': file_size,
|
||||
'indexed_countries': countries,
|
||||
}
|
||||
0
backend/services/navi_geo/tests/__init__.py
Normal file
0
backend/services/navi_geo/tests/__init__.py
Normal file
243
backend/services/navi_geo/tests/test_geocode.py
Normal file
243
backend/services/navi_geo/tests/test_geocode.py
Normal file
|
|
@ -0,0 +1,243 @@
|
|||
"""Hermetic unit tests for the navi-geo geocode engine.
|
||||
|
||||
Phase A §"Tests" flagged that recon's geocode_test.py is a *live* smoke test
|
||||
(hits localhost:8420 + Photon + Netsyms). These are the CI-friendly equivalents:
|
||||
they test the intent classifier, the reranker scoring, match-code building,
|
||||
dedup, and the two short-circuits — with every upstream mocked. Assertions test
|
||||
*meaning* (ordering, classification, signal presence), not magic score numbers.
|
||||
"""
|
||||
import pytest
|
||||
|
||||
import services.navi_geo.geocode as gc
|
||||
from services.navi_geo.app import create_app
|
||||
|
||||
|
||||
# ── Intent classification + parsing ──────────────────────────────────────
|
||||
|
||||
def test_classify_street_address():
|
||||
intent, parsed = gc._classify_and_parse("214 North St, Filer, ID")
|
||||
assert intent == 'ADDRESS'
|
||||
assert parsed['number'] == '214'
|
||||
assert parsed['state'] == 'ID'
|
||||
|
||||
|
||||
def test_classify_coordinates():
|
||||
intent, _ = gc._classify_and_parse("43.6150, -116.2023")
|
||||
assert intent == 'COORD'
|
||||
|
||||
|
||||
def test_classify_locality_from_state_suffix():
|
||||
intent, parsed = gc._classify_and_parse("Filer ID")
|
||||
assert intent == 'LOCALITY'
|
||||
assert parsed['state'] == 'ID'
|
||||
assert parsed['city'] == 'Filer'
|
||||
|
||||
|
||||
def test_classify_full_state_name():
|
||||
intent, parsed = gc._classify_and_parse("Boise Idaho")
|
||||
assert intent == 'LOCALITY'
|
||||
assert parsed['state'] == 'ID'
|
||||
|
||||
|
||||
def test_street_type_abbreviation_expands_in_query():
|
||||
# "st" must expand to "street" in the Photon-bound expanded query, while the
|
||||
# raw street (for Netsyms) keeps the original abbreviation.
|
||||
_, parsed = gc._classify_and_parse("100 Main St, Boise, ID")
|
||||
assert 'street' in parsed['expanded_query'].lower()
|
||||
assert 'st' in parsed['street_raw'].lower().split()
|
||||
|
||||
|
||||
# ── Reranker scoring (relative, not magic numbers) ────────────────────────
|
||||
|
||||
def _addr_parsed(number='214', street='NORTH', city='FILER', state='ID'):
|
||||
return {'number': number, 'street': street, 'city': city, 'state': state,
|
||||
'raw_query': f'{number} {street} {city} {state}'}
|
||||
|
||||
|
||||
def test_exact_housenumber_outranks_mismatch():
|
||||
parsed = _addr_parsed()
|
||||
exact = {'_number': '214', '_street': 'NORTH', '_city': 'FILER', '_state': 'ID',
|
||||
'name': '214 North', 'source': 'netsyms', 'type': 'street_address', 'raw': {}}
|
||||
wrong = {'_number': '999', '_street': 'NORTH', '_city': 'FILER', '_state': 'ID',
|
||||
'name': '999 North', 'source': 'netsyms', 'type': 'street_address', 'raw': {}}
|
||||
s_exact, sig_exact = gc._score_candidate(exact, parsed, 'ADDRESS')
|
||||
s_wrong, _ = gc._score_candidate(wrong, parsed, 'ADDRESS')
|
||||
assert s_exact > s_wrong
|
||||
assert 'housenumber_exact' in sig_exact
|
||||
|
||||
|
||||
def test_netsyms_source_authority_only_for_address_intent():
|
||||
parsed = _addr_parsed()
|
||||
cand = {'_number': '214', '_street': 'NORTH', '_city': 'FILER', '_state': 'ID',
|
||||
'name': '214 North', 'source': 'netsyms', 'type': 'street_address', 'raw': {}}
|
||||
_, sig_addr = gc._score_candidate(cand, parsed, 'ADDRESS')
|
||||
_, sig_poi = gc._score_candidate(cand, parsed, 'POI')
|
||||
assert 'source_authority' in sig_addr
|
||||
assert 'source_authority' not in sig_poi
|
||||
|
||||
|
||||
def test_poi_class_boost_and_highway_penalty_for_business_query():
|
||||
parsed = {'raw_query': 'joes coffee'} # no road keywords
|
||||
shop = {'name': 'Joes Coffee', 'source': 'photon', 'type': 'poi',
|
||||
'raw': {'osm_key': 'amenity'}}
|
||||
road = {'name': 'Joes Coffee Rd', 'source': 'photon', 'type': 'poi',
|
||||
'raw': {'osm_key': 'highway'}}
|
||||
s_shop, sig_shop = gc._score_candidate(shop, parsed, 'POI')
|
||||
s_road, sig_road = gc._score_candidate(road, parsed, 'POI')
|
||||
assert 'poi_class_boost' in sig_shop
|
||||
assert 'highway_class_penalty' in sig_road
|
||||
assert s_shop > s_road
|
||||
|
||||
|
||||
def test_match_code_housenumber_matched_vs_unmatched():
|
||||
parsed = _addr_parsed()
|
||||
matched = gc._build_match_code({'_number': '214', '_street': 'NORTH', '_city': 'FILER'},
|
||||
parsed, 'ADDRESS')
|
||||
unmatched = gc._build_match_code({'_number': '999', '_street': 'NORTH', '_city': 'FILER'},
|
||||
parsed, 'ADDRESS')
|
||||
assert matched['housenumber'] == 'matched'
|
||||
assert unmatched['housenumber'] == 'unmatched'
|
||||
|
||||
|
||||
# ── geocode() short-circuits + retrieval (upstreams mocked) ───────────────
|
||||
|
||||
def test_geocode_empty_query_returns_empty():
|
||||
assert gc.geocode("") == {'query': '', 'results': [], 'count': 0}
|
||||
|
||||
|
||||
def test_geocode_coordinate_short_circuit_no_upstream(monkeypatch):
|
||||
# A coordinate string must not touch Photon/Netsyms/address_book at all.
|
||||
monkeypatch.setattr(gc.requests, 'get', lambda *a, **k: pytest.fail("no upstream"))
|
||||
out = gc.geocode("43.6150, -116.2023")
|
||||
assert out['count'] == 1
|
||||
r = out['results'][0]
|
||||
assert r['source'] == 'coordinates' and r['type'] == 'coordinates'
|
||||
assert r['lat'] == 43.6150 and r['lon'] == -116.2023
|
||||
|
||||
|
||||
def test_geocode_nickname_short_circuit(monkeypatch):
|
||||
import services.navi_geo.address_book as ab
|
||||
monkeypatch.setattr(ab, 'lookup', lambda q: {
|
||||
'name': 'Home', 'address': '1 Main St', 'lat': 43.6, 'lon': -116.2,
|
||||
'confidence': 'exact'})
|
||||
monkeypatch.setattr(gc.requests, 'get', lambda *a, **k: pytest.fail("no upstream"))
|
||||
out = gc.geocode("home") # single word + exact -> short-circuit
|
||||
assert out['count'] == 1
|
||||
assert out['results'][0]['source'] == 'address_book'
|
||||
assert out['results'][0]['type'] == 'nickname'
|
||||
|
||||
|
||||
def test_geocode_address_ranks_exact_housenumber_first(monkeypatch):
|
||||
import services.navi_geo.netsyms as ns
|
||||
import services.navi_geo.address_book as ab
|
||||
monkeypatch.setattr(ab, 'lookup', lambda q: None)
|
||||
monkeypatch.setattr(ab, 'load', lambda: [])
|
||||
# Netsyms returns the exact match; Photon returns a wrong-number distractor.
|
||||
monkeypatch.setattr(ns, 'lookup_by_street', lambda *a, **k: [{
|
||||
'number': '214', 'street': 'NORTH', 'street2': None, 'city': 'FILER',
|
||||
'state': 'ID', 'zipcode': '83328', 'lat': 42.57, 'lon': -114.6,
|
||||
'source': 'netsyms'}])
|
||||
|
||||
class FakeResp:
|
||||
status_code = 200
|
||||
|
||||
def raise_for_status(self):
|
||||
pass
|
||||
|
||||
def json(self):
|
||||
return {'features': [{
|
||||
'properties': {'housenumber': '999', 'street': 'North',
|
||||
'city': 'Filer', 'state': 'ID', 'osm_key': 'place'},
|
||||
'geometry': {'coordinates': [-114.61, 42.58]}}]}
|
||||
monkeypatch.setattr(gc.requests, 'get', lambda *a, **k: FakeResp())
|
||||
|
||||
out = gc.geocode("214 North St, Filer, ID", limit=10)
|
||||
assert out['count'] >= 1
|
||||
top = out['results'][0]
|
||||
assert top['source'] == 'netsyms' # exact-housenumber netsyms wins
|
||||
assert top['confidence'] in ('exact', 'high')
|
||||
|
||||
|
||||
def test_geocode_dedup_collapses_near_duplicates(monkeypatch):
|
||||
import services.navi_geo.address_book as ab
|
||||
monkeypatch.setattr(ab, 'lookup', lambda q: None)
|
||||
monkeypatch.setattr(ab, 'load', lambda: [])
|
||||
|
||||
class FakeResp:
|
||||
status_code = 200
|
||||
|
||||
def raise_for_status(self):
|
||||
pass
|
||||
|
||||
def json(self):
|
||||
# Two features ~0 m apart, same source -> dedup to one.
|
||||
f = {'properties': {'name': 'Park', 'osm_key': 'leisure'},
|
||||
'geometry': {'coordinates': [-116.2, 43.6]}}
|
||||
return {'features': [f, dict(f)]}
|
||||
monkeypatch.setattr(gc.requests, 'get', lambda *a, **k: FakeResp())
|
||||
out = gc.geocode("park", limit=10)
|
||||
assert out['count'] == 1
|
||||
|
||||
|
||||
# ── Trace logger is opt-in (Phase B locked decision #3) ───────────────────
|
||||
|
||||
def test_trace_logger_off_by_default(monkeypatch):
|
||||
monkeypatch.delenv('NAVI_GEO_RERANK_TRACE_LOG', raising=False)
|
||||
gc.setup_trace_logger()
|
||||
assert not any(isinstance(h, gc.logging.FileHandler) for h in gc._trace_logger.handlers)
|
||||
|
||||
|
||||
def test_trace_logger_attaches_when_env_set(tmp_path, monkeypatch):
|
||||
path = tmp_path / 'trace.log'
|
||||
monkeypatch.setenv('NAVI_GEO_RERANK_TRACE_LOG', str(path))
|
||||
gc.setup_trace_logger()
|
||||
assert any(isinstance(h, gc.logging.FileHandler) for h in gc._trace_logger.handlers)
|
||||
monkeypatch.delenv('NAVI_GEO_RERANK_TRACE_LOG', raising=False)
|
||||
gc.setup_trace_logger() # restore default-off for other tests
|
||||
|
||||
|
||||
# ── admin-info: no secrets, expected probes ───────────────────────────────
|
||||
|
||||
def test_admin_info_has_no_secrets_and_two_probes(monkeypatch):
|
||||
# require_auth needs the Authentik header; the edge would supply it.
|
||||
client = create_app().test_client()
|
||||
resp = client.get('/api/admin/navi-geo/info',
|
||||
headers={'X-Authentik-Username': 'matt'})
|
||||
assert resp.status_code == 200
|
||||
info = resp.get_json()
|
||||
assert info['service'] == 'navi-geo' and info['port'] == 8426
|
||||
# No masked secrets present (Phase A §10) — no value contains the mask marker.
|
||||
assert all('...' not in str(e['value']) and e['value'] != '****' for e in info['env'])
|
||||
names = {d['name'] for d in info['dependencies']}
|
||||
assert names == {'photon', 'navi-landclass'}
|
||||
|
||||
|
||||
def test_admin_info_requires_auth():
|
||||
client = create_app().test_client()
|
||||
assert client.get('/api/admin/navi-geo/info').status_code == 401
|
||||
|
||||
|
||||
def test_admin_info_netsyms_entry_enriched_with_health(tmp_path, monkeypatch):
|
||||
# netsyms.health() is wired into the netsyms filesystem entry (review fix #3):
|
||||
# the entry carries row_count / file_size_bytes / indexed_countries on top of
|
||||
# the standard path/exists/readable. Use a tiny real sqlite so health() runs.
|
||||
import sqlite3
|
||||
db = tmp_path / 'netsyms.sqlite'
|
||||
con = sqlite3.connect(db)
|
||||
con.execute('CREATE TABLE addresses (country TEXT)')
|
||||
con.executemany('INSERT INTO addresses (country) VALUES (?)',
|
||||
[('US',), ('US',), ('CA',)])
|
||||
con.commit()
|
||||
con.close()
|
||||
monkeypatch.setenv('NAVI_NETSYMS_DB', str(db))
|
||||
|
||||
client = create_app().test_client() # reset_conn() picks up the new path
|
||||
resp = client.get('/api/admin/navi-geo/info',
|
||||
headers={'X-Authentik-Username': 'matt'})
|
||||
assert resp.status_code == 200
|
||||
fs = resp.get_json()['filesystem']
|
||||
netsyms_entry = next(e for e in fs if e['path'] == str(db))
|
||||
assert netsyms_entry['ok'] is True
|
||||
assert netsyms_entry['row_count'] == 3
|
||||
assert netsyms_entry['file_size_bytes'] > 0
|
||||
assert set(netsyms_entry['indexed_countries']) == {'US', 'CA'}
|
||||
193
backend/services/navi_geo/tests/test_reverse_bundle.py
Normal file
193
backend/services/navi_geo/tests/test_reverse_bundle.py
Normal file
|
|
@ -0,0 +1,193 @@
|
|||
"""Tests for the /api/reverse/<lat>/<lon> enrichment bundle (navi_geo.geo_route).
|
||||
|
||||
Ported from recon's reverse_bundle_test.py (9 tests). Photon/DEM/timezone are
|
||||
mocked the same way; the in-process landclass mock becomes a mock of the
|
||||
HTTP-delegated _reverse_landclass (Phase A §5). Two added tests exercise the
|
||||
real navi-landclass HTTP client mapping (summary -> bundle['landclass']) and the
|
||||
landclass-HTTP-failure -> null path. One timezone test exercises the real
|
||||
SpatiaLite DB when present.
|
||||
"""
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
import services.navi_geo.geo_route as geo_route
|
||||
import services.navi_geo.landclass_client as landclass_client
|
||||
from services.navi_geo.app import create_app
|
||||
|
||||
EXPECTED_KEYS = set(geo_route._BUNDLE_KEYS)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client():
|
||||
return create_app().test_client()
|
||||
|
||||
|
||||
def _patch_all(monkeypatch, *, photon, timezone, landclass, elevation):
|
||||
monkeypatch.setattr(geo_route, '_reverse_photon', photon)
|
||||
monkeypatch.setattr(geo_route, '_reverse_timezone', timezone)
|
||||
monkeypatch.setattr(geo_route, '_reverse_landclass', landclass)
|
||||
monkeypatch.setattr(geo_route, '_reverse_elevation', elevation)
|
||||
|
||||
|
||||
def test_happy_path(client, monkeypatch):
|
||||
_patch_all(
|
||||
monkeypatch,
|
||||
photon=lambda lat, lon: {
|
||||
'name': 'Where you are', 'city': 'Boise', 'county': 'Ada',
|
||||
'state': 'Idaho', 'country': 'United States', 'postal_code': '83701'},
|
||||
timezone=lambda lat, lon: 'America/Boise',
|
||||
landclass=lambda lat, lon: 'Boise National Forest',
|
||||
elevation=lambda lat, lon: 824,
|
||||
)
|
||||
resp = client.get('/api/reverse/43.6150/-116.2023')
|
||||
assert resp.status_code == 200
|
||||
data = resp.get_json()
|
||||
assert set(data.keys()) == EXPECTED_KEYS
|
||||
assert data['city'] == 'Boise' and data['timezone'] == 'America/Boise'
|
||||
assert data['landclass'] == 'Boise National Forest' and data['elevation_m'] == 824
|
||||
|
||||
|
||||
def test_negative_and_integer_coords_parse(client, monkeypatch):
|
||||
# Regression: Flask's <float:> converter would 404 these; manual parse must not.
|
||||
_patch_all(monkeypatch, photon=lambda lat, lon: {}, timezone=lambda lat, lon: None,
|
||||
landclass=lambda lat, lon: None, elevation=lambda lat, lon: None)
|
||||
for path in ('/api/reverse/43.6/-116.2', '/api/reverse/43/-116'):
|
||||
resp = client.get(path)
|
||||
assert resp.status_code == 200, f"{path} -> {resp.status_code}"
|
||||
assert set(resp.get_json().keys()) == EXPECTED_KEYS
|
||||
|
||||
|
||||
def test_partial_failure_returns_200_with_nulls(client, monkeypatch):
|
||||
def boom(lat, lon):
|
||||
raise RuntimeError('down')
|
||||
_patch_all(monkeypatch, photon=boom, timezone=lambda lat, lon: 'America/Boise',
|
||||
landclass=boom, elevation=lambda lat, lon: 824)
|
||||
resp = client.get('/api/reverse/43.6150/-116.2023')
|
||||
assert resp.status_code == 200
|
||||
data = resp.get_json()
|
||||
assert set(data.keys()) == EXPECTED_KEYS
|
||||
assert data['name'] is None and data['city'] is None # photon failed -> nulls
|
||||
assert data['landclass'] is None # landclass failed -> null
|
||||
assert data['timezone'] == 'America/Boise' and data['elevation_m'] == 824
|
||||
|
||||
|
||||
def test_ocean_point_mostly_null(client, monkeypatch):
|
||||
_patch_all(monkeypatch, photon=lambda lat, lon: {}, timezone=lambda lat, lon: 'Etc/GMT+2',
|
||||
landclass=lambda lat, lon: None, elevation=lambda lat, lon: 0)
|
||||
resp = client.get('/api/reverse/0.0/-30.0')
|
||||
assert resp.status_code == 200
|
||||
data = resp.get_json()
|
||||
assert set(data.keys()) == EXPECTED_KEYS
|
||||
assert data['city'] is None and data['country'] is None and data['landclass'] is None
|
||||
|
||||
|
||||
def test_invalid_input_400(client):
|
||||
for path in ('/api/reverse/9999/0', '/api/reverse/0/9999', '/api/reverse/abc/0'):
|
||||
resp = client.get(path)
|
||||
assert resp.status_code == 400, f"{path} -> {resp.status_code}"
|
||||
|
||||
|
||||
def test_cache_hit_serves_without_recompute(client, monkeypatch):
|
||||
calls = {'n': 0}
|
||||
|
||||
def counting_photon(lat, lon):
|
||||
calls['n'] += 1
|
||||
return {'name': 'X'}
|
||||
_patch_all(monkeypatch, photon=counting_photon, timezone=lambda lat, lon: None,
|
||||
landclass=lambda lat, lon: None, elevation=lambda lat, lon: None)
|
||||
client.get('/api/reverse/12.3456/-65.4321')
|
||||
client.get('/api/reverse/12.3456/-65.4321') # same key (rounded) -> cached
|
||||
assert calls['n'] == 1, f"expected 1 compute, got {calls['n']}"
|
||||
|
||||
|
||||
def test_real_timezone_db(monkeypatch):
|
||||
path = geo_route.tz_db_path()
|
||||
if not os.path.exists(path):
|
||||
pytest.skip("real timezone test (timezones.sqlite not present)")
|
||||
assert geo_route._reverse_timezone(43.6150, -116.2023) == 'America/Boise'
|
||||
assert geo_route._reverse_timezone(40.7128, -74.0060) == 'America/New_York'
|
||||
|
||||
|
||||
def test_elevation_from_dem_reader_mock(client, monkeypatch):
|
||||
# elevation_m comes from DEMReader.sample_point; other components stubbed null.
|
||||
class FakeDEM:
|
||||
def __init__(self):
|
||||
self.called = 0
|
||||
|
||||
def sample_point(self, lat, lon):
|
||||
self.called += 1
|
||||
return 824
|
||||
fake = FakeDEM()
|
||||
monkeypatch.setattr(geo_route, '_DEM', fake)
|
||||
monkeypatch.setattr(geo_route, '_reverse_photon', lambda lat, lon: {})
|
||||
monkeypatch.setattr(geo_route, '_reverse_timezone', lambda lat, lon: None)
|
||||
monkeypatch.setattr(geo_route, '_reverse_landclass', lambda lat, lon: None)
|
||||
resp = client.get('/api/reverse/43.6150/-116.2023')
|
||||
assert resp.status_code == 200
|
||||
data = resp.get_json()
|
||||
assert set(data.keys()) == EXPECTED_KEYS
|
||||
assert data['elevation_m'] == 824
|
||||
assert fake.called == 1
|
||||
|
||||
|
||||
def test_elevation_dem_unavailable(client, monkeypatch):
|
||||
# DEMReader failed to init at startup (_DEM is None) -> elevation_m null, 200.
|
||||
monkeypatch.setattr(geo_route, '_DEM', None)
|
||||
monkeypatch.setattr(geo_route, '_reverse_photon', lambda lat, lon: {})
|
||||
monkeypatch.setattr(geo_route, '_reverse_timezone', lambda lat, lon: None)
|
||||
monkeypatch.setattr(geo_route, '_reverse_landclass', lambda lat, lon: None)
|
||||
resp = client.get('/api/reverse/43.6150/-116.2023')
|
||||
assert resp.status_code == 200
|
||||
assert resp.get_json()['elevation_m'] is None
|
||||
|
||||
|
||||
# ── Added: the navi-landclass HTTP coupling (Phase A §5 / Phase B locked) ──
|
||||
|
||||
class _FakeResp:
|
||||
def __init__(self, status_code=200, json_data=None):
|
||||
self.status_code = status_code
|
||||
self._json = json_data or {}
|
||||
|
||||
def raise_for_status(self):
|
||||
if self.status_code >= 400:
|
||||
raise RuntimeError(f'HTTP {self.status_code}')
|
||||
|
||||
def json(self):
|
||||
return self._json
|
||||
|
||||
|
||||
def test_landclass_http_summary_maps_into_bundle(client, monkeypatch):
|
||||
# navi-landclass returns the full dict; navi-geo reads .summary into landclass.
|
||||
captured = {}
|
||||
|
||||
def fake_get(url, params=None, timeout=None):
|
||||
captured['url'] = url
|
||||
captured['params'] = params
|
||||
return _FakeResp(200, {
|
||||
'lat': params['lat'], 'lon': params['lon'],
|
||||
'classifications': [{'unit_name': 'Boise National Forest'}],
|
||||
'count': 1, 'is_public': True, 'is_private': False,
|
||||
'summary': 'Boise National Forest',
|
||||
})
|
||||
monkeypatch.setattr(landclass_client.requests, 'get', fake_get)
|
||||
monkeypatch.setattr(geo_route, '_reverse_photon', lambda lat, lon: {})
|
||||
monkeypatch.setattr(geo_route, '_reverse_timezone', lambda lat, lon: None)
|
||||
monkeypatch.setattr(geo_route, '_reverse_elevation', lambda lat, lon: None)
|
||||
resp = client.get('/api/reverse/43.6150/-116.2023')
|
||||
data = resp.get_json()
|
||||
assert data['landclass'] == 'Boise National Forest' # the summary string only
|
||||
assert captured['url'].endswith('/api/landclass')
|
||||
assert captured['params'] == {'lat': 43.615, 'lon': -116.2023}
|
||||
|
||||
|
||||
def test_landclass_http_failure_yields_null(client, monkeypatch):
|
||||
def boom_get(url, params=None, timeout=None):
|
||||
raise RuntimeError('connection refused')
|
||||
monkeypatch.setattr(landclass_client.requests, 'get', boom_get)
|
||||
monkeypatch.setattr(geo_route, '_reverse_photon', lambda lat, lon: {})
|
||||
monkeypatch.setattr(geo_route, '_reverse_timezone', lambda lat, lon: None)
|
||||
monkeypatch.setattr(geo_route, '_reverse_elevation', lambda lat, lon: None)
|
||||
resp = client.get('/api/reverse/43.6150/-116.2023')
|
||||
assert resp.status_code == 200 # never 5xx
|
||||
assert resp.get_json()['landclass'] is None
|
||||
Loading…
Add table
Add a link
Reference in a new issue