MVUM Data Import:
- Downloaded USFS MVUM Roads (150,636 features) and Trails (28,741 features)
- Imported to navi.db as mvum_roads and mvum_trails tables
- Idaho coverage: ~8,994 roads and ~4,504 trails across 7 national forests
- Preserved all vehicle-class fields (ATV, MOTORCYCLE, HIGHCLEARANCEVEHICLE, etc.)
- Preserved seasonal date ranges (*_DATESOPEN fields)
New mvum.py module:
- MVUMReader class for querying MVUM data by bbox and nearest point
- parse_date_range() for seasonal date string parsing (MM/DD-MM/DD format)
- check_access() for determining open/closed status with date checking
- symbol_to_access() fallback when per-vehicle fields are null
- get_mvum_access_grid() for rasterizing MVUM to pathfinder grid
Cost function integration:
- Added mvum parameter to compute_cost_grid()
- MVUM closures respond to boundary_mode:
* strict = impassable (np.inf)
* pragmatic = 5x friction penalty
* emergency = ignored entirely
- Foot mode skips MVUM (motor-vehicle specific)
Router integration:
- Loads MVUM access grid for motorized modes (mtb, atv, vehicle)
- Tracks mvum_closed_crossings in path summary
Places Panel API:
- GET /api/mvum?lat=XX&lon=XX&radius=50
- Returns MVUM feature with access status for all vehicle classes
- Includes seasonal date ranges, maintenance level, forest/district info
- GeoJSON geometry for map display
Validation:
- MVUM places endpoint tested with Sawtooth NF road
- All four modes validated with strict/pragmatic/emergency boundary modes
- Foot mode correctly ignores MVUM restrictions
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- New friction.py: reads WorldCover friction VRT, resamples to match
elevation grid, provides point sampling for validation
- Modified cost.py: accepts optional friction array, multiplies Tobler
time cost by friction multiplier, inf for water/nodata (255/0)
- Modified prototype.py: loads friction layer, passes to cost function,
validates path avoids water cells (friction=255)
Validated on Idaho test bbox:
- Path avoids Murtaugh Lake (no water cells on path)
- Friction along path: min=10, max=20, mean=10.2
- Effort increased 3.4% vs Phase O1 due to friction multipliers
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds _text_quality_ok() gate that replaces the bare 50-char length
check at each stage of the extraction fallback chain. Checks:
- Word-boundary ratio (≥60% of tokens must be real words)
- Concatenation ratio (lc→UC transitions must be <10% of word count)
When PyPDF2 default extraction fails quality check, retries with
space_width=100 for tighter word-boundary detection. This fixes
Haynes/workshop manuals where tight kerning produces concatenated
words like 'byMike' and 'oftheGuild'.
Also adds -layout flag to pdftotext subprocess calls for better
spatial awareness in the poppler fallback stage.
Note: PyPDF2 3.0.1 does not support layout=True parameter.
The space_width parameter serves the same purpose.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pre-processes HTML tree before lxml .text_content() to prevent
element concatenation:
- <table> cells joined with ' | ' delimiter, rows with newlines
- <br> tags produce newlines
- <li> items get '- ' prefix and newline separation
- <dt>/<dd> definition list items get newline separation
Fixes ~868 mangled Qdrant points where table content was jammed
together (e.g. 'Freq51Primary1A==' instead of 'Freq51 | Primary | 1A==').
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Returns {authenticated: bool, username: string|null} based on
X-Authentik-Username header presence. Used by Navi frontend to
detect auth state without triggering SSO redirect.
The /api/geocode endpoint blended Photon and Netsyms results, but only
Photon respected viewport bias from prior work. Address queries to
Netsyms/AddressDB returned globally-sorted matches regardless of where
the user was looking — searching '214 North St' from Idaho returned
Illinois results.
Now fetches up to 200 Netsyms results when viewport lat/lon provided,
sorts by squared distance from viewport center, then returns top N.
Falls back to default ordering when viewport absent. Photon path
unchanged.
Request polygon_geojson=1 from Nominatim to include admin boundary
polygons in place detail responses. Also fetch boundary via OSM
relation ID for wikidata lookups.
- Add get_place_by_wikidata() to place_detail.py
- Queries Wikidata API for entity details (name, description, coords)
- Extracts population, instance_of, OSM relation ID, Wikipedia link
- Add /api/place/wikidata/<id> route to api.py
Supports Navi basemap label enrichment when OSM details unavailable.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
NAVI-DIRECTIONS-REDESIGN.md belongs in the design-docs repo
(matt/refactored-recon) alongside PROJECT-BIBLE.md, AUTH-PUBLIC-FRONTEND.md,
and other design artifacts. Code repo holds code only.
Design document covering:
- Current state analysis and failure modes
- New DirectionsPanel with visible From/To inputs
- RadialMenu component for map right-click/long-press
- Interaction flows for all directions scenarios
- Mobile considerations (bottom sheet, long-press timing)
- Implementation sequence (10 phases)
- Open questions for Matt
Implementation deferred to dedicated session.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add lat/lon/zoom params to geocode() and _retrieve_photon_freetext()
- Update nav_tools.py wrapper to pass through viewport params
- Add /api/geocode handler support for lat/lon/zoom query params
- Add _safe_float() helper for param validation
- Cast zoom to int for Photon compatibility
Allows the frontend to pass current map center/zoom to bias
search results toward the visible area.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Guest users receive local and cached data only. New Google Places API
calls are only triggered for authenticated users, protecting against
cost exploitation on the public navi.echo6.co frontend.
The pattern: cached Google data flows freely (already paid for by an
authed lookup). New API calls require X-Authentik-Username via
get_user_id() check.
Adds has_contours, has_contours_test, and has_contours_test_10ft flags
to support contour layer toggle in Navi frontend. minimal_pi profile
intentionally excluded (no tile overlays in stripped-down deployment).
Replace /nav-i/api-keys stub with functional admin page for managing
third-party API keys (Gemini, TomTom, Google Places).
- New lib/api_keys_admin.py: list/update/test operations with masked
display, atomic .env writes (.env.bak backup), provider-specific
test calls (Gemini models.list, TomTom geocode, Google Places
searchText)
- 4 new endpoints: GET /api/nav-i/api-keys/list, POST .../update,
POST .../test, POST .../restart-recon
- Full UI: key table with masked values, per-key update modal with
show/hide toggle, inline test results with latency, Gemini detail
sub-table with per-key stats, RECON restart with confirmation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rewrites OSM wikipedia/wikidata/wikivoyage/appropedia extratag values
to local Kiwix URLs (wiki.echo6.co) when the article exists in a loaded
ZIM, falling back silently to public URLs otherwise.
- New lib/wiki_rewrite.py: URL classification, Kiwix OPDS catalog
discovery (xml.etree.ElementTree), HEAD-based availability check,
positive-only SQLite cache, disabled discovery stubs
- place_detail.py: _enrich_wiki_links() at both Nominatim and Overpass
enrichment sites, before cache_put
- Profile flags: has_wiki_rewriting (home/regional: true, minimal: false),
has_wiki_discovery (all: false, stubs for future activation)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Integrates USGS PAD-US 4.0 (651k features) into a local PostGIS database
for point-in-polygon land ownership queries. Adds /api/landclass endpoint
returning classifications, public/private status, and management hierarchy.
- lib/landclass.py: connection pool, lookup_landclass(), domain label maps
- lib/api.py: GET /api/landclass?lat=&lon= (feature-flag gated)
- home.yaml: enable has_landclass flag
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fills opening_hours, phone, and website gaps when OSM + Overture data
is incomplete. Only fires for business-class POIs (amenity, shop, tourism,
leisure, office, craft). Daily API call cap with SQLite tracking.
cache_put now preserves google columns across cache refreshes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a query contains no road-type keywords (st, blvd, ave, etc.),
boost amenity/shop/tourism/leisure/office/craft results (+3.0) and
penalize highway/route results (-4.0). This fixes searches like
"starbucks twin falls" where a named service road outranked the
actual business POI due to Photon position tiebreaking.
Also fixes:
- Intent classifier now recognizes full state names ("idaho" not
just "ID") for LOCALITY classification
- Locality-type Photon results now populate _city from name field
so they participate in locality_fuzz scoring
- Trace logging expanded to all candidates with osm_key/value
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Businesses with housenumbers (e.g. M&W Markets at 130 US-30) were
classified as street_address because the housenumber check fired before
the osm_key check. Reorder so osm_key in amenity/shop/tourism/leisure/office
is evaluated first, ensuring businesses get type=poi regardless of
whether they have a street address. Also adds office to the POI key set.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ingests 20.9M North America places from Overture Maps Foundation
(release 2026-04-15.0) into PostgreSQL. Enriches /api/place responses
with phone, website, and brand data via spatial + fuzzy name matching
when OSM extratags are sparse.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New /api/place/<osm_type>/<osm_id> endpoint returns cleaned OSM tag data
for PlaceDetail panel enrichment. Routes to local Nominatim (Idaho coverage)
first, falls back to Overpass public API for out-of-region queries. Responses
cached in SQLite (data/place_cache.db) with no expiry.
New modules: lib/place_detail.py (proxy + cache), lib/osm_categories.py
(~50 category humanization mappings). Profile YAMLs updated with
place_details config block and has_nominatim_details flag.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add /api/traffic/flow proxy route to hide TomTom API key from frontend
- Add tileset_hillshade and traffic config blocks to all three profiles
- Flip has_hillshade and has_traffic_overlay flags in home and regional profiles
- Minimal profile has config blocks but flags remain false (dormant)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add profile-driven config infrastructure:
- config/profiles/{home,regional_pi,minimal_pi}.yaml templates
- lib/deployment_config.py loader (reads RECON_PROFILE env var)
- GET /api/config returns active profile as JSON (5min cache)
Frontend reads this on startup to determine tile source, defaults,
and feature flags. No existing behavior changed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Accepts lat/lon query params, calls Photon /reverse, returns same
response shape as /api/geocode. Returns 200 with empty results on
no match (graceful degradation for ocean/unmapped areas).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Inverts the /api/geocode chain. Photon is now the primary search
engine; the hand-rolled Netsyms free-text parser is removed.
Address book short-circuits nicknames only ("home", "work") —
full-address queries flow through Photon and address book
entries within 75m annotate matching results with labeled_as.
Coordinate strings detected before search.
Response shape: /api/geocode now returns a ranked candidates
list (always 200 OK, empty list if no match). No more 404 for
unmatched queries. Users can type messy input — wrong case,
missing punctuation, abbreviations, typos — and get results
or close matches.
Netsyms preserved at /api/netsyms/lookup for direct access.
USPS plus4 enrichment of Photon street-address hits is a
planned follow-up.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
lookup() previously did exact-alias-only matching, so "214 north st
filer" missed the home entry with alias "214 north st". Extend to
match when the query begins with an alias followed by a word
boundary, and when an alias appears as a contiguous token sequence
inside the query. Short aliases ("home") keep matching exactly and
also match with trailing text.
Fixes the UX case where typing a known full address falls through
to Netsyms instead of short-circuiting to address_book.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- YAML-backed saved locations (config/address_book.yaml)
- Exact/partial alias matching with case-insensitive lookup
- Flask blueprint: /api/address_book/lookup, /api/address_book/list
- Geocoder short-circuits Photon when address book has exact match
- Test suite for lookup behavior
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add centroid-based query classifier that routes Aurora queries to the
appropriate handler (nav_route, nav_reverse_geocode, direct_answer,
rag_search) before the RAG pipeline runs. Uses TEI embeddings against
pre-computed route centroids from 38 example queries.
- query_router.py: standalone module with lazy centroid init
- query_router_test.py: 7-query test suite (all passing)
- Corresponding recon_rag_tool.py v4.2.0 deployed to Open WebUI DB
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- nav_tools.py: route() geocodes via Photon, routes via Valhalla, returns
summary/maneuvers/polyline. reverse_geocode() for coordinate lookups.
Supports auto/pedestrian/bicycle/truck modes.
- nav_tools_test.py: 5 live tests against local Photon (2322) and Valhalla (8002)
- aurora_nav_tool.py: Open WebUI Tool exposing get_directions to Aurora LLM
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Parse Browsertrix "crawled":N JSON format instead of "N pages"
- Add 3s delay between SIGHUP to kiwix-serve and scan_zims() call
so the OPDS catalog is reloaded before we query it for linking
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Parse "crawled":N from Browsertrix crawlStatus JSON logs instead of
looking for "N pages" pattern. Also check stdout (not just stderr).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
warc2zim (called internally by zimit) requires --name for ZIM metadata.
Without it, argument validation fails with exit code 2.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Must pass `zimit` as command after image name (entrypoint execs args)
- --url → --seeds, --name removed, --lang → --zim-lang, --workers → -w
- Remove --rm so docker logs work after exit, manually rm container
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Extract shared _full_zim_cleanup(source_id) from api_kiwix_remove
- Add SIGHUP to kiwix-serve after kiwix-manage remove
- Delete linked scrape_jobs rows during ZIM removal
- Update api_scraper_delete to do full ZIM cleanup when applicable
- Set chromium_path for single-file browser crawl support
- Add status.db to .gitignore
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
