auto: docs sync 2026-06-19T00:00:09+00:00
Files changed: engine/lint-report.md vault/.obsidian/graph.json vault/.obsidian/workspace.json vault/docs/hardware/environment.md vault/docs/services/services.md vault/docs/software/central.md vault/docs/software/navi.md vault/docs/software/recon.md
This commit is contained in:
echo6-autocommit
parent
eb7eade7fa
commit
75df23b89e
8 changed files with 280 additions and 38 deletions
100
vault/docs/software/central.md
Normal file
100
vault/docs/software/central.md
Normal file
|
|
@ -0,0 +1,100 @@
|
|||
---
|
||||
title: central — Data-Hub Spine
|
||||
type: reference
|
||||
tags: [recon]
|
||||
related: ["[[navi]]", "[[services]]", "[[environment]]"]
|
||||
updated: 2026-06-18
|
||||
---
|
||||
# central — Data-Hub Spine
|
||||
|
||||
## Overview
|
||||
|
||||
central is a multi-domain real-time data-hub spine. Adapters normalize upstream sources, publish CloudEvents to **NATS/JetStream**, and archive to **TimescaleDB/PostGIS** for historical and geospatial query. It is the live data backbone for navi traffic tiles and related situational-awareness feeds.
|
||||
|
||||
- **URL (internal):** http://central.echo6.mesh:8000 (mesh-only, no public exposure)
|
||||
- **Host:** utility CT 104 (unprivileged Ubuntu LXC)
|
||||
- **Repo:** github.com/zvx-echo6/central (public, Python, branch )
|
||||
- **Deploy path:** (Python venv, system user)
|
||||
|
||||
## Host
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| Container | utility CT 104 |
|
||||
| Local IP | 192.168.1.104 |
|
||||
| Tailscale / mesh | 100.64.0.12 → |
|
||||
| Resources | 4 cores / 12 GB RAM / 100 GB disk |
|
||||
| OS | Ubuntu LXC (unprivileged) |
|
||||
|
||||
## Architecture
|
||||
|
||||
The data flow is: upstream APIs → adapters (central-supervisor) → NATS/JetStream :4222 → central-archive (TimescaleDB/PostGIS :5432). The central-gui (FastAPI + HTMX, :8000) exposes the API consumed by navi via .
|
||||
|
||||
## Systemd Services
|
||||
|
||||
All three units are **enabled and active**; deployment survives reboot.
|
||||
|
||||
| Unit | Role |
|
||||
|------|------|
|
||||
| | Adapter scheduler + CloudEvents publisher |
|
||||
| | JetStream → TimescaleDB consumer |
|
||||
| | FastAPI + HTMX web app + API (the :8000 listener) |
|
||||
|
||||
## Ports
|
||||
|
||||
| Port | Protocol | Purpose |
|
||||
|------|----------|---------|
|
||||
| :8000 | HTTP | GUI / API (bound 0.0.0.0) |
|
||||
| :4222 | TCP | NATS client connections |
|
||||
| :8222 | HTTP | NATS monitoring |
|
||||
| :5432 | TCP | PostgreSQL 16 + TimescaleDB/PostGIS |
|
||||
|
||||
## Adapters (~25 active)
|
||||
|
||||
| Domain | Sources |
|
||||
|--------|---------|
|
||||
| Traffic | ITD 511, WZDX, TomTom flow/incidents, 511 cameras |
|
||||
| Wildfire | WFIGS incidents/perimeters, InciWeb, FIRMS |
|
||||
| Weather / Space-weather | NWS, SWPC k-index/protons/alerts |
|
||||
| Hydro | NWIS |
|
||||
| Earthquakes | USGS |
|
||||
| Avalanche | avalanche.org |
|
||||
| Disasters | GDACS, EONET |
|
||||
| Satellite | CelesTrak TLE, sat positions/orbits, N2YO passes, satpass predict |
|
||||
|
||||
## GUI / API Surface
|
||||
|
||||
Authenticated app with login/sessions/CSRF, first-run setup wizard, operator management, adapter configuration, stream viewer, enrichment pipeline, monitoring-area management, API key management, audit log, and manual resend.
|
||||
|
||||
**Auth-exempt tile endpoints** (used by navi, verified HTTP 200):
|
||||
|
||||
| Endpoint | Format |
|
||||
|----------|--------|
|
||||
| | PNG tile (raster) |
|
||||
| | PBF tile (vector) |
|
||||
|
||||
All other endpoints are auth-gated.
|
||||
|
||||
## Consumer — navi Integration
|
||||
|
||||
navi-traffic (navi's in-VM :8421 extraction service) was **retired on 2026-05-26** and cut over to central. recon-vm's nginx () now proxies → . Tile endpoints verified returning HTTP 200.
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Component | Location |
|
||||
|-----------|----------|
|
||||
| NATS / JetStream | Local (central CT 104) |
|
||||
| PostgreSQL 16 + TimescaleDB + PostGIS | Local (central CT 104) |
|
||||
| Upstream APIs | ~20 external sources (see Adapters table) |
|
||||
|
||||
## Deploy / Drift Notes (as of 2026-06-18 audit)
|
||||
|
||||
- **Deployed HEAD:** v0.14.4
|
||||
- **Repo :** v0.14.5 — deployment is **1 release behind**
|
||||
- **Uncommitted local migration:** exists on disk in the deployment but was **never committed to the repo**
|
||||
- **Stale README:** repo README still reads *"Phase 0 — scaffold, not yet operational"* — central is fully operational
|
||||
- **Stale :** version field shows 0.3.0; real version is the git tag (v0.14.x)
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2026-06-18 — Initial documentation; central was previously undocumented. Deployment confirmed live, all three systemd units active.*
|
||||
94
vault/docs/software/navi.md
Normal file
94
vault/docs/software/navi.md
Normal file
|
|
@ -0,0 +1,94 @@
|
|||
---
|
||||
title: navi — Offline Navigation Platform
|
||||
type: reference
|
||||
tags: [recon]
|
||||
related: ["[[recon]]", "[[services]]", "[[environment]]"]
|
||||
updated: 2026-06-18
|
||||
---
|
||||
# navi — Offline Navigation Platform
|
||||
|
||||
## Overview
|
||||
|
||||
navi is an offline-capable navigation web app served from **recon-vm** (VM 1130, 192.168.1.130). It provides geocoding, routing, land classification, fleet admin, and DEM-backed elevation data — all from self-hosted geo backends. Frontend is a Vite SPA; backend is a suite of 8 Python microservices behind nginx.
|
||||
|
||||
- **URL:** https://navi.echo6.co (fronted by utility Caddy + Authentik)
|
||||
- **Host:** recon-vm (data node, VM 1130)
|
||||
- **Repos:** `github.com/zvx-echo6/navi` (canonical), Forge mirror `matt/navi`
|
||||
- **Layout:** monorepo — `backend/` (Python) + `frontend/` (Vite)
|
||||
- **Deploy path:** `/home/zvx/projects/repos/navi-mono` (branch `main`, in sync with origin)
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Internet → Caddy (utility CT 101) → Authentik → nginx :8440 (navi-mono frontend)
|
||||
├─ /api/* → navi-* backends :8421-:8428
|
||||
├─ /tiles/* → tile proxy
|
||||
└─ /dem/* → dem-origin :8441 (range-cached)
|
||||
└─ /mnt/nas/nav/planet-dem.pmtiles (658 GB)
|
||||
```
|
||||
|
||||
## nginx Vhosts
|
||||
|
||||
| Port | Vhost | Role |
|
||||
|------|-------|------|
|
||||
| :8440 | navi.echo6.co | Public front door — SPA from `/mnt/nav/frontend`, API gateway (`/api/*` → backends), tile/DEM proxies; range-caches DEM requests from :8441 |
|
||||
| :8441 | dem-origin (localhost only) | Serves 658 GB `planet-dem.pmtiles` from `/mnt/nas/nav/`; never exposed directly |
|
||||
|
||||
## Backend Services
|
||||
|
||||
All 8 are gunicorn processes, bound to `127.0.0.1`, working directory `navi-mono/backend`, env files in `/etc/navi-backend/*.env`.
|
||||
|
||||
| # | Service | Port | Status | Purpose |
|
||||
|---|---------|------|--------|---------|
|
||||
| 1 | navi-traffic | :8421 | **DISABLED** | Traffic — now proxied externally to `central.echo6.mesh:8000` |
|
||||
| 2 | navi-config | :8422 | Active | Deployment profile API |
|
||||
| 3 | navi-contacts | :8423 | Active | Contacts + address book |
|
||||
| 4 | navi-landclass | :8424 | Active | PAD-US land classification (Postgres `padus` DB) |
|
||||
| 5 | navi-places | :8425 | Active | OSM place detail/enrichment (Postgres `overture` DB) |
|
||||
| 6 | navi-geo | :8426 | Active | Geocode + reverse geocode (Photon + DEM + timezone) |
|
||||
| 7 | navi-admin | :8427 | Active | Fleet admin-info aggregator (auth-gated) |
|
||||
| 8 | navi-offroute | :8428 | Active | Off-network router + MVUM (Valhalla + PostGIS) |
|
||||
|
||||
Shared backend modules: `shared/auth.py` (Authentik header validation), `shared/admin_info.py`.
|
||||
|
||||
## Geo Backends (co-resident on recon-vm)
|
||||
|
||||
| Service | Port | Runtime | Purpose |
|
||||
|---------|------|---------|---------|
|
||||
| Valhalla | :8002 | Docker | Routing engine |
|
||||
| Nominatim | :8010 | Docker | Geocoder |
|
||||
| Photon | :2322 | Direct | Geocoder (embedded Elasticsearch :9201) |
|
||||
| PostgreSQL/PostGIS | :5432 | System | DBs: `padus` (land classification), `overture` (OSM enrichment) |
|
||||
|
||||
## Data
|
||||
|
||||
| Path | Contents |
|
||||
|------|---------|
|
||||
| `/mnt/nav/` | Addresses, timezones, OSM PBF, worldcover friction/barriers/trails, HPA, Photon data, frontend SPA build |
|
||||
| `/mnt/nas/nav/planet-dem.pmtiles` | 658 GB global DEM tile archive |
|
||||
| `/var/lib/navi-backend/` | Writable runtime state: `contacts.db`, `place_cache.db`, `wiki_index.db` |
|
||||
|
||||
NFS: pi-nas exports `/mnt/nav` and `/mnt/nas` to recon-vm.
|
||||
|
||||
## Deployment
|
||||
|
||||
The repo ships its own deploy templates under `backend/deploy/`:
|
||||
|
||||
- `deploy/systemd/` — all 8 unit files (match live)
|
||||
- `deploy/nginx/` — nginx vhost snippets (match live)
|
||||
- `deploy/env/` — env file templates (instances in `/etc/navi-backend/`)
|
||||
- `deploy/caddy/` — Caddy upstream fragment
|
||||
|
||||
## Gotchas / Operational Notes
|
||||
|
||||
**Three checkouts exist** under `~/projects/repos/`:
|
||||
|
||||
| Checkout | Status |
|
||||
|----------|--------|
|
||||
| `navi-mono/` | **Live — this is what runs** |
|
||||
| `navi/` | Stale — do not use |
|
||||
| `navi-backend/` | Stale — **do not delete without auditing** (see below) |
|
||||
|
||||
**`navi-backend/` cannot be deleted yet:** several live env files (`navi-config`, `navi-contacts`, `navi-places`, `navi-geo`, `navi-offroute`) reference config, profile, and address-book files under `/home/zvx/projects/repos/navi-backend/config/...`. Document those paths before any cleanup.
|
||||
|
||||
**Stale README:** `backend/README.md` in the monorepo still refers to "extraction #1: navi-traffic" and predates the full monorepo consolidation. Treat `backend/deploy/` as the authoritative source.
|
||||
|
|
@ -22,9 +22,11 @@ RECON extracts knowledge from PDFs and web content into a searchable vector data
|
|||
|
||||
- **Host:** recon-vm (VM 1130 on data node, 192.168.1.240) — migrated from CT 130 on 2026-04-19
|
||||
- **IP:** 192.168.1.130 / 100.64.0.24 (Tailscale)
|
||||
- **Install:** `/opt/recon/`
|
||||
- **Repo:** github.com/zvx-echo6/recon (branch `master`; Forge mirror `matt/recon`)
|
||||
- **Install:** `/opt/recon/` (Python venv, in sync with origin)
|
||||
- **User:** zvx
|
||||
- **Service:** `recon.service`, `recon-watchdog.service`, `kiwix.service` (systemd)
|
||||
- **Entrypoints:** `recon.py service` (dashboard/API :8420) + `recon.py pipeline watch` (watchdog, auto-ingests new PDFs in `/mnt/library`)
|
||||
- **Dashboard:** https://recon.echo6.co (internal: http://100.64.0.24:8420)
|
||||
- **Health:** https://recon.echo6.co/api/health
|
||||
|
||||
|
|
@ -37,7 +39,8 @@ RECON extracts knowledge from PDFs and web content into a searchable vector data
|
|||
| Status DB | SQLite (WAL mode) | /opt/recon/data/recon.db |
|
||||
| Vector DB | Qdrant | cortex:6333 (Docker) |
|
||||
| Embeddings | TEI (bge-m3, 1024-dim) | cortex:8090 (Docker) |
|
||||
| Enrichment | Gemini 2.5 Flash Lite | Google API (4 keys) |
|
||||
| Sparse embeddings | recon-sparse (bge-m3 SPLADE) | cortex:8091 (systemd) |
|
||||
| Enrichment | Gemini `gemini-2.5-flash-lite` (enforced) | Google API (4 keys) |
|
||||
| Vision Ocr | Gemini 2.5 Flash Lite | Google API (shared keys) |
|
||||
| Text extraction | PyPDF2, poppler-utils, Tesseract | Local |
|
||||
| PDF source | NFS | pi-nas:/export/library → /mnt/library |
|
||||
|
|
@ -67,7 +70,7 @@ Method tracking saved in `data/text/{hash}/meta.json` as `ocr_methods` dict.
|
|||
|
||||
- ~10,162 documents in pipeline
|
||||
- ~95,000+ vectors in Qdrant (HNSW index, <10ms search latency)
|
||||
- Collection: `recon_knowledge`
|
||||
- Collection: `recon_knowledge_hybrid`
|
||||
- ~13,239 PDFs catalogued from NFS library
|
||||
|
||||
## Resilience
|
||||
|
|
@ -142,4 +145,4 @@ Key sections:
|
|||
|
||||
---
|
||||
|
||||
*Last updated: 2026-02-16 — Initial creation*
|
||||
*Last updated: 2026-06-18 — Updated: repo/branch, recon-sparse :8091, recon_knowledge_hybrid collection, Entrypoints; PROJECT-BIBLE.md dated 2026-02-16 (predates current deployment) — verified against live 2026-06-18*
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue