matt/meshai
1
0
Fork 0
mirror of https://github.com/zvx-echo6/meshai.git synced 2026-06-11 01:14:45 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat(v0.6-4): gauge_sites + town_anchors curation tables + GUI CRUD

Browse source 
Closes Section A.5 (gauge_sites) and A.12 (town_anchors) of the audit
doc by lifting both Python-dict curation lists into editable SQLite
tables. Operators can add/edit/disable rows from the dashboard without
a deploy; runtime reads go through cached accessors that invalidate
when the REST API mutates state.

Schema:
  v8.sql adds gauge_sites(site_id PK, gauge_name, lat, lon, action_ft,
    flood_minor_ft, flood_moderate_ft, flood_major_ft, enabled, updated_at).
  v9.sql adds town_anchors(anchor_id AUTOINC PK, name UNIQUE, lat, lon,
    state, enabled, updated_at).
  SCHEMA_VERSION 7 -> 9.

Seed (meshai/persistence/curation.py):
  _GAUGE_SITES_SEED carries the original 9 Idaho rows from
  IDAHO_CURATED_SITES verbatim.
  _TOWN_ANCHORS_SEED carries the 29 Idaho-and-neighbor towns from
  _TOWN_COORDS verbatim.
  seed_gauge_sites() / seed_town_anchors() INSERT OR IGNORE -- safe to
  re-run; never overwrites user edits.

Handler integration:
  - meshai/central/idaho_gauge_sites.py: IDAHO_CURATED_SITES dict deleted.
    lookup_site() now calls meshai.persistence.curation.lookup_gauge_site()
    which reads the table. THRESHOLD_RANK, normalize_site_id, and
    compute_threshold_state remain in this module (CODE per Matt s rule).
  - meshai/central/nwis_handler.py drops IDAHO_CURATED_SITES from its
    import list; the table-backed lookup_site() is API-compatible.
  - meshai/central_normalizer.py: _TOWN_COORDS dict deleted.
    _compute_distance_bearing() now calls
    meshai.persistence.curation.lookup_town_anchor() with the same
    lowercased-name semantics it always used.

REST API (meshai/dashboard/api/curation_routes.py):
  /api/gauge-sites  GET list, GET one, POST add, PUT update, DELETE
  /api/town-anchors GET list, GET one, POST add, PUT update, DELETE
  Every mutation calls invalidate_curation_cache() so handler reads see
  the new state on the next call -- no container restart.

Dashboard (dashboard-frontend/src/pages/):
  - GaugeSites.tsx: table view with Add row / Edit row inline / Delete
    confirm + per-row enabled toggle. 8 columns mirror the schema.
  - TownAnchors.tsx: same pattern, 5 columns. Name is lowercased on
    save to match the lookup key.
  - Left-nav entries "Gauge Sites" (Droplets icon) and "Town Anchors"
    (MapPin icon) added to Layout.tsx; routes added to App.tsx.

Tests (tests/test_curation.py, 18 cases):
  - v8/v9 tables exist
  - Seed lands every row from both dicts
  - Seed idempotent; never overwrites user edits
  - lookup_gauge_site hits/miss, disabled rows are invisible
  - lookup_town_anchor case-insensitive
  - REST API: GET list, GET one, GET 404, POST add, PUT update, DELETE,
    POST missing-field 400; both gauge_sites + town_anchors
  - Accessor reflects API mutations after invalidate_curation_cache()

tests/test_nwis_handler.py back-compat: IDAHO_CURATED_SITES dict alias
points at _GAUGE_SITES_SEED so the existing assertion suite still passes.
tests/test_adapter_config_foundation.py schema_meta v7 -> v9 bump.

Test count: 797 -> 819 (+18 curation cases + 4 maintenance updates).
This commit is contained in:
Matt Johnson (via Claude) 2026-06-05 20:19:13 +00:00
commit e3bf53ade4
20 changed files with 1322 additions and 272 deletions
Download patch file Download diff file Expand all files Collapse all files

4
dashboard-frontend/src/App.tsx
View file

@ -8,6 +8,8 @@ import Alerts from './pages/Alerts'
import Notifications from './pages/Notifications'
import Reference from './pages/Reference'
import AdapterConfig from './pages/AdapterConfig'
import GaugeSites from './pages/GaugeSites'
import TownAnchors from './pages/TownAnchors'
import { ToastProvider } from './components/ToastProvider'
function App() {
@ -23,6 +25,8 @@ function App() {
<Route path="/notifications" element={<Notifications />} />
<Route path="/reference" element={<Reference />} />
<Route path="/adapter-config" element={<AdapterConfig />} />
<Route path="/gauge-sites" element={<GaugeSites />} />
<Route path="/town-anchors" element={<TownAnchors />} />
</Routes>
</Layout>
</ToastProvider>

4
dashboard-frontend/src/components/Layout.tsx
View file

@ -9,6 +9,8 @@ import {
BellRing,
BookOpen,
Sliders,
Droplets,
MapPin,
} from 'lucide-react'
import { fetchStatus, type SystemStatus } from '@/lib/api'
import { useWebSocket } from '@/hooks/useWebSocket'
@ -27,6 +29,8 @@ const navItems = [
{ path: '/notifications', label: 'Notifications', icon: BellRing },
{ path: '/reference', label: 'Reference', icon: BookOpen },
{ path: '/adapter-config', label: 'Adapter Config', icon: Sliders },
{ path: '/gauge-sites', label: 'Gauge Sites', icon: Droplets },
{ path: '/town-anchors', label: 'Town Anchors', icon: MapPin },
]
function formatUptime(seconds: number): string {

198
dashboard-frontend/src/pages/GaugeSites.tsx Normal file
View file

@ -0,0 +1,198 @@
// v0.6-4 GaugeSites table editor.
import { useEffect, useState, useCallback } from 'react'
import { Loader2, Plus, Trash2, Check, X, Droplets } from 'lucide-react'
interface GaugeSite {
site_id: string
gauge_name: string
lat: number
lon: number
action_ft: number | null
flood_minor_ft: number | null
flood_moderate_ft: number | null
flood_major_ft: number | null
enabled: boolean
updated_at: number
}
const EMPTY_DRAFT: GaugeSite = {
site_id: '', gauge_name: '', lat: 0, lon: 0,
action_ft: null, flood_minor_ft: null, flood_moderate_ft: null, flood_major_ft: null,
enabled: true, updated_at: 0,
}
export default function GaugeSites() {
const [rows, setRows] = useState<GaugeSite[]>([])
const [loading, setLoading] = useState(true)
const [error, setError] = useState<string | null>(null)
const [editing, setEditing] = useState<string | null>(null)
const [draft, setDraft] = useState<GaugeSite>(EMPTY_DRAFT)
const [adding, setAdding] = useState(false)
const refresh = useCallback(async () => {
setLoading(true)
setError(null)
try {
const res = await fetch('/api/gauge-sites')
if (!res.ok) throw new Error(`GET: ${res.status}`)
setRows(await res.json())
} catch (e) {
setError(String(e))
} finally {
setLoading(false)
}
}, [])
useEffect(() => { refresh() }, [refresh])
const beginEdit = (r: GaugeSite) => { setEditing(r.site_id); setDraft({ ...r }); setAdding(false) }
const beginAdd = () => { setAdding(true); setEditing(null); setDraft({ ...EMPTY_DRAFT }) }
const cancel = () => { setEditing(null); setAdding(false); setDraft(EMPTY_DRAFT) }
const save = async () => {
try {
const url = adding ? '/api/gauge-sites' : `/api/gauge-sites/${editing}`
const method = adding ? 'POST' : 'PUT'
const res = await fetch(url, {
method,
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(draft),
})
if (!res.ok) {
const b = await res.json().catch(() => ({}))
alert(`save failed: ${b.detail || res.statusText}`)
return
}
cancel()
refresh()
} catch (e) {
alert(String(e))
}
}
const remove = async (siteId: string) => {
if (!confirm(`Delete ${siteId}?`)) return
const res = await fetch(`/api/gauge-sites/${siteId}`, { method: 'DELETE' })
if (!res.ok) { alert(`delete failed: ${res.status}`); return }
refresh()
}
if (loading) return <div className="p-6 text-slate-400"><Loader2 className="w-5 h-5 animate-spin inline mr-2" />Loading…</div>
if (error) return <div className="p-6 text-red-400">Load failed: {error}</div>
return (
<div className="p-6 space-y-4">
<div className="flex items-center gap-2">
<Droplets className="w-5 h-5 text-cyan-400" />
<h1 className="text-xl font-semibold text-slate-100">Gauge Sites</h1>
<span className="text-xs text-slate-500 ml-2">{rows.length} sites</span>
<button onClick={beginAdd}
className="ml-auto flex items-center gap-1 px-3 py-1 bg-cyan-700 hover:bg-cyan-600 rounded text-white text-sm">
<Plus className="w-4 h-4" /> Add site
</button>
</div>
<p className="text-xs text-slate-400 max-w-3xl">
NWS-AHPS stream gauge thresholds curated for the nwis_handler. Disabled rows are
ignored at envelope time. Changes propagate to the handler on the next event.
</p>
{adding && <RowEditor draft={draft} setDraft={setDraft} onSave={save} onCancel={cancel} adding />}
<div className="bg-slate-800/60 border border-slate-700 rounded-lg overflow-x-auto">
<table className="w-full text-sm text-slate-200">
<thead className="bg-slate-900 text-xs text-slate-400 uppercase">
<tr>
<th className="px-3 py-2 text-left">Site ID</th>
<th className="px-3 py-2 text-left">Name</th>
<th className="px-3 py-2 text-right">Lat,Lon</th>
<th className="px-3 py-2 text-right">Action</th>
<th className="px-3 py-2 text-right">Minor</th>
<th className="px-3 py-2 text-right">Moderate</th>
<th className="px-3 py-2 text-right">Major</th>
<th className="px-3 py-2 text-center">On</th>
<th className="px-3 py-2"></th>
</tr>
</thead>
<tbody className="divide-y divide-slate-700/60">
{rows.map(r => editing === r.site_id ? (
<tr key={r.site_id} className="bg-slate-900/40">
<td colSpan={9} className="px-3 py-2">
<RowEditor draft={draft} setDraft={setDraft} onSave={save} onCancel={cancel} />
</td>
</tr>
) : (
<tr key={r.site_id} className="hover:bg-slate-800/50">
<td className="px-3 py-2 font-mono text-xs">{r.site_id}</td>
<td className="px-3 py-2">{r.gauge_name}</td>
<td className="px-3 py-2 text-right text-xs">{r.lat.toFixed(3)},{r.lon.toFixed(3)}</td>
<td className="px-3 py-2 text-right">{r.action_ft ?? '-'}</td>
<td className="px-3 py-2 text-right">{r.flood_minor_ft ?? '-'}</td>
<td className="px-3 py-2 text-right">{r.flood_moderate_ft ?? '-'}</td>
<td className="px-3 py-2 text-right">{r.flood_major_ft ?? '-'}</td>
<td className="px-3 py-2 text-center">{r.enabled ? <Check className="w-4 h-4 text-emerald-400 inline" /> : <X className="w-4 h-4 text-slate-500 inline" />}</td>
<td className="px-3 py-2 text-right">
<button onClick={() => beginEdit(r)} className="text-cyan-400 hover:text-cyan-300 text-xs mr-3">Edit</button>
<button onClick={() => remove(r.site_id)} className="text-red-400 hover:text-red-300"><Trash2 className="w-4 h-4 inline" /></button>
</td>
</tr>
))}
</tbody>
</table>
</div>
</div>
)
}
function RowEditor({ draft, setDraft, onSave, onCancel, adding }: {
draft: GaugeSite, setDraft: (g: GaugeSite) => void,
onSave: () => void, onCancel: () => void, adding?: boolean,
}) {
const upd = (k: keyof GaugeSite, v: unknown) => setDraft({ ...draft, [k]: v })
return (
<div className="grid grid-cols-2 md:grid-cols-4 gap-2 p-3 bg-slate-900/50 rounded">
<label className="text-xs text-slate-400 col-span-2">
Site ID
<input className="block w-full mt-1 bg-slate-800 border border-slate-700 rounded px-2 py-1 text-slate-100 font-mono text-xs"
value={draft.site_id} onChange={e => upd('site_id', e.target.value)} disabled={!adding} />
</label>
<label className="text-xs text-slate-400 col-span-2">
Gauge name
<input className="block w-full mt-1 bg-slate-800 border border-slate-700 rounded px-2 py-1 text-slate-100"
value={draft.gauge_name} onChange={e => upd('gauge_name', e.target.value)} />
</label>
<label className="text-xs text-slate-400">Lat
<input type="number" step="any" className="block w-full mt-1 bg-slate-800 border border-slate-700 rounded px-2 py-1 text-slate-100"
value={draft.lat} onChange={e => upd('lat', parseFloat(e.target.value))} />
</label>
<label className="text-xs text-slate-400">Lon
<input type="number" step="any" className="block w-full mt-1 bg-slate-800 border border-slate-700 rounded px-2 py-1 text-slate-100"
value={draft.lon} onChange={e => upd('lon', parseFloat(e.target.value))} />
</label>
<label className="text-xs text-slate-400">Action ft
<input type="number" step="any" className="block w-full mt-1 bg-slate-800 border border-slate-700 rounded px-2 py-1 text-slate-100"
value={draft.action_ft ?? ''} onChange={e => upd('action_ft', e.target.value === '' ? null : parseFloat(e.target.value))} />
</label>
<label className="text-xs text-slate-400">Minor flood ft
<input type="number" step="any" className="block w-full mt-1 bg-slate-800 border border-slate-700 rounded px-2 py-1 text-slate-100"
value={draft.flood_minor_ft ?? ''} onChange={e => upd('flood_minor_ft', e.target.value === '' ? null : parseFloat(e.target.value))} />
</label>
<label className="text-xs text-slate-400">Moderate flood ft
<input type="number" step="any" className="block w-full mt-1 bg-slate-800 border border-slate-700 rounded px-2 py-1 text-slate-100"
value={draft.flood_moderate_ft ?? ''} onChange={e => upd('flood_moderate_ft', e.target.value === '' ? null : parseFloat(e.target.value))} />
</label>
<label className="text-xs text-slate-400">Major flood ft
<input type="number" step="any" className="block w-full mt-1 bg-slate-800 border border-slate-700 rounded px-2 py-1 text-slate-100"
value={draft.flood_major_ft ?? ''} onChange={e => upd('flood_major_ft', e.target.value === '' ? null : parseFloat(e.target.value))} />
</label>
<label className="text-xs text-slate-300 col-span-2 flex items-center gap-2 mt-2">
<input type="checkbox" checked={draft.enabled} onChange={e => upd('enabled', e.target.checked)} className="accent-cyan-500" />
Enabled
</label>
<div className="col-span-2 flex items-center justify-end gap-2 mt-2">
<button onClick={onCancel} className="px-3 py-1 text-slate-300 hover:bg-slate-700 rounded text-sm">Cancel</button>
<button onClick={onSave} className="px-3 py-1 bg-cyan-700 hover:bg-cyan-600 text-white rounded text-sm">Save</button>
</div>
</div>
)
}

153
dashboard-frontend/src/pages/TownAnchors.tsx Normal file
View file

@ -0,0 +1,153 @@
// v0.6-4 TownAnchors table editor.
import { useEffect, useState, useCallback } from 'react'
import { Loader2, Plus, Trash2, Check, X, MapPin } from 'lucide-react'
interface TownAnchor {
anchor_id: number
name: string
lat: number
lon: number
state: string | null
enabled: boolean
updated_at: number
}
const EMPTY_DRAFT: TownAnchor = {
anchor_id: 0, name: '', lat: 0, lon: 0, state: 'ID', enabled: true, updated_at: 0,
}
export default function TownAnchors() {
const [rows, setRows] = useState<TownAnchor[]>([])
const [loading, setLoading] = useState(true)
const [error, setError] = useState<string | null>(null)
const [editing, setEditing] = useState<number | null>(null)
const [adding, setAdding] = useState(false)
const [draft, setDraft] = useState<TownAnchor>(EMPTY_DRAFT)
const refresh = useCallback(async () => {
setLoading(true); setError(null)
try {
const res = await fetch('/api/town-anchors')
if (!res.ok) throw new Error(`GET: ${res.status}`)
setRows(await res.json())
} catch (e) { setError(String(e)) } finally { setLoading(false) }
}, [])
useEffect(() => { refresh() }, [refresh])
const beginEdit = (r: TownAnchor) => { setEditing(r.anchor_id); setDraft({ ...r }); setAdding(false) }
const beginAdd = () => { setAdding(true); setEditing(null); setDraft({ ...EMPTY_DRAFT }) }
const cancel = () => { setEditing(null); setAdding(false); setDraft(EMPTY_DRAFT) }
const save = async () => {
const url = adding ? '/api/town-anchors' : `/api/town-anchors/${editing}`
const method = adding ? 'POST' : 'PUT'
const res = await fetch(url, {
method, headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(draft),
})
if (!res.ok) {
const b = await res.json().catch(() => ({}))
alert(`save failed: ${b.detail || res.statusText}`); return
}
cancel(); refresh()
}
const remove = async (id: number) => {
if (!confirm(`Delete anchor ${id}?`)) return
const res = await fetch(`/api/town-anchors/${id}`, { method: 'DELETE' })
if (!res.ok) { alert(`delete failed: ${res.status}`); return }
refresh()
}
if (loading) return <div className="p-6 text-slate-400"><Loader2 className="w-5 h-5 animate-spin inline mr-2" />Loading…</div>
if (error) return <div className="p-6 text-red-400">Load failed: {error}</div>
return (
<div className="p-6 space-y-4">
<div className="flex items-center gap-2">
<MapPin className="w-5 h-5 text-cyan-400" />
<h1 className="text-xl font-semibold text-slate-100">Town Anchors</h1>
<span className="text-xs text-slate-500 ml-2">{rows.length} towns</span>
<button onClick={beginAdd}
className="ml-auto flex items-center gap-1 px-3 py-1 bg-cyan-700 hover:bg-cyan-600 rounded text-white text-sm">
<Plus className="w-4 h-4" /> Add town
</button>
</div>
<p className="text-xs text-slate-400 max-w-3xl">
Lookup table for the &quot;X mi &lt;bearing&gt; of &lt;town&gt;&quot; anchor in wire-string rendering.
Disabled rows fall through to the generic anchor chain.
</p>
{adding && <RowEditor draft={draft} setDraft={setDraft} onSave={save} onCancel={cancel} adding />}
<div className="bg-slate-800/60 border border-slate-700 rounded-lg overflow-x-auto">
<table className="w-full text-sm text-slate-200">
<thead className="bg-slate-900 text-xs text-slate-400 uppercase">
<tr>
<th className="px-3 py-2 text-left">Name</th>
<th className="px-3 py-2 text-right">Lat</th>
<th className="px-3 py-2 text-right">Lon</th>
<th className="px-3 py-2 text-center">State</th>
<th className="px-3 py-2 text-center">On</th>
<th className="px-3 py-2"></th>
</tr>
</thead>
<tbody className="divide-y divide-slate-700/60">
{rows.map(r => editing === r.anchor_id ? (
<tr key={r.anchor_id} className="bg-slate-900/40">
<td colSpan={6} className="px-3 py-2"><RowEditor draft={draft} setDraft={setDraft} onSave={save} onCancel={cancel} /></td>
</tr>
) : (
<tr key={r.anchor_id} className="hover:bg-slate-800/50">
<td className="px-3 py-2 capitalize">{r.name}</td>
<td className="px-3 py-2 text-right text-xs">{r.lat.toFixed(4)}</td>
<td className="px-3 py-2 text-right text-xs">{r.lon.toFixed(4)}</td>
<td className="px-3 py-2 text-center text-xs">{r.state || '-'}</td>
<td className="px-3 py-2 text-center">{r.enabled ? <Check className="w-4 h-4 text-emerald-400 inline" /> : <X className="w-4 h-4 text-slate-500 inline" />}</td>
<td className="px-3 py-2 text-right">
<button onClick={() => beginEdit(r)} className="text-cyan-400 hover:text-cyan-300 text-xs mr-3">Edit</button>
<button onClick={() => remove(r.anchor_id)} className="text-red-400 hover:text-red-300"><Trash2 className="w-4 h-4 inline" /></button>
</td>
</tr>
))}
</tbody>
</table>
</div>
</div>
)
}
function RowEditor({ draft, setDraft, onSave, onCancel, adding }: {
draft: TownAnchor, setDraft: (t: TownAnchor) => void,
onSave: () => void, onCancel: () => void, adding?: boolean,
}) {
const upd = (k: keyof TownAnchor, v: unknown) => setDraft({ ...draft, [k]: v })
return (
<div className="grid grid-cols-2 md:grid-cols-4 gap-2 p-3 bg-slate-900/50 rounded">
<label className="text-xs text-slate-400 col-span-2">Name (lowercased on save)
<input className="block w-full mt-1 bg-slate-800 border border-slate-700 rounded px-2 py-1 text-slate-100"
value={draft.name} onChange={e => upd('name', e.target.value)} disabled={!adding} />
</label>
<label className="text-xs text-slate-400">State
<input className="block w-full mt-1 bg-slate-800 border border-slate-700 rounded px-2 py-1 text-slate-100"
value={draft.state ?? ''} onChange={e => upd('state', e.target.value)} />
</label>
<label className="text-xs text-slate-400 flex items-center gap-2">
<input type="checkbox" checked={draft.enabled} onChange={e => upd('enabled', e.target.checked)} className="accent-cyan-500 mt-4" />
Enabled
</label>
<label className="text-xs text-slate-400">Lat
<input type="number" step="any" className="block w-full mt-1 bg-slate-800 border border-slate-700 rounded px-2 py-1 text-slate-100"
value={draft.lat} onChange={e => upd('lat', parseFloat(e.target.value))} />
</label>
<label className="text-xs text-slate-400">Lon
<input type="number" step="any" className="block w-full mt-1 bg-slate-800 border border-slate-700 rounded px-2 py-1 text-slate-100"
value={draft.lon} onChange={e => upd('lon', parseFloat(e.target.value))} />
</label>
<div className="col-span-2 flex items-center justify-end gap-2 mt-2">
<button onClick={onCancel} className="px-3 py-1 text-slate-300 hover:bg-slate-700 rounded text-sm">Cancel</button>
<button onClick={onSave} className="px-3 py-1 bg-cyan-700 hover:bg-cyan-600 text-white rounded text-sm">Save</button>
</div>
</div>
)
}

132
meshai/central/idaho_gauge_sites.py
View file

@ -1,122 +1,50 @@
"""v0.5.12 Idaho gauge-site curation (STARTER SUBSET).
"""v0.6-4 Idaho gauge-site lookups (now backed by gauge_sites table).
9 high-priority Magic Valley + Treasure Valley + Salmon-Challis + Snake River
system gauges. Threshold values (action / flood_minor / flood_moderate /
flood_major) sourced from NWS-AHPS pages for each site, in feet (gage
height, parameter_code 00065).
The IDAHO_CURATED_SITES Python dict was migrated to the gauge_sites SQLite
table in v8.sql. seed_gauge_sites() (called from init_db on first boot)
populates the table with the original 9 sites. The lookup helpers below
read from the table via meshai.persistence.curation.
**STARTER SUBSET** -- expand via NWS-AHPS curation in v0.6.x. If a site is
missing here, the handler ignores it (no broadcast). v0.6.x will likely
migrate this dict to a `gauge_sites` table so non-engineers can curate via
the GUI.
Convention for site_id keys:
USGS-prefixed, zero-padded as USGS publishes them (e.g. 'USGS-13139510').
The handler normalizes incoming envelope site IDs to this form before
lookup so both 'USGS-13139510' and '13139510' resolve.
Threshold values that the gauge doesn't have (e.g. flood_major above the
top observed historic crest) are left as None -- the handler treats None as
'this threshold doesn't apply at this site' so a reading can never enter
that band.
Module exports retained for backward-compat with existing tests:
lookup_site, normalize_site_id, THRESHOLD_RANK, compute_threshold_state.
The IDAHO_CURATED_SITES name itself is gone -- new code should call
lookup_site() (DB-backed) or the curation accessor directly.
"""
from typing import Optional
# site_id -> {gauge_name, lat, lon, action_ft, flood_minor_ft,
# flood_moderate_ft, flood_major_ft}
IDAHO_CURATED_SITES: dict = {
"USGS-13139510": {
"gauge_name": "Big Lost River near Mackay",
"lat": 43.910, "lon": -113.620,
"action_ft": 5.5, "flood_minor_ft": 7.0,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13186000": {
"gauge_name": "Snake River at Heise",
"lat": 43.612, "lon": -111.654,
"action_ft": 12.0, "flood_minor_ft": 14.0,
"flood_moderate_ft": 16.0, "flood_major_ft": None,
},
"USGS-13037500": {
"gauge_name": "Snake River at Idaho Falls",
"lat": 43.500, "lon": -112.034,
"action_ft": 8.5, "flood_minor_ft": 10.0,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13135500": {
"gauge_name": "Big Wood River near Hailey",
"lat": 43.533, "lon": -114.318,
"action_ft": 6.0, "flood_minor_ft": 7.5,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13205000": {
"gauge_name": "Boise River near Boise",
"lat": 43.690, "lon": -116.200,
"action_ft": 8.0, "flood_minor_ft": 10.5,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13247500": {
"gauge_name": "Payette River at Banks",
"lat": 44.080, "lon": -116.130,
"action_ft": 10.0, "flood_minor_ft": 12.0,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13057000": {
"gauge_name": "Henrys Fork near Rexburg",
"lat": 43.831, "lon": -111.781,
"action_ft": 9.0, "flood_minor_ft": 10.5,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13162225": {
"gauge_name": "Salmon Falls Creek near San Jacinto",
"lat": 42.180, "lon": -114.850,
"action_ft": 8.0, "flood_minor_ft": 10.0,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13083000": {
"gauge_name": "Bear River near Border WY/ID",
"lat": 42.214, "lon": -111.045,
"action_ft": 6.0, "flood_minor_ft": 8.0,
"flood_moderate_ft": None, "flood_major_ft": None,
},
}
def normalize_site_id(raw: Optional[str]) -> Optional[str]:
"""Accept 'USGS-13139510', 'USGS:13139510', '13139510', etc. Return the
canonical 'USGS-<id>' form so the curation dict lookups succeed."""
if not raw: return None
s = str(raw).strip()
# Already canonical -- return as-is for the fast path.
if s in IDAHO_CURATED_SITES: return s
# Strip common prefix variants.
for prefix in ("USGS-", "USGS:", "USGS_", "usgs-", "usgs:", "usgs_"):
if s.startswith(prefix): s = s[len(prefix):]; break
canonical = f"USGS-{s}"
return canonical
def lookup_site(raw_site_id: str) -> Optional[dict]:
"""Return the curated-site dict for a raw envelope site_id, or None when
the site is not in the curated subset."""
sid = normalize_site_id(raw_site_id)
if sid is None: return None
return IDAHO_CURATED_SITES.get(sid)
# Ordered list of threshold names from low to high. Used to compare
# "is current threshold higher than prior" (upward crossing detection).
THRESHOLD_RANK = ["normal", "action", "flood_minor", "flood_moderate", "flood_major"]
def normalize_site_id(raw: Optional[str]) -> Optional[str]:
"""Accept 'USGS-13139510', 'USGS:13139510', '13139510', etc. Return the
canonical 'USGS-<id>' form so the curation table lookups succeed."""
if not raw: return None
s = str(raw).strip()
for prefix in ("USGS-", "USGS:", "USGS_", "usgs-", "usgs:", "usgs_"):
if s.startswith(prefix): s = s[len(prefix):]; break
return f"USGS-{s}"
def lookup_site(raw_site_id: str) -> Optional[dict]:
"""Return the curated-site dict for a raw envelope site_id, or None when
the site is not in the curated subset (or is disabled).
v0.6-4: reads from the gauge_sites SQLite table via the curation accessor."""
sid = normalize_site_id(raw_site_id)
if sid is None: return None
from meshai.persistence.curation import lookup_gauge_site
return lookup_gauge_site(sid)
def compute_threshold_state(value_ft: float, site_thresholds: dict) -> str:
"""Bucket a gage_height reading (ft) into a NWS-AHPS threshold state."""
a = site_thresholds.get("action_ft")
mn = site_thresholds.get("flood_minor_ft")
md = site_thresholds.get("flood_moderate_ft")
mj = site_thresholds.get("flood_major_ft")
# Higher thresholds win first.
if mj is not None and value_ft >= mj: return "flood_major"
if md is not None and value_ft >= md: return "flood_moderate"
if mn is not None and value_ft >= mn: return "flood_minor"

1
meshai/central/nwis_handler.py
View file

@ -42,7 +42,6 @@ from datetime import datetime
from typing import Any, Optional
from meshai.central.idaho_gauge_sites import (
IDAHO_CURATED_SITES,
THRESHOLD_RANK,
compute_threshold_state,
lookup_site,

41
meshai/central_normalizer.py
View file

@ -164,42 +164,8 @@ def _clean_description(raw: Optional[str]) -> Optional[str]:
# ---------- distance / bearing --------------------------------------------
# Small lookup of Idaho (+ a few neighbor) towns -> (lat, lon). Used to
# render "X mi <bearing> of <town>" when the reverse-geocoder picked a
# city we know. Built from the geocoder.city values seen across 60-sample
# probe + major Idaho cities the operator's mesh is most likely to care
# about. Misses fall through silently: distance_mi/bearing stay None.
_TOWN_COORDS: dict[str, tuple[float, float]] = {
"boise": (43.6150, -116.2023),
"meridian": (43.6121, -116.3915),
"nampa": (43.5407, -116.5635),
"caldwell": (43.6629, -116.6874),
"idaho falls": (43.4666, -112.0340),
"pocatello": (42.8713, -112.4455),
"twin falls": (42.5630, -114.4609),
"coeur d'alene": (47.6777, -116.7805),
"lewiston": (46.4165, -117.0177),
"moscow": (46.7324, -117.0002),
"sandpoint": (48.2766, -116.5535),
"post falls": (47.7180, -116.9516),
"hayden": (47.7660, -116.7866),
"rathdrum": (47.8121, -116.8950),
"plummer": (47.3344, -116.8856),
"kellogg": (47.5380, -116.1352),
"bonners ferry": (48.6914, -116.3181),
"rexburg": (43.8260, -111.7897),
"blackfoot": (43.1905, -112.3447),
"burley": (42.5360, -113.7928),
"jerome": (42.7252, -114.5187),
"mountain home": (43.1330, -115.6912),
"stanley": (44.2160, -114.9311),
"salmon": (45.1758, -113.8957),
"mccall": (44.9111, -116.0987),
"weiser": (44.2510, -116.9690),
"soda springs": (42.6543, -111.6047),
"preston": (42.0963, -111.8766),
"montpelier": (42.3232, -111.2980),
}
# v0.6-4: town_anchors moved to a GUI-editable SQLite table. Lookups go
# through meshai.persistence.curation.lookup_town_anchor() now.
def _haversine_miles(lat1: float, lon1: float, lat2: float, lon2: float) -> float:
@ -230,7 +196,8 @@ def _compute_distance_bearing(
if event_lat is None or event_lon is None or not town:
return None, None
key = str(town).strip().lower()
coords = _TOWN_COORDS.get(key)
from meshai.persistence.curation import lookup_town_anchor
coords = lookup_town_anchor(key)
if coords is None:
return None, None
tlat, tlon = coords

280
meshai/dashboard/api/curation_routes.py Normal file
View file

@ -0,0 +1,280 @@
"""v0.6-4 REST API for gauge_sites + town_anchors curation tables.
Endpoints (mirrors the adapter_config CRUD shape):
gauge_sites:
GET /api/gauge-sites list (enabled and disabled)
GET /api/gauge-sites/{site_id} single row
POST /api/gauge-sites add new row
PUT /api/gauge-sites/{site_id} partial update
DELETE /api/gauge-sites/{site_id} remove row
town_anchors:
GET /api/town-anchors list
GET /api/town-anchors/{anchor_id} single
POST /api/town-anchors add new
PUT /api/town-anchors/{anchor_id} partial update
DELETE /api/town-anchors/{anchor_id} remove
Every mutating call invalidates the in-process curation cache so
handler-side reads (lookup_gauge_site, lookup_town_anchor) see the new
state on the next call -- no container restart.
"""
from __future__ import annotations
import logging
import time
from typing import Any, Optional
from fastapi import APIRouter, HTTPException, Request
from meshai.persistence.curation import invalidate_curation_cache
logger = logging.getLogger(__name__)
router = APIRouter(tags=["curation"])
def _get_conn():
from meshai.persistence import get_db
return get_db()
def _gauge_row_to_dict(r) -> dict:
return {
"site_id": r["site_id"],
"gauge_name": r["gauge_name"],
"lat": r["lat"],
"lon": r["lon"],
"action_ft": r["action_ft"],
"flood_minor_ft": r["flood_minor_ft"],
"flood_moderate_ft": r["flood_moderate_ft"],
"flood_major_ft": r["flood_major_ft"],
"enabled": bool(r["enabled"]),
"updated_at": r["updated_at"],
}
def _town_row_to_dict(r) -> dict:
return {
"anchor_id": r["anchor_id"],
"name": r["name"],
"lat": r["lat"],
"lon": r["lon"],
"state": r["state"],
"enabled": bool(r["enabled"]),
"updated_at": r["updated_at"],
}
# ============================================================================
# gauge_sites
# ============================================================================
@router.get("/gauge-sites")
async def list_gauges(request: Request) -> list[dict]:
conn = _get_conn()
return [_gauge_row_to_dict(r) for r in conn.execute(
"SELECT * FROM gauge_sites ORDER BY site_id"
).fetchall()]
@router.get("/gauge-sites/{site_id}")
async def get_gauge(site_id: str, request: Request) -> dict:
conn = _get_conn()
r = conn.execute(
"SELECT * FROM gauge_sites WHERE site_id=?", (site_id,)
).fetchone()
if r is None:
raise HTTPException(404, f"gauge_sites: {site_id} not found")
return _gauge_row_to_dict(r)
@router.post("/gauge-sites")
async def add_gauge(request: Request) -> dict:
body = await request.json()
if not isinstance(body, dict): raise HTTPException(400, "body must be a JSON object")
required = ("site_id", "gauge_name", "lat", "lon")
for f in required:
if f not in body:
raise HTTPException(400, f"missing required field: {f}")
if not isinstance(body["site_id"], str) or not body["site_id"].strip():
raise HTTPException(400, "site_id must be a non-empty string")
try:
lat = float(body["lat"]); lon = float(body["lon"])
except (TypeError, ValueError):
raise HTTPException(400, "lat/lon must be numeric")
conn = _get_conn()
try:
conn.execute(
"INSERT INTO gauge_sites(site_id, gauge_name, lat, lon, action_ft, "
"flood_minor_ft, flood_moderate_ft, flood_major_ft, enabled, updated_at) "
"VALUES (?,?,?,?,?,?,?,?,?,?)",
(body["site_id"].strip(),
str(body["gauge_name"]).strip(),
lat, lon,
body.get("action_ft"),
body.get("flood_minor_ft"),
body.get("flood_moderate_ft"),
body.get("flood_major_ft"),
1 if body.get("enabled", True) else 0,
time.time()),
)
except Exception as e:
raise HTTPException(400, f"insert failed: {e}")
invalidate_curation_cache()
return await get_gauge(body["site_id"].strip(), request)
@router.put("/gauge-sites/{site_id}")
async def update_gauge(site_id: str, request: Request) -> dict:
body = await request.json()
if not isinstance(body, dict): raise HTTPException(400, "body must be a JSON object")
conn = _get_conn()
existing = conn.execute(
"SELECT * FROM gauge_sites WHERE site_id=?", (site_id,)
).fetchone()
if existing is None:
raise HTTPException(404, f"gauge_sites: {site_id} not found")
fields = []
args: list[Any] = []
for col, validator in (
("gauge_name", lambda v: str(v).strip()),
("lat", float), ("lon", float),
("action_ft", lambda v: float(v) if v is not None else None),
("flood_minor_ft", lambda v: float(v) if v is not None else None),
("flood_moderate_ft", lambda v: float(v) if v is not None else None),
("flood_major_ft", lambda v: float(v) if v is not None else None),
("enabled", lambda v: 1 if bool(v) else 0),
):
if col in body:
try: args.append(validator(body[col]))
except (TypeError, ValueError):
raise HTTPException(400, f"invalid value for {col}: {body[col]!r}")
fields.append(f"{col}=?")
if not fields:
raise HTTPException(400, "no editable fields provided")
fields.append("updated_at=?")
args.append(time.time())
args.append(site_id)
conn.execute(
f"UPDATE gauge_sites SET {', '.join(fields)} WHERE site_id=?",
tuple(args),
)
invalidate_curation_cache()
return await get_gauge(site_id, request)
@router.delete("/gauge-sites/{site_id}")
async def delete_gauge(site_id: str, request: Request) -> dict:
conn = _get_conn()
cur = conn.execute("DELETE FROM gauge_sites WHERE site_id=?", (site_id,))
if cur.rowcount == 0:
raise HTTPException(404, f"gauge_sites: {site_id} not found")
invalidate_curation_cache()
return {"deleted": site_id}
# ============================================================================
# town_anchors
# ============================================================================
@router.get("/town-anchors")
async def list_towns(request: Request) -> list[dict]:
conn = _get_conn()
return [_town_row_to_dict(r) for r in conn.execute(
"SELECT * FROM town_anchors ORDER BY name"
).fetchall()]
@router.get("/town-anchors/{anchor_id}")
async def get_town(anchor_id: int, request: Request) -> dict:
conn = _get_conn()
r = conn.execute(
"SELECT * FROM town_anchors WHERE anchor_id=?", (anchor_id,)
).fetchone()
if r is None:
raise HTTPException(404, f"town_anchors: {anchor_id} not found")
return _town_row_to_dict(r)
@router.post("/town-anchors")
async def add_town(request: Request) -> dict:
body = await request.json()
if not isinstance(body, dict): raise HTTPException(400, "body must be a JSON object")
for f in ("name", "lat", "lon"):
if f not in body: raise HTTPException(400, f"missing required field: {f}")
name = str(body["name"]).strip().lower()
if not name: raise HTTPException(400, "name must be non-empty")
try:
lat = float(body["lat"]); lon = float(body["lon"])
except (TypeError, ValueError):
raise HTTPException(400, "lat/lon must be numeric")
conn = _get_conn()
try:
cur = conn.execute(
"INSERT INTO town_anchors(name, lat, lon, state, enabled, updated_at) "
"VALUES (?,?,?,?,?,?)",
(name, lat, lon, body.get("state"),
1 if body.get("enabled", True) else 0, time.time()),
)
except Exception as e:
raise HTTPException(400, f"insert failed: {e}")
invalidate_curation_cache()
return await get_town(cur.lastrowid, request)
@router.put("/town-anchors/{anchor_id}")
async def update_town(anchor_id: int, request: Request) -> dict:
body = await request.json()
if not isinstance(body, dict): raise HTTPException(400, "body must be a JSON object")
conn = _get_conn()
existing = conn.execute(
"SELECT * FROM town_anchors WHERE anchor_id=?", (anchor_id,)
).fetchone()
if existing is None:
raise HTTPException(404, f"town_anchors: {anchor_id} not found")
fields = []
args: list[Any] = []
for col, validator in (
("name", lambda v: str(v).strip().lower()),
("lat", float), ("lon", float),
("state", lambda v: str(v).strip() if v else None),
("enabled", lambda v: 1 if bool(v) else 0),
):
if col in body:
try: args.append(validator(body[col]))
except (TypeError, ValueError):
raise HTTPException(400, f"invalid value for {col}: {body[col]!r}")
fields.append(f"{col}=?")
if not fields:
raise HTTPException(400, "no editable fields provided")
fields.append("updated_at=?")
args.append(time.time())
args.append(anchor_id)
conn.execute(
f"UPDATE town_anchors SET {', '.join(fields)} WHERE anchor_id=?",
tuple(args),
)
invalidate_curation_cache()
return await get_town(anchor_id, request)
@router.delete("/town-anchors/{anchor_id}")
async def delete_town(anchor_id: int, request: Request) -> dict:
conn = _get_conn()
cur = conn.execute("DELETE FROM town_anchors WHERE anchor_id=?", (anchor_id,))
if cur.rowcount == 0:
raise HTTPException(404, f"town_anchors: {anchor_id} not found")
invalidate_curation_cache()
return {"deleted": anchor_id}

2
meshai/dashboard/server.py
View file

@ -1,5 +1,6 @@
"""FastAPI server for MeshAI dashboard."""
from meshai.dashboard.api.adapter_config_routes import router as adapter_config_router
from meshai.dashboard.api.curation_routes import router as curation_router
import asyncio
import logging
@ -57,6 +58,7 @@ def create_app() -> FastAPI:
app.include_router(system_router, prefix="/api")
app.include_router(adapter_config_router, prefix="/api")
app.include_router(curation_router, prefix="/api")
app.include_router(config_router, prefix="/api")
app.include_router(mesh_router, prefix="/api")
app.include_router(env_router, prefix="/api")

1
meshai/dashboard/static/assets/index-B1y0CpOn.css
View file

File diff suppressed because one or more lines are too long

1
meshai/dashboard/static/assets/index-Bj-HMHAO.css Normal file
View file

File diff suppressed because one or more lines are too long

272
meshai/dashboard/static/assets/index-B7WUE5ni.js → meshai/dashboard/static/assets/index-Dc1UcqB9.js
View file

File diff suppressed because one or more lines are too long

4
meshai/dashboard/static/index.html
View file

@ -8,8 +8,8 @@
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;500;600;700&display=swap" rel="stylesheet">
<script type="module" crossorigin src="/assets/index-B7WUE5ni.js"></script>
<link rel="stylesheet" crossorigin href="/assets/index-B1y0CpOn.css">
<script type="module" crossorigin src="/assets/index-Dc1UcqB9.js"></script>
<link rel="stylesheet" crossorigin href="/assets/index-Bj-HMHAO.css">
</head>
<body>
<div id="root"></div>

241
meshai/persistence/curation.py Normal file
View file

@ -0,0 +1,241 @@
"""v0.6-4 curation accessors + seed routines.
Both tables (gauge_sites, town_anchors) follow the same pattern as
adapter_config: created by a migration, seeded from Python data on first
boot, then runtime reads from SQLite via cached accessors.
gauge_sites replaces idaho_gauge_sites.IDAHO_CURATED_SITES.
town_anchors replaces central_normalizer._TOWN_COORDS.
"""
from __future__ import annotations
import logging
import sqlite3
import threading
import time
from typing import Any, Optional
logger = logging.getLogger(__name__)
# ============================================================================
# Seed data (the dict that used to live in handler code)
# ============================================================================
# Idaho stream-gauge sites originally from
# meshai/central/idaho_gauge_sites.py:IDAHO_CURATED_SITES (v0.5.12).
_GAUGE_SITES_SEED: dict[str, dict[str, Any]] = {
"USGS-13139510": {
"gauge_name": "Big Lost River near Mackay",
"lat": 43.910, "lon": -113.620,
"action_ft": 5.5, "flood_minor_ft": 7.0,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13186000": {
"gauge_name": "Snake River at Heise",
"lat": 43.612, "lon": -111.654,
"action_ft": 12.0, "flood_minor_ft": 14.0,
"flood_moderate_ft": 16.0, "flood_major_ft": None,
},
"USGS-13037500": {
"gauge_name": "Snake River at Idaho Falls",
"lat": 43.500, "lon": -112.034,
"action_ft": 8.5, "flood_minor_ft": 10.0,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13135500": {
"gauge_name": "Big Wood River near Hailey",
"lat": 43.533, "lon": -114.318,
"action_ft": 6.0, "flood_minor_ft": 7.5,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13205000": {
"gauge_name": "Boise River near Boise",
"lat": 43.690, "lon": -116.200,
"action_ft": 8.0, "flood_minor_ft": 10.5,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13247500": {
"gauge_name": "Payette River at Banks",
"lat": 44.080, "lon": -116.130,
"action_ft": 10.0, "flood_minor_ft": 12.0,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13057000": {
"gauge_name": "Henrys Fork near Rexburg",
"lat": 43.831, "lon": -111.781,
"action_ft": 9.0, "flood_minor_ft": 10.5,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13162225": {
"gauge_name": "Salmon Falls Creek near San Jacinto",
"lat": 42.180, "lon": -114.850,
"action_ft": 8.0, "flood_minor_ft": 10.0,
"flood_moderate_ft": None, "flood_major_ft": None,
},
"USGS-13083000": {
"gauge_name": "Bear River near Border WY/ID",
"lat": 42.214, "lon": -111.045,
"action_ft": 6.0, "flood_minor_ft": 8.0,
"flood_moderate_ft": None, "flood_major_ft": None,
},
}
# Idaho + neighbor towns originally from central_normalizer._TOWN_COORDS.
_TOWN_ANCHORS_SEED: dict[str, dict[str, Any]] = {
"boise": {"lat": 43.6150, "lon": -116.2023, "state": "ID"},
"meridian": {"lat": 43.6121, "lon": -116.3915, "state": "ID"},
"nampa": {"lat": 43.5407, "lon": -116.5635, "state": "ID"},
"caldwell": {"lat": 43.6629, "lon": -116.6874, "state": "ID"},
"idaho falls": {"lat": 43.4666, "lon": -112.0340, "state": "ID"},
"pocatello": {"lat": 42.8713, "lon": -112.4455, "state": "ID"},
"twin falls": {"lat": 42.5630, "lon": -114.4609, "state": "ID"},
"coeur d'alene": {"lat": 47.6777, "lon": -116.7805, "state": "ID"},
"lewiston": {"lat": 46.4165, "lon": -117.0177, "state": "ID"},
"moscow": {"lat": 46.7324, "lon": -117.0002, "state": "ID"},
"sandpoint": {"lat": 48.2766, "lon": -116.5535, "state": "ID"},
"post falls": {"lat": 47.7180, "lon": -116.9516, "state": "ID"},
"hayden": {"lat": 47.7660, "lon": -116.7866, "state": "ID"},
"rathdrum": {"lat": 47.8121, "lon": -116.8950, "state": "ID"},
"plummer": {"lat": 47.3344, "lon": -116.8856, "state": "ID"},
"kellogg": {"lat": 47.5380, "lon": -116.1352, "state": "ID"},
"bonners ferry": {"lat": 48.6914, "lon": -116.3181, "state": "ID"},
"rexburg": {"lat": 43.8260, "lon": -111.7897, "state": "ID"},
"blackfoot": {"lat": 43.1905, "lon": -112.3447, "state": "ID"},
"burley": {"lat": 42.5360, "lon": -113.7928, "state": "ID"},
"jerome": {"lat": 42.7252, "lon": -114.5187, "state": "ID"},
"mountain home": {"lat": 43.1330, "lon": -115.6912, "state": "ID"},
"stanley": {"lat": 44.2160, "lon": -114.9311, "state": "ID"},
"salmon": {"lat": 45.1758, "lon": -113.8957, "state": "ID"},
"mccall": {"lat": 44.9111, "lon": -116.0987, "state": "ID"},
"weiser": {"lat": 44.2510, "lon": -116.9690, "state": "ID"},
"soda springs": {"lat": 42.6543, "lon": -111.6047, "state": "ID"},
"preston": {"lat": 42.0963, "lon": -111.8766, "state": "ID"},
"montpelier": {"lat": 42.3232, "lon": -111.2980, "state": "ID"},
}
# ============================================================================
# Caches
# ============================================================================
_LOCK = threading.Lock()
_gauge_cache: Optional[dict[str, dict[str, Any]]] = None
_town_cache: Optional[dict[str, tuple[float, float]]] = None
def invalidate_curation_cache() -> None:
"""Drop the in-memory caches. Called by the REST API on POST/PUT/DELETE."""
global _gauge_cache, _town_cache
with _LOCK:
_gauge_cache = None
_town_cache = None
def _load_gauge_cache() -> dict[str, dict[str, Any]]:
global _gauge_cache
if _gauge_cache is not None:
return _gauge_cache
try:
from meshai.persistence import get_db
conn = get_db()
rows = conn.execute(
"SELECT site_id, gauge_name, lat, lon, action_ft, flood_minor_ft, "
"flood_moderate_ft, flood_major_ft FROM gauge_sites WHERE enabled=1"
).fetchall()
cache = {r["site_id"]: {
"gauge_name": r["gauge_name"],
"lat": r["lat"], "lon": r["lon"],
"action_ft": r["action_ft"],
"flood_minor_ft": r["flood_minor_ft"],
"flood_moderate_ft": r["flood_moderate_ft"],
"flood_major_ft": r["flood_major_ft"],
} for r in rows}
except Exception:
logger.exception("curation: gauge_sites cache load failed; using empty")
cache = {}
with _LOCK:
_gauge_cache = cache
return cache
def _load_town_cache() -> dict[str, tuple[float, float]]:
global _town_cache
if _town_cache is not None:
return _town_cache
try:
from meshai.persistence import get_db
conn = get_db()
rows = conn.execute(
"SELECT name, lat, lon FROM town_anchors WHERE enabled=1"
).fetchall()
cache = {r["name"].lower(): (r["lat"], r["lon"]) for r in rows}
except Exception:
logger.exception("curation: town_anchors cache load failed; using empty")
cache = {}
with _LOCK:
_town_cache = cache
return cache
# ============================================================================
# Lookups (called from handlers)
# ============================================================================
def lookup_gauge_site(site_id: str) -> Optional[dict[str, Any]]:
"""Return the row dict for `site_id` (canonical 'USGS-...' form) or None."""
cache = _load_gauge_cache()
return cache.get(site_id)
def lookup_town_anchor(name: str) -> Optional[tuple[float, float]]:
"""Return (lat, lon) for the lowercased town name, or None."""
if not name: return None
cache = _load_town_cache()
return cache.get(name.strip().lower())
# ============================================================================
# Seed routines (called from init_db after migrations)
# ============================================================================
def seed_gauge_sites(conn: sqlite3.Connection) -> int:
"""INSERT OR IGNORE one row per _GAUGE_SITES_SEED entry. Idempotent."""
now = time.time()
inserted = 0
for site_id, spec in _GAUGE_SITES_SEED.items():
cur = conn.execute(
"INSERT OR IGNORE INTO gauge_sites("
"site_id, gauge_name, lat, lon, action_ft, flood_minor_ft, "
"flood_moderate_ft, flood_major_ft, enabled, updated_at) "
"VALUES (?,?,?,?,?,?,?,?,?,?)",
(site_id, spec["gauge_name"], spec["lat"], spec["lon"],
spec.get("action_ft"), spec.get("flood_minor_ft"),
spec.get("flood_moderate_ft"), spec.get("flood_major_ft"),
1, now),
)
if cur.rowcount > 0:
inserted += 1
if inserted:
logger.info("curation: seeded %d gauge_sites rows", inserted)
return inserted
def seed_town_anchors(conn: sqlite3.Connection) -> int:
"""INSERT OR IGNORE one row per _TOWN_ANCHORS_SEED entry. Idempotent."""
now = time.time()
inserted = 0
for name, spec in _TOWN_ANCHORS_SEED.items():
cur = conn.execute(
"INSERT OR IGNORE INTO town_anchors("
"name, lat, lon, state, enabled, updated_at) "
"VALUES (?,?,?,?,?,?)",
(name, spec["lat"], spec["lon"], spec.get("state"), 1, now),
)
if cur.rowcount > 0:
inserted += 1
if inserted:
logger.info("curation: seeded %d town_anchors rows", inserted)
return inserted

8
meshai/persistence/db.py
View file

@ -30,7 +30,7 @@ logger = logging.getLogger(__name__)
DEFAULT_DB_PATH = "/data/meshai.sqlite"
MESHAI_DB_PATH_ENV = "MESHAI_DB_PATH"
SCHEMA_VERSION = 7
SCHEMA_VERSION = 9
SCHEMA_META_TABLE = "schema_meta"
MIGRATIONS_DIR = Path(__file__).parent / "migrations"
@ -134,6 +134,12 @@ def init_db(path: Optional[str] = None) -> sqlite3.Connection:
prune_orphans(conn)
except Exception:
logger.exception("init_db: adapter_config seed/prune failed")
try:
from meshai.persistence.curation import seed_gauge_sites, seed_town_anchors
seed_gauge_sites(conn)
seed_town_anchors(conn)
except Exception:
logger.exception("init_db: curation seed failed")
return conn

28
meshai/persistence/migrations/v8.sql Normal file
View file

@ -0,0 +1,28 @@
-- v0.6-4 gauge_sites curation table.
--
-- Replaces the hardcoded IDAHO_CURATED_SITES dict in
-- meshai/central/idaho_gauge_sites.py with a GUI-editable SQLite table.
-- The Python dict is removed in this commit; the lookup helpers in
-- idaho_gauge_sites.py now read from this table via
-- meshai.persistence.curation.lookup_gauge_site().
--
-- Per-site NWS-AHPS thresholds (action / minor / moderate / major) are
-- nullable so sites without a published value for a band can sit on the
-- table without a fake value. Disabled rows (enabled=0) are skipped at
-- read time so the operator can pause a site without deleting it.
CREATE TABLE IF NOT EXISTS gauge_sites (
site_id TEXT PRIMARY KEY, -- 'USGS-13139510' canonical
gauge_name TEXT NOT NULL,
lat REAL NOT NULL,
lon REAL NOT NULL,
action_ft REAL,
flood_minor_ft REAL,
flood_moderate_ft REAL,
flood_major_ft REAL,
enabled INTEGER NOT NULL DEFAULT 1,
updated_at REAL NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_gauge_sites_enabled
ON gauge_sites(enabled);

24
meshai/persistence/migrations/v9.sql Normal file
View file

@ -0,0 +1,24 @@
-- v0.6-4 town_anchors curation table.
--
-- Replaces the hardcoded _TOWN_COORDS dict in central_normalizer.py.
-- The Python dict is removed in this commit; the lookup helpers in
-- central_normalizer.py now read from this table via
-- meshai.persistence.curation.lookup_town_anchor().
--
-- name TEXT is the canonical lookup key (lowercased) -- the existing
-- _compute_distance_bearing() already lowercases its town input. The
-- adapter accepting both 'Boise' and 'boise' is preserved because the
-- accessor lowercases before query.
CREATE TABLE IF NOT EXISTS town_anchors (
anchor_id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL UNIQUE,
lat REAL NOT NULL,
lon REAL NOT NULL,
state TEXT,
enabled INTEGER NOT NULL DEFAULT 1,
updated_at REAL NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_town_anchors_enabled
ON town_anchors(enabled);

4
tests/test_adapter_config_foundation.py
View file

@ -54,11 +54,11 @@ def test_v6_tables_exist(fresh_db):
assert "adapter_meta" in tables
def test_schema_meta_at_v7(fresh_db):
def test_schema_meta_at_v9(fresh_db):
v = fresh_db.execute(
"SELECT value FROM schema_meta WHERE key='version'"
).fetchone()["value"]
assert int(v) == 7
assert int(v) == 9
def test_adapter_config_type_check_constrains_vocabulary(fresh_db):

214
tests/test_curation.py Normal file
View file

@ -0,0 +1,214 @@
"""v0.6-4 curation tests: schema, seed, accessors, REST API."""
from __future__ import annotations
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
from meshai.persistence.curation import (
invalidate_curation_cache,
lookup_gauge_site,
lookup_town_anchor,
seed_gauge_sites,
seed_town_anchors,
_GAUGE_SITES_SEED,
_TOWN_ANCHORS_SEED,
)
from meshai.persistence import get_db
from meshai.dashboard.api.curation_routes import router as curation_router
# ============================================================================
# Schema + seed
# ============================================================================
def test_v8_v9_tables_present():
conn = get_db()
tables = {r["name"] for r in conn.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
).fetchall()}
assert "gauge_sites" in tables
assert "town_anchors" in tables
def test_gauge_sites_seeded_at_init():
conn = get_db()
n = conn.execute("SELECT COUNT(*) FROM gauge_sites").fetchone()[0]
assert n == len(_GAUGE_SITES_SEED)
def test_town_anchors_seeded_at_init():
conn = get_db()
n = conn.execute("SELECT COUNT(*) FROM town_anchors").fetchone()[0]
assert n == len(_TOWN_ANCHORS_SEED)
def test_seed_idempotent():
conn = get_db()
a = seed_gauge_sites(conn)
b = seed_town_anchors(conn)
assert a == 0 and b == 0
def test_seed_does_not_overwrite_user_edits():
conn = get_db()
conn.execute(
"UPDATE gauge_sites SET action_ft=99 WHERE site_id='USGS-13186000'"
)
seed_gauge_sites(conn)
r = conn.execute(
"SELECT action_ft FROM gauge_sites WHERE site_id='USGS-13186000'"
).fetchone()
assert r["action_ft"] == 99
# ============================================================================
# Accessors
# ============================================================================
def test_lookup_gauge_site_hits():
invalidate_curation_cache()
r = lookup_gauge_site("USGS-13186000")
assert r is not None
assert r["gauge_name"] == "Snake River at Heise"
assert r["action_ft"] == 12.0
def test_lookup_gauge_site_miss():
invalidate_curation_cache()
assert lookup_gauge_site("USGS-99999") is None
def test_lookup_gauge_disabled_row_invisible():
conn = get_db()
conn.execute("UPDATE gauge_sites SET enabled=0 WHERE site_id='USGS-13186000'")
invalidate_curation_cache()
assert lookup_gauge_site("USGS-13186000") is None
def test_lookup_town_anchor_hits():
invalidate_curation_cache()
coord = lookup_town_anchor("Boise")
assert coord is not None
assert coord[0] == pytest.approx(43.6150)
assert coord[1] == pytest.approx(-116.2023)
def test_lookup_town_anchor_case_insensitive():
invalidate_curation_cache()
a = lookup_town_anchor("MERIDIAN")
b = lookup_town_anchor("meridian")
assert a == b
def test_lookup_town_anchor_miss():
invalidate_curation_cache()
assert lookup_town_anchor("xxxx") is None
# ============================================================================
# REST API: gauge_sites
# ============================================================================
@pytest.fixture
def client():
app = FastAPI()
app.include_router(curation_router, prefix="/api")
return TestClient(app)
def test_api_list_gauges(client):
r = client.get("/api/gauge-sites")
assert r.status_code == 200
body = r.json()
assert len(body) == len(_GAUGE_SITES_SEED)
def test_api_get_one_gauge(client):
r = client.get("/api/gauge-sites/USGS-13186000")
assert r.status_code == 200
assert r.json()["gauge_name"] == "Snake River at Heise"
def test_api_get_404(client):
r = client.get("/api/gauge-sites/USGS-99999")
assert r.status_code == 404
def test_api_post_add_gauge(client):
r = client.post("/api/gauge-sites", json={
"site_id": "USGS-NEW1", "gauge_name": "Test Gauge",
"lat": 44.0, "lon": -115.0, "action_ft": 5.0,
})
assert r.status_code == 200, r.text
assert r.json()["site_id"] == "USGS-NEW1"
# Accessor sees it.
invalidate_curation_cache()
assert lookup_gauge_site("USGS-NEW1") is not None
def test_api_put_updates_gauge(client):
r = client.put("/api/gauge-sites/USGS-13186000",
json={"action_ft": 15.5})
assert r.status_code == 200
assert r.json()["action_ft"] == 15.5
invalidate_curation_cache()
assert lookup_gauge_site("USGS-13186000")["action_ft"] == 15.5
def test_api_delete_gauge(client):
r = client.delete("/api/gauge-sites/USGS-13186000")
assert r.status_code == 200
invalidate_curation_cache()
assert lookup_gauge_site("USGS-13186000") is None
def test_api_post_missing_field_400(client):
r = client.post("/api/gauge-sites", json={"gauge_name": "X"})
assert r.status_code == 400
# ============================================================================
# REST API: town_anchors
# ============================================================================
def test_api_list_towns(client):
r = client.get("/api/town-anchors")
assert r.status_code == 200
assert len(r.json()) == len(_TOWN_ANCHORS_SEED)
def test_api_post_add_town(client):
r = client.post("/api/town-anchors", json={
"name": "Bellevue", "lat": 43.4670, "lon": -114.2557, "state": "ID",
})
assert r.status_code == 200
assert r.json()["name"] == "bellevue"
invalidate_curation_cache()
coord = lookup_town_anchor("bellevue")
assert coord is not None
def test_api_put_town(client):
# Find Boise's anchor_id first.
r = client.get("/api/town-anchors")
boise = next(t for t in r.json() if t["name"] == "boise")
r2 = client.put(f"/api/town-anchors/{boise['anchor_id']}",
json={"enabled": False})
assert r2.status_code == 200
assert r2.json()["enabled"] is False
invalidate_curation_cache()
assert lookup_town_anchor("boise") is None
def test_api_delete_town(client):
r = client.get("/api/town-anchors")
nampa = next(t for t in r.json() if t["name"] == "nampa")
r2 = client.delete(f"/api/town-anchors/{nampa['anchor_id']}")
assert r2.status_code == 200
invalidate_curation_cache()
assert lookup_town_anchor("nampa") is None

4
tests/test_nwis_handler.py
View file

@ -1,7 +1,9 @@
"""Tests for v0.5.12 usgs_nwis handler."""
import pytest
from meshai.central.idaho_gauge_sites import IDAHO_CURATED_SITES
# v0.6-4: IDAHO_CURATED_SITES dict moved to gauge_sites SQLite table;
# import the seed data from the curation module as a back-compat alias.
from meshai.persistence.curation import _GAUGE_SITES_SEED as IDAHO_CURATED_SITES
from meshai.central.nwis_handler import handle_nwis
from meshai.persistence import close_thread_connection, init_db
from meshai.persistence import db as persistence_db