Migrations 025-029 (wzdx + traffic-family adapters/streams) were applied
out-of-band via direct psql during their deploys and never recorded in
schema_migrations. Result: a fresh restore would replay them, and an audit of
the tracking table understated what was actually live. All five are pure
additive INSERT ... ON CONFLICT DO NOTHING (verified idempotent by inspection),
so they were back-filled directly into schema_migrations (5 rows, applied_at =
now() = the reconciliation event; the original out-of-band apply dates are
unrecoverable and are documented as such rather than guessed).
Adds `central-migrate --check` to catch this class of drift going forward:
- find_drift(): pure function comparing schema_migrations rows vs *.sql files,
returning (untracked, orphan). Unit-tested, no DB dependency.
- check_drift(): CI-assertion form -- exit 1 if any file is untracked, else 0.
- log_drift(): WARNs per drifted entry, called on every migrate run too.
No migration 031 for the v0.9.17 wzdx state default: that default lives in
adapter code (_read_states falls back to _DEFAULT_STATES), so the live row's
explicit 7-state array is behaviorally identical to a fresh-install null.
Materializing it into SQL would freeze a snapshot of _DEFAULT_STATES and create
a new drift vector. Deferred to a future "show effective defaults" UI PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Narrow which WZDx registry feeds are fetched at poll time, so out-of-state
feeds are never requested (no bandwidth/quota cost), rather than fetching
nationally and dropping out-of-bounds events at the archive INSERT (v0.9.12).
- states is now list[StateCode] (Literal of 56 canonical 2-letter codes),
default ["ID","WA","OR","NV","UT","WY","MT"]. Semantic flip: empty/unset/null
now means the Idaho-region default, NOT "every eligible feed". Poll nationally
by selecting all states in the GUI.
- list[Literal] renders as the existing checkboxes multi-select widget (firms.py
precedent); pydantic rejects malformed codes at save time.
- Add informational quota_estimate (cap=None: never warns/blocks) so the edit
page shows the fetch-volume reduction.
- adapters_edit.html: standalone quota panel for flat-config adapters, guarded
by not has_model_list so model_list adapters don't double-render; reuses the
identical quota.* keys + CSS so the v0.9.15 client recompute works for them too.
- 7 new tests (default/null fallback, _discover filtering, malformed rejection,
quota math, offline Jinja panel render).
Event data shape is unchanged (poll-behavior-only), so no published_ids reset
is needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mechanical lint fixes only — ruff 31 -> 0 across the two files.
- E402 module-import-not-at-top: 15 -> 0 (relocated module-level
`logger = logging.getLogger(...)` from mid-import to below the import
block in routes.py; imports now contiguous at top of file)
- F811 redefined-while-unused: 10 -> 0 (removed 10 redundant function-local
`from central.gui.csrf import reuse_or_generate_pre_auth_csrf` re-imports;
the module-level import at line 26 is now load-bearing)
- F401 unused-import: 4 -> 0 (routes.py: fastapi.Depends, and
central.gui.csrf.reuse_or_generate_pre_auth_csrf resolved by the F811 fix;
test_wfigs.py: sqlite3, central.config_models.RegionConfig)
- E702 multiple-statements-on-one-line-semicolon: 2 -> 0 (split the two
semicolon-joined statements in _fused_bbox, indentation preserved)
Deliberate function-local wizard imports (circular-import workaround) left
untouched. pytest: 890 passed / 1 skipped, unchanged across 3 runs.
no behavior change; ruff mechanical fixes only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirror TomTomIncidentsAdapter.quota_estimate in client-side JS so the
quota panel updates live as the operator edits a per-row cadence or the
adapter cadence, and on Add/Delete bbox -- instead of only on Save.
- tomtom_incidents.py: expose seconds_per_month + default_cadence_s in
the quota dict (no JS magic numbers).
- model_list.html: data-quota-* attrs + .quota-detail/.quota-msg spans;
self-contained DOMContentLoaded-guarded IIFE applying the same
max(adapter_cadence, bbox_cadence||default) floor and 80%/100%
warn/block thresholds. No-op when no quota panel present.
- tests: structural asserts for data-attrs + span hooks (JS exec needs
a browser; live behaviour eyeballed manually).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pairs each live WFIGS perimeter with its nearby/contemporaneous FIRMS hotspots
into a single "fire" on the /events map. FIRMS hotspots carry no IrwinID, so the
link is spatial+temporal: a hotspot is confirmed (part of a known fire) when it
lies within 1km of a perimeter AND within 72h of it; hotspots matching no
perimeter render amber as "unconfirmed" -- a possible new fire detected by
satellite before an official perimeter exists (early-warning signal).
- routes.py: read-only /events/fire-fused.json (PostGIS ST_DWithin geography join)
- events_list.html: "Fuse fire layers" toggle (default on); centroid fire glyph
that expands to polygon + hotspot dots on click; amber unconfirmed hotspots
- central.css: --fire-confirmed / --fire-unconfirmed vars (retune without code)
- 11 tests (shaping, bbox parse, R/T + bbox param wiring); spatial correctness
verified on prod (Summit Creek: perimeter + 90 hotspots; 191 unconfirmed)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hourly supervisor sweep deletes archived events older than their stream's
config.streams.max_age_s (the /streams 1/7/14/30/365d buttons -- the per-stream
source-of-truth). Explicit STREAM_CATEGORY_DOMAINS map (category domains do not
equal stream subject domains for the traffic family); fail-safe skip on
missing/<=0 max_age_s; unmapped-domain warning so a new adapter cannot silently
dodge retention.
TimescaleDB hypertable gotcha: events.ctid is chunk-local, NOT globally unique --
DELETE batching keys on the composite PK (id, time). See the inline NOTE in
config_store.delete_events_older_than and the PR body for the incident write-up.
- config_store: delete_events_older_than (batched on (id,time)) + unmapped_event_domains
- supervisor: STREAM_CATEGORY_DOMAINS, _sweep_events_retention, _events_retention_loop
- streams_list.html: max-age also governs archived-events retention
- 9 tests
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a single archive-level geographic filter at the events INSERT path: events
whose geometry falls entirely outside a system-configured monitoring area are
dropped before archival. Null-geom events (SWPC trio, .removed tombstones) are
always kept. Uses a pure shapely intersects() predicate so border-straddlers
are kept (matches ST_Intersects), and is fail-open on unparseable geometry.
- config.system gains monitor_{north,south,east,west} (migration 030, Idaho default)
- archive refreshes the bbox every 60s (no restart needed to change it); adds a
per-adapter dropped-count counter, debug log per drop, cumulative INFO rollup
- new GUI editor at /monitoring-area (Leaflet draggable rectangle, N/S/E/W inputs)
- no adapter code changes; well-behaved adapters keep upstream API filtering
- 12 tests covering all five verdicts, drop-and-count, border/point-on-edge, fail-open
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The /events feed was dominated by *.removed tombstone events (audit records
for features dropped from upstream feeds), burying geometry-bearing events
like fire perimeters (wfigs_perimeters: 54 real perimeters vs 1015 tombstones).
The GUI now default-hides any event whose category ends in .removed, with a
"Show removed" checkbox to restore them; URL state is preserved (HX-Push-Url)
so a shared link shows what the sharer saw. events.json is unchanged (still
returns tombstones) so API consumers are unaffected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a passive Leaflet map to the generic model_list editor that renders
every bbox row as a labeled translucent rectangle, auto-detected via the
min/max lon+lat sub-fields (TomTom incidents). Read-only: no drag/draw,
precise tuning stays in the per-row coord inputs. Rectangles redraw live
on input/add/remove; viewport fits only on initial render so typing never
jumps the map. Non-bbox model_lists (StateConfig, TileCoord) are unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixes a shared form_descriptors 500 (NotImplementedError: unsupported list
type) that broke the Edit page for ALL FOUR adapters whose settings carry a
list[<BaseModel>] field: tomtom_incidents, tomtom_flow, state_511_atis,
state_511_atis_cameras.
- form_descriptors: list[BaseModel] -> generic "model_list" widget with
recursive per-column sub_field descriptors.
- New _partials/model_list.html: vanilla-JS repeatable-row editor
(add/remove/renumber), driven entirely by sub_fields (no adapter-name
branching). Single-region edit pages render byte-identically.
- TomTom: BBox/Settings Pydantic validators (10,000 km^2 cap, coord ranges,
min<max, cadence_s>=60, unique names) as the single source of truth
(enforced at supervisor load AND GUI POST). Duck-typed quota_estimate hook
+ read-only quota panel; POST hard-blocks estimates over the 2,500/mo free
tier (422). TOMTOM_FREE_TIER_CALLS_PER_MONTH is a tunable for paid tiers.
- routes: model_list form parse, row-aware ValidationError messages, 422 for
model_list failures (single-region region errors still re-render at 200).
- tests: 11 new (real-Jinja render across 3 adapters + byte-identical nws
no-regression guard, POST persist + oversized/degenerate/duplicate/cadence/
quota 422 matrix, quota estimate). Full suite 848 passed, 1 skipped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plumb the upstream GeoJSON Polygon/MultiPolygon into Geo.geometry so the
archive renders the real fire-perimeter shape (ST_GeomFromGeoJSON) instead
of the bbox 4-corner fallback. Adapter-local change; the v0.9.3 Geo.geometry
framework already carries it end-to-end. Graceful null + dict-or-string
coercion via _as_geometry; fall-off removal events stay NULL.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Castle Rock caps each DataTables page at 100 rows regardless of `length`, so the
single-request fetcher silently dropped rows on any layer over 100 (confirmed
live: Construction recordsFiltered=114, returned 100 -> 14 rows invisible).
Backports the v0.9.6 cameras pagination loop into _fetch_details.
- _LIST_PAGE_LENGTH 1000 -> 100; new _list_body(start) builder; new @retry
_fetch_page(base_url, layer, start).
- _fetch_details loops start+=100 until recordsFiltered collected or an empty
page, with a _MAX_PAGES=50 ceiling that warns+breaks. Mid-pagination failure
returns rows-so-far (retried next poll).
- Incidents (1) / Closures (29) are under 100 today but pagination applies
uniformly; future-proof.
central-supervisor restart only (no stream, migration, template, or dep change).
Full suite: 833 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New CENTRAL_TRAFFIC_CAMERAS stream + state_511_atis_cameras adapter. Telemetry
half of Castle Rock (events shipped in v0.9.2). Each Idaho camera -> one
telemetry event on /telemetry; detail drawer renders <img> direct from the
source (no blob storage / proxy in Central -- URL only).
supervisor + gui + ARCHIVE restart (NEW event-bearing stream
central.traffic_cameras.>). Ships disabled; public-unauth (no api key). Idaho only.
- Full camera list via POST /List/GetData/Cameras (DataTables), PAGINATED at
100/page (Idaho ~455 = 5 pages). GetUserCameras was a red herring (4 default
cams). The 100-row page cap also means v0.9.2 state_511_atis silently
truncates its 114-row Construction layer -> separate v0.9.7 fix.
- Subject central.traffic_cameras.{state}.{camera_id}; category
camera.state_511_atis_cameras -> GUI event_type "camera". data_class=telemetry.
- Per-UTC-day dedup {state}:cam:{id}:{YYYY-MM-DD}: one event per camera per day
-- always shows today's cameras, no per-poll flooding, no retention
coordination. Inherits the v0.9.1 dedup mixin.
- All sources included (Idaho511/ITDNET/RWIS/UDOT/ODOT/WYDOT/MTD border cameras);
source surfaced in data + the drawer for provenance. WKT POINT (lon lat) -> geo.
- No upstream image-capture timestamp (lastUpdated is config-edit time); drawer
shows no false "Captured" line. Cadence 600s. Severity 1 (telemetry).
Full suite: 829 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lets each incident bbox poll at its own interval so busy metros refresh more
often than quiet corridors. Backward-compatible, code-only patch.
- Optional BBox.cadence_s (int | None = None) -> per-bbox poll interval; None
falls back to the adapter's default_cadence_s. Existing settings without the
field keep their current behavior.
- In-memory _last_polled {bbox_name: datetime}, per process. _bbox_due() gates
fetches; poll() fetches only due bboxes. First poll after (re)start fetches all
(one-shot catch-up; storage dedup on <state>:tomtom:<id> collapses overlap).
- _last_polled is recorded ONLY after a successful fetch -- a failed bbox stays
due and retries next cycle (regression-guarded).
- Supervisor wakes the adapter at the adapter-level cadence; set that to the GCD
of the per-bbox cadences for exact intervals (extra wakeups cost zero API calls).
central-supervisor restart only. No gui/archive restart, no migration, no new dep.
Full suite: 815 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fourth CENTRAL_TRAFFIC event adapter. Complements wzdx (federal work zones) and
state_511_atis (state-DOT reports) with TomTom commercial vehicle-telematics
coverage. Polls the Orbis incidentDetails endpoint per metro bbox, emits one
event per incident to central.traffic.incident.<state>. Ships disabled.
central-supervisor + central-gui restart only -- adapter row on the EXISTING
CENTRAL_TRAFFIC stream, so NO archive restart and no new stream/dependency.
Reuses the existing "tomtom" api key.
- Bbox limit refutation: incidentDetails rejects bbox > 10,000 km^2, so coverage
is per-metro bboxes (Treasure Valley / Boise, 8,601 km^2), NOT statewide. One
bbox @ 1800s = 1,440 calls/mo = 58% of the 2,500/mo free-tier cap. Expansion
rows must respect N*(43200/cadence_min) <= 2500.
- category="incident.tomtom_incidents" -> GUI event_type "incident" (shared with
state_511_atis; cross-source overlap is by design = additive coverage, distinct
dedup ids + categories, no Central-side cross-source dedup).
- Severity from magnitudeOfDelay (0->1,1->1,2->2,3->3,4->4; 4=closure). Never None.
- geo.geometry carries TomTom's Point/LineString directly (already lon/lat GeoJSON;
the v0.9.3 framework renders the affected road as a polyline). No decode needed.
- Dedup id <state_code>:tomtom:<tomtom_id> (upstream id stable across polls,
verified 154/154 over 60s). Inherits the v0.9.1 dedup mixin.
- aiohttp params= URL-encodes the fields{} GraphQL braces (no curl-glob issue);
key redacted from logs; poll skips cleanly without a key.
Full suite: 809 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phased PR2 of v0.9.3. Adds the on-demand tile passthrough so Navi can call
Central instead of navi-traffic, and so tiles outside the polled coverage set
still get persisted. central-gui restart only (new route); no supervisor, no
archive, no migration, no new stream/dep.
GET /api/traffic/flow/<z>/<x>/<y>.{png,pbf} -- exactly matches navi-traffic's
route for a drop-in frontend flip (s/navi-traffic/central:8000/). No auth
(tailnet-trusted, matches navi-traffic's existing posture; exempted in
middleware). Cache-Control max-age=60 (TomTom's advertised tile TTL).
- .png -> passthrough + cache only (raster unparseable for storage).
- .pbf -> passthrough + decode via the shared tomtom_flow_parse.decode_flow_tile
and publish segments to CENTRAL_TRAFFIC_FLOW via the GUI's NATS connection
(wrap_event, supervisor's exact envelope shape). Same minute-bucketed dedup id
as the polling adapter, so adapter+passthrough fetches of one tile in the same
minute collapse at the archive's (id, time) upsert -- intentional collision.
- Publish is best-effort: wrapped in try/except + guards get_js() None, so NATS
state never blocks serving the tile (HTTP > storage).
- API key via ConfigStore(get_pool()).get_api_key("tomtom"); 503 if unset;
redacted from any logged upstream error; 502 on upstream failure (no publish).
Single-flight (shared in-flight upstream request per tile) queued for v0.9.5.
navi-frontend flip + navi-traffic deprecation live in the navi repo (Matt's call).
Full suite: 787 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Third CENTRAL_TRAFFIC-family member, first telemetry traffic source. Polls a
configured tile coverage set (Idaho metros, z=10), fetches Orbis vector flow
tiles, decodes per-segment relative_speed + road geometry, emits one telemetry
Event per road segment per poll to the new CENTRAL_TRAFFIC_FLOW stream. Renders
as colored polylines (green free-flow -> red jam) on the /telemetry map.
Production code; supervisor + gui + ARCHIVE restart (NEW event-bearing stream
central.traffic_flow.> -> archive must resubscribe). Ships disabled; needs a
"tomtom" api key in config.api_keys before enable.
- Subject central.traffic_flow.{z}.{x}.{y} (token traffic_flow, non-overlapping
with central.traffic.>). category="flow.tomtom_flow" -> GUI event_type "flow".
- Severity from relative_speed: >=0.75=1, 0.5-0.75=2, 0.25-0.5=3, <0.25=4.
- Cadence 300s; 7-day retention (high-volume telemetry). Dedup minute-bucketed,
inherited from the v0.9.1 SourceAdapter mixin.
- Shared tomtom_flow_parse module (decode + slippy-tile georeference) reused by
the v0.9.4 on-demand passthrough endpoint.
- Generic framework change (Option A, ~3 lines, inert for the other 14
adapters): Geo.geometry optional field + archive _build_geom_sql prefers it,
so segments persist their real LineString to the PostGIS geom column.
- Idaho-only (Orbis tier confirmed live). Cameras + Navi passthrough are follow-ups.
- deps: mapbox-vector-tile (vector PBF decode); itsdangerous promoted to an
explicit dependency (gui/csrf.py + gui/wizard.py imported it as an undeclared
transitive that uv re-lock would otherwise prune).
Full suite: 780 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second CENTRAL_TRAFFIC adapter. Production code; central-supervisor + central-gui
restart (new adapter class + ADAPTER_GROUPS). No new stream -> no archive restart;
migration 026 adds the adapter row only. Ships disabled.
Two-endpoint join per layer: GET /map/mapIcons/<Layer> (markers: itemId + coords)
joined on id with POST /List/GetData/<Layer> (DataTables detail: roadwayName,
description, county, severity). The marker feed has coords but no text; the List
feed has text but no coords.
Layers -> event_types (wzdx category/subject precedent): Incidents->incident,
Closures->closure, Construction (type "Roadwork")->work_zone. category is
"<event_type>.state_511_atis"; subject central.traffic.<event_type>.<state>.
Severity 3 if isFullClosure else 1. Cadence 300s. Dedup inherited from the
v0.9.1 SourceAdapter mixin. enrichment_locations canonical (latitude,longitude)
from the marker join; county/state come upstream.
Templatized per state via settings {"states":[{code,base_url}]} but ships
Idaho-only: cross-state spot-checks refuted the shared-URL hypothesis (Oregon
TripCheck is HTML, Wyoming wyoroad 404 -- neither is Castle Rock). Add states as
settings rows once each host is verified.
Also fixes a latent test bug: test_consumer_doc per-adapter heading regex was
[a-z_]+ (no digits); state_511_atis is the first adapter name with digits, so
widened to [a-z0-9_]+.
Full suite: 759 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two v0.9.0 fast-follows. Production code; central-supervisor + central-gui
restart (adapter base change + template change). No migration, no new stream.
(a) Work-zone subject + detail no longer leak vehicle direction "unknown"
(common in AZ mcdot etc.) -- gated on direction not in (None, "unknown") in both
wzdx partials. Was "Work zone on MORELAND ST unknown".
(b) is_published/mark_published/sweep_old_ids extracted from per-adapter inline
copies onto the SourceAdapter base (beside bump_last_seen); a dedup_sweep_days
class attr parameterizes the retention window (NWIS=30, default=14). Inline
copies deleted from inciweb/nwis/wzdx; the other 10 adapters keep theirs as a
future cleanup. Net dedup code down ~52 lines.
Full suite: 744 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Opens Phase 4 transportation aggregation (Design B, Central-direct). New
registry-driven wzdx adapter polls the FHWA WZDx Feed Registry, fetches each
eligible v4.x GeoJSON feed concurrently, and emits work_zone events into the new
CENTRAL_TRAFFIC stream. Production code; central-supervisor AND central-gui
restart (new adapter class + stream + ADAPTER_GROUPS). Ships disabled.
First adapter to use the category/subject split: category="work_zone.wzdx" (GUI
event_type "work_zone" via split_part) while the NATS subject is
central.traffic.work_zone.{state}. Subject state from the registry row, geocoder
state as fallback. Severity from vehicle_impact (all-lanes-closed=3,
some-lanes-closed=2, all-lanes-open=1, unknown/missing=1). Feed filter
geojson + active + needapikey=false + version 4.x (21 of 39 feeds). 600s cadence.
Dedup composite <data_source_id>:<feature_id> in the shared cursors.db; stateless
discovery (no conftest isolation entry). enrichment_locations uses the canonical
("latitude","longitude") paths.
Full suite: 739 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Opens the v0.8.x data-quality cleanup arc. Production code; central-gui AND
central-supervisor restart (adapter contract + enrichment behavior change).
NWIS events rendered as a bare "Water reading: 111 ft3/s" with an empty Location
column -- an operator couldn't tell where the gauge is or whether 111 ft3/s is
drought-low, normal, or near-flood. Coordinates were present but the reverse
geocoder returns null city/state/county for rural gauge points, and USGS site +
percentile data was never fetched. v0.8.0 fetches both.
Approach B (adapter-owned, per the proposal decision): the NWIS adapter -- which
already owns the USGS APIs -- fetches site metadata and daily stats itself and
writes two provenance bundles under event.data["_enriched"]:
- usgs_site {name, lat, lon, state, county} from the OGC monitoring-locations
item-by-id (the API family the adapter already speaks; JSON, no RDB parser).
- usgs_stats {value, percentile, class_label, severity_band, p10..p90, record_max,
count, period} from the legacy RDB daily-statistics service (the OGC API has no
stats endpoint). USGS percentiles are % of days at-or-below, so higher = higher
flow; classified to the WaterWatch bands -> severity 0-4 (record=4, much
above/below=3, above/below=2, normal=1; None reserved for "no stats", distinct
from a normal-flow gauge). Severity is set on the event, so it drives the v0.7.1
severity chip-picker filter + v0.7.2 map-marker opacity.
- new nwis_enrich.py: pure parse/classify/percentile/band helpers + a sqlite
SiteStatsCache (site TTL 365d, stats TTL 90d -- one fetch per site+param serves
every reading for the window, so a warm cache makes zero USGS calls). USGS down
-> cached-if-present else all-null bundle; the event still publishes.
Framework: the single agreed generic change -- supervisor apply_enrichment now
MERGES into _enriched instead of overwriting, so the still-global geocoder phase
doesn't clobber the adapter's bundles. No other adapter writes _enriched, so this
is inert for them.
GUI: _event_summaries/nwis.html -> "<site> -- <value> <units> (<band>, <Nth>
percentile)", with graceful fallback to "<site> -- <value>" then the bare
"Water reading:". _event_rows/nwis.html detail gains site/normalcy/typical/location
rows. _events_rows.html Location column falls back generically to any
_enriched.<source> carrying state/county when the geocoder is null (works for
future enrichers). events.json contract unchanged (additions under _enriched only).
conftest isolate_enrichment_cache also redirects NWIS_CACHE_DB_PATH off the prod
path (unprivileged-user test isolation). Adds tests/test_nwis_enrichment.py (28
tests: parse, band edges incl P0/P9/P10/P75/P90/record, percentile interpolation,
cache hit/miss/expire, adapter enrich + graceful-null + cache-hit-no-refetch,
summary rendering per band).
Full suite: 710 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #6 of the v0.7.x GUI rework arc, and its close-out. Production code;
central-gui restart only (templates + a static stylesheet; supervisor untouched).
The v0.7.0-v0.7.4 arc shipped the FEATURES the gui_preview.html mock shows
(chip-pickers, active pills, markercluster + shape/opacity markers, collapsed
grouped legend, real paginator, telemetry tab) but on top of the old CSS
framework. v0.7.5 closes the visual gap: the framework is removed entirely and
replaced by a hand-authored design system that IS the preview (neutral greys, a
single blue accent, 6px radii, 14px type, bordered cards/tables).
- New static/css/central.css: design tokens + element base (forms, buttons,
tables, cards, nav) + every events-feed component, authored so the existing
template class names + element IDs the page JS/HTMX depends on (.chip-picker*,
.events-table, .event-row, .legend-chip, .evt-marker, #fit-to-results,
#map-filter-toggle, ...) render in the preview's language unchanged. Mounted at
/static (already wired in __init__.py); linked from base.html + base_wizard.html.
- base.html / base_wizard.html: framework <link> removed; nav restructured to the
preview shell (.nav / .brand / .nav-links / .logout-btn); flash messages moved
off framework color vars to .flash-* classes.
- events_list.html: 190-line inline <style> deleted (moved to central.css); map
wrapped in .map-container with a floating .map-toolbar (Fit / map-filter toggle),
legend below. HTMX wiring (hx-get/hx-target/hx-push-url), the v0.7.1 chip-picker
JS, the v0.7.2 marker shapes/opacity, and the v0.7.4 base_path split are
untouched.
- Every other page (dashboard, adapters, api-keys, enrichment, streams, login,
change-password, and the full setup wizard) de-framework'd: framework class
names (outline/secondary/contrast/grid/container) renamed to neutral
.btn-*/.cols; no framework default rendering; same tokens/typography/forms/tables.
- /events.json + the v0.7.1 registry-index adapter palette + v0.7.2 severity
opacity + v0.7.4 stable cross-tab chip colors are all unchanged -- this is a
visual shell swap, not a behavior or data change.
grep -rn pico over templates + static returns zero. All GUI routes return 200
(authed) / 302 (unauthed); no 5xx. Full suite: 682 passed, 1 skipped (central and
unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #5 of the v0.7.x GUI rework arc. Production code; central-gui restart only
(supervisor untouched -- data_class is read only by central-gui per request).
- SourceAdapter gains a `data_class` class attr (Literal["event","telemetry"],
default "event"). NWIS opts in as "telemetry" (continuous high-volume water
gauges); every other adapter stays "event". The /events vs /telemetry split is
thus registry-derived from class attrs -- no hardcoded adapter-name lists.
- routes.py refactor: `_class_adapter_names(data_class)` and a `data_class` arg
on `_adapter_filter_options` scope the flat + domain-grouped chip/legend lists
to a class (colors stay keyed to the FULL registry, so an adapter keeps one
color across tabs). `_fetch_events` accepts `class_adapters` and adds an
`adapter = ANY(...)` condition. Shared `_events_query`, `_events_page(data_class,
base_path)` and `_events_rows_fragment(...)` back both tabs; `/events`,
`/events/rows`, `/telemetry`, `/telemetry/rows` are thin wrappers.
- Templates parameterized with a `base_path` context var (form action, hx-get,
hx-push-url header, clear-all redirect, JS BASE_PATH const); the `_events_rows`
paginator macro takes `base`. Same templates serve both tabs; nav gains a
Telemetry link.
- /events.json UNCHANGED -- the cursor path sets no `class_adapters`, so the
subject + pagination contract is intact (TestEventsJsonSubject still passes).
Adds TestTelemetrySeparation (data_class defaults, registry split 11 event / 1
telemetry, class-scoped filter options, color stability, and the `adapter =
ANY(...)` SQL shape incl. the no-class events.json path). Updates the events
frontend tests for the base_path-parameterized templates.
Full suite: 682 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #4 of the v0.7.x GUI rework arc. Production code; central-gui restart only.
- Adapter legend: collapsed by default ("{n} adapters · Show legend ▾"). Expands
to domain-grouped chips (same grouping as the v0.7.1 chip-picker) with uniform
ellipsis-truncated names + full-name title tooltips. Clicking a legend chip
toggles that adapter's filter (reuses the chip-picker's hidden CSV via
syncField), so the legend doubles as a filter affordance.
- Row stability: time cell is single-line MM-DD HH:MM UTC (year dropped from the
cell; full ISO in the cell tooltip + a new Time row in the expanded detail).
Adapter cell is a chip (color swatch + short name; display_name is the
tooltip). table-layout:fixed + per-column widths + fixed 37px row height with
nowrap/ellipsis cells -> no per-row wrap variation.
- Real paginator: _fetch_events offset-mode returns the exact page slice plus the
grand total via count(*) OVER() in one roundtrip. Previous/Next + windowed page
numbers (1 ... 4 5 [6] 7 8 ... 47) + "showing X-Y of N" + a 25/50/100/250
per-page selector. URL state persists offset + limit. events.json keeps cursor
pagination (back-compat): offset param presence selects offset-mode, its
absence keeps the cursor path -- cleanly separable by endpoint.
Adds TestEventsPagination (12 tests: offset/limit parse incl. max 250,
offset-vs-cursor query shape, _build_pagination windowing). Updates the time
format + adapter-cell + pagination-mode assertions in the existing frontend
tests to the new contract.
Full suite: 674 passed, 1 skipped (central and unprivileged zvx). count(*) OVER()
is ~7.5ms at current volume; vanilla JS + HTMX; CSS functional-only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #3 of the v0.7.x GUI rework arc. Makes the /events Leaflet map readable and
intentional. Production code; central-gui restart only (no adapter change).
- Fit-to-results default: the map now fits the actual event distribution on
load (previously disabled -> fixed global zoom-4). Empty result set falls
back to the CONUS setView (no crash). Re-fits after each HTMX swap, but only
when the map-filter toggle is OFF (when ON the viewport drives the bbox, so
re-fitting would fight/loop the filter).
- leaflet.markercluster (1.5.3, via CDN): point markers cluster into numbered
badges (disableClusteringAtZoom=9, showCoverageOnHover=false,
spiderfyOnMaxZoom=true). markercluster supports point markers only, so
polygons/lines render in a separate un-clustered featureGroup; fit unions both.
- Map-filter toggle ("Filter table by map view"), default OFF. When off the
table shows all filter-matching events regardless of map zoom; the backend
ignores region_* unless map_filter is set (guards bookmarked URLs too). URL
carries map_filter=1 only when on (hidden input disabled otherwise).
- Per-event_type marker shape (derived event_type = first category segment):
circle = quake/hydro/space (points), square = fire (areas),
triangle = wx (NWS alerts/warnings), star = disaster (GDACS/EONET).
Rendered as divIcon + CSS clip-path; point markers switched from circleMarker
to L.marker(divIcon) (also required for markercluster compatibility).
- Per-severity opacity: critical(4)=1.0, high(3)=0.85, moderate(2)=0.7,
low(1)=0.5, unknown(0/NULL)=0.4. Needed adding severity to the _fetch_events
SELECT + event dict (row.get for mock-tolerance) + a data-severity row attr.
Adds 4 tests (map_filter gating on/off, bbox reaches query only when on,
severity in SELECT); updates test_events_bbox_guard for the new toggle contract.
Full suite: 662 passed, 1 skipped (central and unprivileged zvx). Vanilla JS +
HTMX + Leaflet/markercluster; CSS functional-only (polish deferred).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
