Commit graph

61 commits

Author SHA1 Message Date
Matt Johnson
966661305f feat(gui): generic adapter edit form
Implement Central 2-A2: generic adapter edit form feature.

- Add form_descriptors.py with describe_fields() and FieldDescriptor
  - Maps Pydantic types to HTML widgets (text, number, checkbox, csv, region)
  - Handles Optional types by recursively resolving inner type
  - Uses PydanticUndefined handling for proper default values

- Update routes.py GET/POST handlers:
  - Use cached _adapter_classes() for adapter class lookup
  - Generate field descriptors from adapter settings_schema
  - Parse form values based on widget type in POST handler
  - Validate settings via Pydantic ValidationError

- Update adapters_edit.html template:
  - Render form dynamically from field descriptors
  - Support all widget types (text, number, checkbox, csv, region)
  - Use adapter.display_name and adapter.description from class

- Delete per-adapter templates:
  - adapters_edit_nws.html
  - adapters_edit_firms.html
  - adapters_edit_usgs_quake.html

- Add tests/test_form_descriptors.py with comprehensive coverage
- Update tests/test_adapters.py to include last_error in mock rows
- Update tests/test_region_picker.py to include last_error in mock rows

Adding a new adapter no longer requires GUI template work.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-18 23:16:37 +00:00
Matt Johnson
4ee3d8bd14 fix(adapters): complete self-describing adapter attributes
- Replace settings_schema classmethod with Pydantic model class attribute
- Add display_name, description, requires_api_key, wizard_order, default_cadence_s
- Remove stream_name from adapters (JetStream routes by subject filter)
- Define NWSSettings, FIRMSSettings, USGSQuakeSettings Pydantic models
- Make discover_adapters() public with error handling
- Move adapter registry to Supervisor instance (self._adapters)
- Add subject_for tests for all 6 quake magnitude tiers
- Fix test_supervisor_integration to use injected mock adapters

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-18 22:33:19 +00:00
Matt Johnson
4573bf6ee2 refactor(adapters): self-describing adapter pattern with auto-discovery
- Add stream_name, subject_for(), and settings_schema() to SourceAdapter ABC
- Implement all three methods in NWSAdapter, FIRMSAdapter, USGSQuakeAdapter
- Replace manual _ADAPTER_REGISTRY with pkgutil.iter_modules auto-discovery
- Remove subject_for_event from models.py (each adapter owns its subject logic)
- Update supervisor to use adapter.subject_for(event) instead of helper
- Fix quake events going to wrong stream (was publishing to CENTRAL_WX)
- Update test files to use adapter methods

This fixes the quake stream bug where events were published to
central.wx.alert.us.unknown instead of central.quake.event.<tier>.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-18 22:14:12 +00:00
3de81f392a
1b-9c: Events feed UX iteration — colors, popups, viewport filter, expandable rows (#28)
* feat: events feed UX iteration - colors, popups, viewport filter

A. Color-code polygons by adapter (NWS amber, FIRMS red, USGS violet)
B. Click popup on polygons showing time + adapter + category + subject
C. Map viewport drives spatial filter - pan/zoom updates table via HTMX
D. Add legend showing adapter color mapping
E. Remove draw-bbox control, region inputs now hidden (auto-managed)

Template changes:
- _events_rows.html: add data-adapter, data-category, data-time, data-subject
- events_list.html: ADAPTER_COLORS mapping, bindPopup, moveend handler

Test: verify template renders adapter/category/subject for JS consumption

* fix: remove isoformat() call on already-formatted time string

* feat: full events feed UX iteration

A. Color-code polygons by adapter with legend
B. Click popup on polygons with "View details" link
C. Viewport-driven spatial filter - pan/zoom updates table via HTMX
   Map never auto-fits after initial load (user controls viewport)
D. Expandable row details showing full event data payload

Changes:
- _events_rows.html: add data-event-id, expand button, detail row
- events_list.html: eventLayerGroup pattern, buildPopup, rebindEventLayers
  Fit to results button, expand/collapse handlers, CSS.escape for IDs

* fix: add programmaticMove flag to prevent viewport refresh loop

Suppress moveend handler during fitBounds/setView calls to prevent
feedback loop: fitBounds -> moveend -> applyViewportFilter -> HTMX
swap -> repeat.

* fix: map never auto-fits - user controls viewport

- Disable initial fitToAllLayers on page load
- Remove fitBounds/setView from row click handler
- Map only moves when user pans/zooms
- Table filters based on visible viewport

* fix: map shows all events always, only table filters

Map polygons are drawn once on load and never cleared/redrawn.
HTMX swap only updates the table, not the map layers.
User viewport is fully preserved.

* fix: use htmx.trigger instead of dispatchEvent for HTMX swap

dispatchEvent(submit) was triggering native form submission (full page
reload). htmx.trigger() properly triggers HTMX swap.

Also re-enable initial rebindEventLayers so polygons load on first render.

---------

Co-authored-by: Matt Johnson <mj@k7zvx.com>
2026-05-18 14:19:27 -06:00
55e68d038f
feat(gui): add events feed frontend with map and filters (1b-9b) (#26)
* feat(gui): add events feed frontend with map and filters

GET /events: Full page with filter form, table, and Leaflet map
GET /events/rows: HTMX fragment for table updates

Features:
- Filterable by adapter, category, time range, region bbox
- Cursor-based pagination with Next button
- Leaflet map showing event geometries
- Click/hover row highlights geometry on map
- Draw rectangle on map to filter by region
- Validation errors shown as banner, not 400
- Events link added to nav between Adapters and Streams

Refactored events query into shared helper for JSON and HTML routes.

Tests: 14 new tests covering filters, fragments, geometry handling.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(events): use shared helpers for /events.json, fix tests

- Refactor /events.json to use _parse_events_params and _fetch_events
  helpers, removing ~200 lines of duplicate query logic
- Delete smoke test (test_events_unauthenticated_redirects) that had
  no assertions
- Add TestCrossEndpointParity: verify /events.json and /events return
  identical results with same params, test category filter and cursor
  pagination on both endpoints
- Add TestErrorSemantics: verify /events.json returns 400 on bad params
  while /events returns 200 with error banner (intentional API vs HTML
  divergence)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add real assertions to since/until and region filter tests

Replace trivial status_code==200 assertions with checks that verify
the filter values were actually parsed and passed to the template.
These tests now fail if the handler ignores the filter parameters.

* fix: remove double-escaping from data-geometry attribute

tojson already produces HTML-attribute-safe JSON. The extra |e filter
was double-escaping, causing JSON.parse to fail in the browser JS.
Switch to single-quoted attribute to avoid conflicts with JSON double
quotes.

---------

Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-18 11:23:38 -06:00
78b6fcf150
1b-8: Wizard redesign (deferred-commit) + map fixes + favicon CSRF race fix (#27)
* feat(wizard): implement deferred-commit pattern for setup wizard

Replace the current "POST each step -> DB write -> redirect" architecture
with "collect values across steps in a signed cookie, commit everything
in one transaction at Finish."

Key changes:
- Add wizard.py: WizardState dataclass and cookie helpers
- csrf.py: Add reuse_or_generate_pre_auth_csrf helper
- routes.py: All wizard handlers now use cookie state, no DB writes until finish
- middleware.py: Cookie-based wizard step routing instead of DB queries
- setup_operator.html: Remove "Operator Already Configured" branch

Benefits:
- Back navigation works: can return to any step and edit values
- Atomic commit: all DB writes happen in single transaction at finish
- No orphaned state: failed wizard leaves no DB artifacts
- Simpler auth: pre-auth CSRF for all 5 steps (no session until finish)

Tests updated for new behavior. 287 tests passing.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(templates): correct SRI hashes for leaflet.draw assets

The integrity hashes for leaflet.draw.css and leaflet.draw.js were
incorrect, causing browsers to silently block these resources. This
broke the Leaflet.draw toolbar and map rendering for FIRMS/USGS
adapter region pickers.

Updated both setup_adapters.html and adapters_edit.html with the
correct sha512 hashes computed from the actual CDN files.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(gui): return 204 for browser-noise paths to prevent CSRF races

Browser requests for /favicon.ico, /apple-touch-icon.png, etc. were
triggering parallel GET requests that could race with form loads,
causing CSRF token rotation issues.

Added BROWSER_NOISE_PATHS constant and early 204 response in both
SetupGateMiddleware and SessionMiddleware to short-circuit these
requests before any cookie/token handling occurs.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-18 08:18:04 -06:00
246cd75051
feat(api): add paginated events feed JSON endpoint (#25)
GET /events.json with cursor-based pagination and filtering:
- Filter by adapter, category, since/until, region bbox
- Cursor pagination via (time DESC, id DESC) ordering
- Returns events with GeoJSON geometry parsed as objects
- Validation returns 400 with clear error messages

Migration 014 adds composite index for efficient pagination.

Tests: 17 new tests covering filters, pagination, validation.

Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 22:31:00 -06:00
494ad1c799
feat(gui): implement first-run setup wizard (1b-8) (#24)
* feat(gui): implement first-run setup wizard (1b-8)

Add a 5-step setup wizard that replaces the single-step /setup:
1. Create Operator - create initial operator account
2. System Settings - configure map tile URL and attribution
3. API Keys - optionally add API keys for adapters
4. Configure Adapters - enable/disable adapters with region picker
5. Finish Setup - review and complete setup

Key changes:
- Update middleware to handle wizard URL structure and step routing
- Add wizard routes for each step with proper auth checks
- Create new templates using base_wizard.html for consistent styling
- Add audit events for system.update and setup.complete
- Update tests for new middleware behavior

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(gui): handle CSRF errors on wizard paths

Update csrf_exception_handler to re-render wizard forms with error
message instead of redirecting to /login when CSRF validation fails.

- /setup/operator: re-render with error
- /setup/system: re-render with current system values + error
- /setup/keys: re-render with current keys list + error
- /setup/adapters: re-render with current adapter config + error
- /setup/finish: re-render with summary data + error
- /setup: redirect to /setup (middleware routes to appropriate step)

Add error display to setup_keys.html and setup_finish.html templates.
Add 7 new CSRF handler tests for wizard paths.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(gui): region picker render + click-to-draw

Bug A: Maps render blank on /setup/adapters for FIRMS and USGS
because Leaflet computed zero dimensions before container layout
settled. Fix: add setTimeout invalidateSize() after map creation.

Bug B: No click-to-draw functionality - only drag corners. Fix:
add L.Control.Draw for rectangle drawing with CREATED event handler
to replace existing rectangle.

Both fixes applied to:
- setup_adapters.html (wizard inline JS)
- _region_picker.html (standalone edit page)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(gui): handle revisiting /setup/operator after operator created

When an operator already exists, /setup/operator now shows a
confirmation page instead of the create form. This prevents:
- Unique constraint violations on duplicate username
- Silent creation of duplicate operators

GET /setup/operator: queries config.operators; if any exist,
renders confirmation state with existing_operator context.

POST /setup/operator: checks operator count before INSERT; if
non-zero, renders confirmation state without inserting.

Template updated with conditional to show "Operator Already
Configured" message when existing_operator is set.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(csrf): replace fastapi-csrf-protect with session-bound CSRF

Fixes CSRF race condition where every GET rotated the CSRF token,
causing POST failures when users had multiple tabs or slow connections.

Changes:
- Remove fastapi-csrf-protect dependency
- Add session-bound CSRF tokens stored in config.sessions table
- Add pre-auth CSRF for unauthenticated routes (/login, /setup/operator)
- Add csrf.py module for pre-auth token generation/validation
- Update routes to use new CSRF token handling
- Add migration 013 to add csrf_token column to sessions

The session-bound approach ensures CSRF tokens remain stable for the
duration of a session, eliminating the race condition.

Note: Route tests (test_wizard.py, test_adapters.py, etc.) need
refactoring to mock get_settings() instead of CsrfProtect dependency.
Core auth/CSRF handler tests pass (74 tests).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(csrf): update test suite for session-bound CSRF tokens

- Add CSRF fixtures to conftest.py for pre-auth and session CSRF
- Update test_wizard.py: use bypass_pre_auth_csrf and patch_route_settings
- Update test_adapters.py: set request.state.csrf_token and form mock data
- Update test_api_keys.py: add CSRF token to form data for POST routes
- Update test_streams.py: change return_value to side_effect for CSRF support
- Update test_region_picker.py: add CSRF token handling
- Update test_config_store.py: set CENTRAL_CSRF_SECRET env var in fixture

All 285 tests now pass with session-bound CSRF validation.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Matt Johnson <mj@k7zvx.com>
2026-05-17 22:06:22 -06:00
96ec88883c
feat(gui): add API keys management routes (#23)
Implement CRUD-lite for config.api_keys with:
- List view showing all keys with usage info (which adapters reference each)
- Add form with alias validation (letters, numbers, underscores only)
- Rotate form to replace encrypted value
- Delete with protection against removing keys still referenced by adapters

Security:
- Plaintext keys never displayed back to user
- Values encrypted via crypto.encrypt() before storage
- Audit logs contain only metadata, never plaintext or encrypted values

Routes:
- GET /api-keys - list all keys
- GET /api-keys/new - add form
- POST /api-keys - create key
- GET /api-keys/{alias} - edit/rotate/delete form
- POST /api-keys/{alias} - rotate key
- POST /api-keys/{alias}/delete - delete key

Tests: 11 new tests covering list, create, rotate, delete, and audit
verification.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 18:46:39 -06:00
e097b504af
feat(gui): add streams view (1b-6) (#21)
* feat(gui): add streams view (1b-6)

Add streams list and edit routes with live JetStream data:
- GET /streams: list all streams with live size/messages
- POST /streams/{name}: update max_age_s with validation

Features:
- Live data from JetStream (bytes, messages, timestamps)
- Graceful degradation when NATS unavailable
- Preset chip buttons (1d, 7d, 14d, 30d, 365d)
- Custom days input with Save button
- Current selection highlighted
- Managed by supervisor badge
- Audit logging with before/after max_age_s

Files:
- src/central/gui/audit.py: add STREAM_UPDATE constant
- src/central/gui/routes.py: add streams_list and streams_update handlers
- src/central/gui/templates/base.html: add Streams nav link
- src/central/gui/templates/streams_list.html: new template
- tests/test_streams.py: 9 tests covering all requirements

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(gui): use get_msg().time for stream timestamps, fix badge layout

- nats-py StreamState doesn't expose first_ts/last_ts
- Fetch timestamps via js.get_msg(stream, seq=N).time instead
- Handle edge cases: empty streams, single-message streams, get_msg failures
- Fix badge overlap using flex layout instead of float:right
- Change label from "Max bytes (config)" to "Max bytes (current)"

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 18:04:23 -06:00
1dbc54e182
feat(gui): Leaflet region picker (1b-5) (#19)
* feat(gui): add Leaflet region picker to adapter edit (1b-5)

- Add _region_picker.html template with Leaflet map and editable rectangle
- Add Leaflet 1.9.4 and Leaflet.draw 1.0.4 CDN deps to adapters_edit.html
- Update GET /adapters/{name} to fetch map_tile_url from config.system
- Update POST /adapters/{name} to validate and save region coordinates
- Validation: -90 <= south < north <= 90, -180 <= west < east <= 180
- Region changes flow through to audit log via existing settings capture

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(tests): update adapter tests for region picker mocks

Add region coordinates to form data mocks and system settings rows
to fetchrow.side_effect for tests that re-render on validation errors.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Ubuntu <zvx@cortex.echo6.co>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Matt Johnson <mj@k7zvx.com>
2026-05-17 16:53:27 -06:00
Ubuntu
9396e5dbe8 fix(gui): dashboard polls card + CSRF exception handler
Fix A - /dashboard/polls:
- Use get_last_msg instead of pull_subscribe (no durable consumers)
- Fix subject filter: central.meta.adapter.{name}.status
- Parse correct fields: ts and ok from status message
- Handle NotFoundError gracefully when no status exists

Fix B - CSRF exception handler:
- Add global CsrfProtectError handler in __init__.py
- Return friendly "session expired" message instead of 500
- Re-render forms with error or redirect to /login
- Update templates to display error messages

Tests:
- Add get_last_msg mocking tests for polls
- Add regression test verifying no pull_subscribe
- Add CSRF handler tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 22:34:13 +00:00
Ubuntu
0f127399b3 fix(gui): remove JSONB double-encoding in adapter updates
The GUI pool has init=_setup_json_codec registered, which makes asyncpg
auto-serialize Python dicts to JSONB. Calling json.dumps() on a dict
before passing it to asyncpg double-encodes - the value gets stored as
a JSON-encoded string rather than a JSON object.

Changes:
- Remove json.dumps() from UPDATE statement in adapters_edit_submit
- Remove defensive isinstance(settings, str) checks that masked the bug
- Add regression tests to verify settings is passed as dict, not string

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 21:33:48 +00:00
Matt Johnson
dec8ce8545 feat(gui): add adapters list and edit UI (1b-4)
- Add GET /adapters route for listing all adapters
- Add GET /adapters/{name} for edit form with per-adapter fields
- Add POST /adapters/{name} for validation, update, and audit
- Add ADAPTER_UPDATE audit constant
- Add Adapters nav link to base.html
- Server-side validation for cadence (60-3600), email format,
  api_key_alias existence, satellites, and feed values
- Region displayed read-only with 1b-5 placeholder
- Hot reload via existing NOTIFY trigger (no new mechanism)
- Add comprehensive tests (9 tests)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 21:19:40 +00:00
Matt Johnson
736b637d31 feat(gui): add read-only dashboard with HTMX polling
- Add NATS connection module (nats.py) for JetStream access
- Add three dashboard cards: events (24h), stream sizes, poll times
- Replace placeholder index with HTMX-polling dashboard
- Graceful degradation when NATS unavailable (200 with error, not 500)
- Per-stream/adapter failure isolation
- Add comprehensive dashboard tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 20:09:05 +00:00
Matt Johnson
6b5f6709e4 fix(archive): subscribe to all event streams
- One durable consumer per event-bearing stream (CENTRAL_WX,
  CENTRAL_FIRE, CENTRAL_QUAKE) for independent ack tracking
- max_deliver=5 prevents poison-message infinite loops
- Orphaned 'archive' consumer on CENTRAL_WX cleaned up on startup
- Consumer naming: archive-{stream_name_lower}

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 19:29:38 +00:00
Ubuntu
8601a19f60 feat(schema): add adapter column to events, drop source
Replaces module-path-based source column (e.g. "central/adapters/nws")
with stable adapter identifier (e.g. "nws") that foreign-keys to
config.adapters.name.

Migration 011:
- ADD COLUMN adapter TEXT
- Backfill via REPLACE(source, 'central/adapters/', '')
- SET NOT NULL + FK RESTRICT
- CREATE INDEX (adapter, received DESC) for dashboard queries
- DROP COLUMN source

Code changes:
- Event model: source field renamed to adapter
- All adapters: use adapter="name" instead of source="central/adapters/name"
- Archive: write adapter column instead of source

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 16:09:59 +00:00
Ubuntu
e469c3833b fix(gui): pass raw CSRF token to form templates
The library's validate_csrf expects the raw token in the form and
the signed token in the cookie. Previously we were putting the signed
token in both places, which caused signature mismatch errors.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 07:05:25 +00:00
Ubuntu
17dd653bd8 fix(gui): use fastapi-csrf-protect native body-token validation
The library supports form-data tokens via token_location="body" and
token_key config options, which we missed in the initial integration.
Removed hand-rolled _validate_csrf_form helper in favor of the
library's validate_csrf method.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 07:00:57 +00:00
Matt Johnson
c529708c75 fix(gui): add form-based CSRF validation and fix index context
- Add _validate_csrf_form helper for form-based CSRF token validation
  (compares form csrf_token with fastapi-csrf-token cookie)
- Fix index route to pass operator and csrf_token to template context

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 06:28:16 +00:00
Matt Johnson
1fefc0f491 fix(gui): revert port to 8000, use 302 for setup gate redirect
- Revert uvicorn port from 8088 to 8000 (1b-1 pinned value)
- Change SetupGateMiddleware redirect from 307 to 302 for consistency
  with all other redirects in the codebase

Port 8000 confirmed free on CT104. Earlier change to 8088 was
incorrect — 8080 is held by NATS WebSocket, not 8000.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 06:13:13 +00:00
Matt Johnson
f059f982bc feat(gui): add auth core, setup gate, and first-run operator creation
- Add migrations 007-010 for system config, operators, sessions, audit_log
- Implement argon2id password hashing via argon2-cffi
- Implement session-based authentication with database-stored tokens
- Add SetupGateMiddleware to redirect to /setup until first operator created
- Add SessionMiddleware to load session from cookie and attach operator
- Create /setup, /login, /logout, /change-password routes with CSRF protection
- Add periodic session cleanup task (hourly)
- Add audit logging for auth events
- Update systemd unit with EnvironmentFile for /etc/central/central.env
- Add comprehensive tests for auth, middleware, and audit modules

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 05:30:49 +00:00
Matt Johnson
614312db36 feat(gui): add FastAPI + Jinja2 + HTMX scaffold
- FastAPI app with Jinja2 templates and Pico CSS + HTMX from CDN
- Routes: GET / (placeholder page), GET /health (JSON healthcheck)
- systemd unit (no Install section - manual start only)
- TestClient tests for both endpoints

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-17 04:32:39 +00:00
Matt Johnson
374a8c067f chore: normalize line endings to LF 2026-05-16 22:26:12 +00:00
Matt Johnson
39d5226661 feat(supervisor): wire USGS quake adapter
- Add USGSQuakeAdapter to _ADAPTER_REGISTRY
- Add CENTRAL_QUAKE stream to STREAM_SUBJECTS

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 20:51:46 +00:00
Matt Johnson
668027b442 feat(models): add quake event subject routing
Update subject_for_event to handle quake.* category events.
Subject format: central.quake.event.<magnitude_tier>

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 20:51:41 +00:00
Matt Johnson
aacf06499b feat(adapters): add USGS earthquake adapter
USGS Earthquake Hazards Program adapter:
- Polls GeoJSON feed (all_hour default, configurable)
- Magnitude tier classification (minor/light/moderate/strong/major/great)
- Deduplication via USGS stable event ID
- Region filter via shapely point-in-bbox
- Skips events with null magnitude (quarry blasts, etc.)

Includes comprehensive unit tests.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 20:51:36 +00:00
Matt Johnson
cbe9e50383 refactor(supervisor): use adapter registry pattern
- Add _ADAPTER_REGISTRY dict for adapter class lookup
- Unify adapter __init__ signatures (all take config, config_store, cursor_db_path)
- NWSAdapter now accepts config_store param (unused, for signature uniformity)
- Adding new adapters requires only one dict entry, no supervisor changes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 20:21:34 +00:00
Matt Johnson
95853200b2 fix(firms): use public sweep_old_ids method
Match NWS adapter pattern for supervisor compatibility.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 20:11:12 +00:00
Matt Johnson
22c50d3176 fix(firms): use public is_published/mark_published methods
Match NWS adapter pattern for supervisor compatibility.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 20:03:34 +00:00
Matt Johnson
5dbaf1dd5c feat(supervisor): wire FIRMS adapter
- Add FIRMSAdapter import and factory case
- Add CENTRAL_FIRE stream to STREAM_SUBJECTS

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 19:58:41 +00:00
Matt Johnson
a007418e0a feat(models): add fire event subject routing
Update subject_for_event to handle fire.* category events:
- Fire events: central.<category> (e.g., central.fire.hotspot.viirs_snpp.high)
- Weather events: existing geo-based subject logic

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 19:58:37 +00:00
Matt Johnson
0097163edf feat(adapters): add FIRMS fire hotspot adapter
NASA FIRMS adapter for VIIRS satellite fire detections:
- Polls VIIRS_SNPP_NRT and VIIRS_NOAA20_NRT satellites
- Deduplication via stable ID (satellite📅time:lat:lon)
- Hot-reload support for region, satellites, and API key
- Confidence mapping: l/n/h -> low/nominal/high
- Severity: high=3, nominal=2, low=1

Includes comprehensive unit tests for:
- CSV parsing and event generation
- Deduplication logic
- URL building and config application

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 19:58:31 +00:00
Matt Johnson
a157f39fe0 fix(nws): replace centroid filter with polygon intersection
- Add shapely dependency for geometry intersection
- Replace _point_in_region with _geometry_intersects_region
- Uses Shapely shape() and box() for proper GeoJSON handling
- Avoids false negatives on large alert polygons

Also adds antimeridian-crossing bbox rejection to RegionConfig validator.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 19:05:05 +00:00
Matt Johnson
f9426caa27 feat: add stream management infrastructure
- config_store: add stream CRUD methods
- stream_manager: ensure_stream, apply_retention, recompute_max_bytes
- Auto-clamp max_bytes to [1GB floor, 30% ceiling]
- Parse server max_file_store from nats-server.conf

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 18:50:12 +00:00
Matt Johnson
ab7126ec8d refactor(supervisor): remove adapter-specific branches, add stream wiring
- Replace if name == nws with generic apply_config call
- Add _create_adapter factory method
- Add stream management: ensure_stream, retention recompute loop
- Handle streams config changes via NOTIFY

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 18:49:53 +00:00
Matt Johnson
dfcc0c3a5c refactor(nws): migrate from states to bbox region filtering
- Add RegionConfig pydantic model with validators
- NWSAdapter now uses bbox for client-side alert filtering
- Implement apply_config for hot-reload of region changes
- Remove states-based filtering logic

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 18:49:46 +00:00
Matt Johnson
1ea56b67fd refactor(adapter): add abstract apply_config method
SourceAdapter now requires apply_config() for hot-reload support.
Each adapter implements its own config extraction from settings.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 18:49:40 +00:00
Matt Johnson
0eba319071 refactor(supervisor): remove adapter.cadence_s update
The NWSAdapter no longer has a cadence_s attribute since the
internal limiter was removed. The supervisor's rate limiting
via state.config.cadence_s and last_completed_poll is sufficient.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 17:18:55 +00:00
Matt Johnson
9d4ba97537 refactor(nws): remove internal AsyncLimiter rate limiting
The NWSAdapter had an internal AsyncLimiter that duplicated the
supervisor's rate-limit guarantee. When cadence changed, only
state.adapter.cadence_s was updated, not the internal limiter,
causing the cadence-decrease bug.

Since the supervisor already enforces rate limiting via
last_completed_poll + cadence_s scheduling, the adapter-level
limiter was redundant and caused the 30-second blocking observed
in diagnostic logs.

Removes:
- aiolimiter import
- self.cadence_s attribute (unused elsewhere)
- self._limiter creation
- async with self._limiter context in _fetch_alerts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 17:17:11 +00:00
Matt Johnson
4215744a30 fix: move cancel_event signal outside lock for immediate delivery
The cancel_event.set() call was inside the async lock context in
_on_config_change, causing delayed signal delivery to the sleeping
loop. This manifested as cadence decreases not applying without a
restart - the loop would sleep its full original timeout before
seeing the new cadence.

Fix: _reschedule_adapter now returns the AdapterState to signal,
and _on_config_change signals AFTER releasing the lock. This ensures
immediate event delivery per asyncio semantics.

The lock protects state consistency during config fetches and updates.
The cancel_event is a one-way notification that does not need lock
protection - it simply wakes the sleeping coroutine.

See docs/BUG-CADENCE-DECREASE.md for full investigation.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 05:59:45 +00:00
Ubuntu
c6f4f3b081 refactor: supervisor always uses DbConfigSource
Remove conditional config source loading, simplify async_main.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 03:42:38 +00:00
Ubuntu
a1e81bae8a refactor: remove config_source flag from bootstrap settings
Database config is now the only option, no need for feature flag.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 03:42:33 +00:00
Ubuntu
4376588baf refactor: remove TomlConfigSource, keep only DbConfigSource
TOML config is now retired. Database is the sole configuration source.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 03:42:28 +00:00
Matt Johnson
41439c52b3 refactor: rename DEFAULT_CLOUDEVENTS_CONFIG to CLOUDEVENTS_CONFIG
These are protocol-level constants, not defaults that get overridden.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 03:08:09 +00:00
Matt Johnson
73beb90b25 chore: remove unused ABC import from config_source.py
ConfigSource uses Protocol, not ABC. Removed unused import.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 02:38:15 +00:00
Matt Johnson
b16151abf1 refactor: move CloudEvents config to code constants
CloudEvents envelope format is protocol-level (not operator config).
When using DB config source without TOML, wrap_event() now uses
DEFAULT_CLOUDEVENTS_CONFIG from cloudevents_constants.py.

Changes:
- Add cloudevents_constants.py with DEFAULT_CLOUDEVENTS_CONFIG
- Update wrap_event() to accept Config, CloudEventsConfig, or None
- Simplify supervisor: always use wrap_event (has defaults)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 02:38:11 +00:00
Matt Johnson
c39e3174b8 fix: preserve last_completed_poll across adapter disable/enable
Previously, _stop_adapter() used pop() to remove adapter state,
which lost last_completed_poll. On re-enable, a fresh state was
created, causing immediate poll and violating rate-limit guarantee.

Changes:
- Add is_running property to AdapterState
- _stop_adapter: preserve state, just cancel task
- _start_adapter: reuse existing stopped state if present
- Add _remove_adapter for full cleanup when adapter is deleted
- _on_config_change: call _remove_adapter for deleted adapters

Integration tests verify:
- Test A: gap > cadence -> immediate poll (correct)
- Test B: gap < cadence -> wait until last_poll + cadence (was broken)
- Test C: delete + re-add -> fresh state (correct)

Tests-fail-before-fix verified: Test A/B failed on unfixed code
with "State was removed on stop!", pass with fix.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 02:37:57 +00:00
Ubuntu
daa7852cc0 refactor(archive): use bootstrap_config for connection strings
Archive now reads NATS URL and Postgres DSN from bootstrap_config
instead of TOML file. This is sufficient for archive since it only
needs connection strings, not adapter configuration.

No ConfigSource wiring needed - archive just consumes from JetStream.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 01:55:39 +00:00
Ubuntu
29fa49c5c2 feat(supervisor): add hot-reload support with rate-limit guarantee
Refactors supervisor to use ConfigSource abstraction:
- AdapterState tracks last_completed_poll for rate limiting
- Hot-reload via NOTIFY: cadence/enable/disable changes take effect
- Rate-limit guarantee: next poll at last_poll + new_cadence, not now
- Logs config source at startup (toml or db)
- Logs reschedule decisions with next poll timestamp

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 01:55:33 +00:00