central/tests/conftest.py

125 lines
3.5 KiB
Python
Raw Normal View History

"""Shared fixtures for auth tests."""
import asyncio
import tempfile
from pathlib import Path
from typing import AsyncGenerator
import asyncpg
import pytest
import pytest_asyncio
from unittest.mock import AsyncMock, MagicMock, patch
from central.bootstrap_config import Settings
@pytest.fixture(autouse=True)
def isolate_enrichment_cache(tmp_path, monkeypatch):
"""Redirect the supervisor's enrichment cache off the production path.
`central.supervisor.ENRICHMENT_CACHE_DB_PATH` defaults to
/var/lib/central/enrichment_cache.db. Constructing a Supervisor opens it,
so without this fixture the suite writes to (or, for any user without write
access to /var/lib/central, fails on) the live cache. Point it at a
per-test temp dir so no test ever touches the production path.
feat(nwis): site + stats enrichment — named location + WaterWatch normalcy band (v0.8.0) Opens the v0.8.x data-quality cleanup arc. Production code; central-gui AND central-supervisor restart (adapter contract + enrichment behavior change). NWIS events rendered as a bare "Water reading: 111 ft3/s" with an empty Location column -- an operator couldn't tell where the gauge is or whether 111 ft3/s is drought-low, normal, or near-flood. Coordinates were present but the reverse geocoder returns null city/state/county for rural gauge points, and USGS site + percentile data was never fetched. v0.8.0 fetches both. Approach B (adapter-owned, per the proposal decision): the NWIS adapter -- which already owns the USGS APIs -- fetches site metadata and daily stats itself and writes two provenance bundles under event.data["_enriched"]: - usgs_site {name, lat, lon, state, county} from the OGC monitoring-locations item-by-id (the API family the adapter already speaks; JSON, no RDB parser). - usgs_stats {value, percentile, class_label, severity_band, p10..p90, record_max, count, period} from the legacy RDB daily-statistics service (the OGC API has no stats endpoint). USGS percentiles are % of days at-or-below, so higher = higher flow; classified to the WaterWatch bands -> severity 0-4 (record=4, much above/below=3, above/below=2, normal=1; None reserved for "no stats", distinct from a normal-flow gauge). Severity is set on the event, so it drives the v0.7.1 severity chip-picker filter + v0.7.2 map-marker opacity. - new nwis_enrich.py: pure parse/classify/percentile/band helpers + a sqlite SiteStatsCache (site TTL 365d, stats TTL 90d -- one fetch per site+param serves every reading for the window, so a warm cache makes zero USGS calls). USGS down -> cached-if-present else all-null bundle; the event still publishes. Framework: the single agreed generic change -- supervisor apply_enrichment now MERGES into _enriched instead of overwriting, so the still-global geocoder phase doesn't clobber the adapter's bundles. No other adapter writes _enriched, so this is inert for them. GUI: _event_summaries/nwis.html -> "<site> -- <value> <units> (<band>, <Nth> percentile)", with graceful fallback to "<site> -- <value>" then the bare "Water reading:". _event_rows/nwis.html detail gains site/normalcy/typical/location rows. _events_rows.html Location column falls back generically to any _enriched.<source> carrying state/county when the geocoder is null (works for future enrichers). events.json contract unchanged (additions under _enriched only). conftest isolate_enrichment_cache also redirects NWIS_CACHE_DB_PATH off the prod path (unprivileged-user test isolation). Adds tests/test_nwis_enrichment.py (28 tests: parse, band edges incl P0/P9/P10/P75/P90/record, percentile interpolation, cache hit/miss/expire, adapter enrich + graceful-null + cache-hit-no-refetch, summary rendering per band). Full suite: 710 passed, 1 skipped (central and unprivileged zvx, 3x each). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 15:30:19 +00:00
Also redirects the NWIS adapter's site/stats cache (v0.8.0,
`central.adapters.nwis.NWIS_CACHE_DB_PATH`, same /var/lib/central prod
default) for the same reason NWISAdapter.startup() opens it.
"""
import central.supervisor as supervisor_mod
feat(nwis): site + stats enrichment — named location + WaterWatch normalcy band (v0.8.0) Opens the v0.8.x data-quality cleanup arc. Production code; central-gui AND central-supervisor restart (adapter contract + enrichment behavior change). NWIS events rendered as a bare "Water reading: 111 ft3/s" with an empty Location column -- an operator couldn't tell where the gauge is or whether 111 ft3/s is drought-low, normal, or near-flood. Coordinates were present but the reverse geocoder returns null city/state/county for rural gauge points, and USGS site + percentile data was never fetched. v0.8.0 fetches both. Approach B (adapter-owned, per the proposal decision): the NWIS adapter -- which already owns the USGS APIs -- fetches site metadata and daily stats itself and writes two provenance bundles under event.data["_enriched"]: - usgs_site {name, lat, lon, state, county} from the OGC monitoring-locations item-by-id (the API family the adapter already speaks; JSON, no RDB parser). - usgs_stats {value, percentile, class_label, severity_band, p10..p90, record_max, count, period} from the legacy RDB daily-statistics service (the OGC API has no stats endpoint). USGS percentiles are % of days at-or-below, so higher = higher flow; classified to the WaterWatch bands -> severity 0-4 (record=4, much above/below=3, above/below=2, normal=1; None reserved for "no stats", distinct from a normal-flow gauge). Severity is set on the event, so it drives the v0.7.1 severity chip-picker filter + v0.7.2 map-marker opacity. - new nwis_enrich.py: pure parse/classify/percentile/band helpers + a sqlite SiteStatsCache (site TTL 365d, stats TTL 90d -- one fetch per site+param serves every reading for the window, so a warm cache makes zero USGS calls). USGS down -> cached-if-present else all-null bundle; the event still publishes. Framework: the single agreed generic change -- supervisor apply_enrichment now MERGES into _enriched instead of overwriting, so the still-global geocoder phase doesn't clobber the adapter's bundles. No other adapter writes _enriched, so this is inert for them. GUI: _event_summaries/nwis.html -> "<site> -- <value> <units> (<band>, <Nth> percentile)", with graceful fallback to "<site> -- <value>" then the bare "Water reading:". _event_rows/nwis.html detail gains site/normalcy/typical/location rows. _events_rows.html Location column falls back generically to any _enriched.<source> carrying state/county when the geocoder is null (works for future enrichers). events.json contract unchanged (additions under _enriched only). conftest isolate_enrichment_cache also redirects NWIS_CACHE_DB_PATH off the prod path (unprivileged-user test isolation). Adds tests/test_nwis_enrichment.py (28 tests: parse, band edges incl P0/P9/P10/P75/P90/record, percentile interpolation, cache hit/miss/expire, adapter enrich + graceful-null + cache-hit-no-refetch, summary rendering per band). Full suite: 710 passed, 1 skipped (central and unprivileged zvx, 3x each). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 15:30:19 +00:00
import central.adapters.nwis as nwis_mod
monkeypatch.setattr(
supervisor_mod,
"ENRICHMENT_CACHE_DB_PATH",
tmp_path / "enrichment_cache.db",
)
feat(nwis): site + stats enrichment — named location + WaterWatch normalcy band (v0.8.0) Opens the v0.8.x data-quality cleanup arc. Production code; central-gui AND central-supervisor restart (adapter contract + enrichment behavior change). NWIS events rendered as a bare "Water reading: 111 ft3/s" with an empty Location column -- an operator couldn't tell where the gauge is or whether 111 ft3/s is drought-low, normal, or near-flood. Coordinates were present but the reverse geocoder returns null city/state/county for rural gauge points, and USGS site + percentile data was never fetched. v0.8.0 fetches both. Approach B (adapter-owned, per the proposal decision): the NWIS adapter -- which already owns the USGS APIs -- fetches site metadata and daily stats itself and writes two provenance bundles under event.data["_enriched"]: - usgs_site {name, lat, lon, state, county} from the OGC monitoring-locations item-by-id (the API family the adapter already speaks; JSON, no RDB parser). - usgs_stats {value, percentile, class_label, severity_band, p10..p90, record_max, count, period} from the legacy RDB daily-statistics service (the OGC API has no stats endpoint). USGS percentiles are % of days at-or-below, so higher = higher flow; classified to the WaterWatch bands -> severity 0-4 (record=4, much above/below=3, above/below=2, normal=1; None reserved for "no stats", distinct from a normal-flow gauge). Severity is set on the event, so it drives the v0.7.1 severity chip-picker filter + v0.7.2 map-marker opacity. - new nwis_enrich.py: pure parse/classify/percentile/band helpers + a sqlite SiteStatsCache (site TTL 365d, stats TTL 90d -- one fetch per site+param serves every reading for the window, so a warm cache makes zero USGS calls). USGS down -> cached-if-present else all-null bundle; the event still publishes. Framework: the single agreed generic change -- supervisor apply_enrichment now MERGES into _enriched instead of overwriting, so the still-global geocoder phase doesn't clobber the adapter's bundles. No other adapter writes _enriched, so this is inert for them. GUI: _event_summaries/nwis.html -> "<site> -- <value> <units> (<band>, <Nth> percentile)", with graceful fallback to "<site> -- <value>" then the bare "Water reading:". _event_rows/nwis.html detail gains site/normalcy/typical/location rows. _events_rows.html Location column falls back generically to any _enriched.<source> carrying state/county when the geocoder is null (works for future enrichers). events.json contract unchanged (additions under _enriched only). conftest isolate_enrichment_cache also redirects NWIS_CACHE_DB_PATH off the prod path (unprivileged-user test isolation). Adds tests/test_nwis_enrichment.py (28 tests: parse, band edges incl P0/P9/P10/P75/P90/record, percentile interpolation, cache hit/miss/expire, adapter enrich + graceful-null + cache-hit-no-refetch, summary rendering per band). Full suite: 710 passed, 1 skipped (central and unprivileged zvx, 3x each). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 15:30:19 +00:00
monkeypatch.setattr(
nwis_mod,
"NWIS_CACHE_DB_PATH",
tmp_path / "nwis_cache.db",
)
@pytest.fixture(scope="session")
def event_loop():
"""Create an event loop for the test session."""
loop = asyncio.new_event_loop()
yield loop
loop.close()
@pytest.fixture
def mock_settings():
"""Create mock settings for testing."""
return Settings(
db_dsn="postgresql://test:test@localhost/test",
nats_url="nats://localhost:4222",
csrf_secret="test-csrf-secret-for-testing-only-32chars",
)
@pytest.fixture
def mock_pool():
"""Create a mock database pool."""
pool = MagicMock()
pool.acquire = MagicMock()
pool.close = AsyncMock()
return pool
@pytest.fixture
def mock_conn():
"""Create a mock database connection."""
conn = MagicMock()
conn.fetchrow = AsyncMock()
conn.fetchval = AsyncMock()
conn.execute = AsyncMock()
return conn
feat(gui): implement first-run setup wizard (1b-8) (#24) * feat(gui): implement first-run setup wizard (1b-8) Add a 5-step setup wizard that replaces the single-step /setup: 1. Create Operator - create initial operator account 2. System Settings - configure map tile URL and attribution 3. API Keys - optionally add API keys for adapters 4. Configure Adapters - enable/disable adapters with region picker 5. Finish Setup - review and complete setup Key changes: - Update middleware to handle wizard URL structure and step routing - Add wizard routes for each step with proper auth checks - Create new templates using base_wizard.html for consistent styling - Add audit events for system.update and setup.complete - Update tests for new middleware behavior Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gui): handle CSRF errors on wizard paths Update csrf_exception_handler to re-render wizard forms with error message instead of redirecting to /login when CSRF validation fails. - /setup/operator: re-render with error - /setup/system: re-render with current system values + error - /setup/keys: re-render with current keys list + error - /setup/adapters: re-render with current adapter config + error - /setup/finish: re-render with summary data + error - /setup: redirect to /setup (middleware routes to appropriate step) Add error display to setup_keys.html and setup_finish.html templates. Add 7 new CSRF handler tests for wizard paths. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gui): region picker render + click-to-draw Bug A: Maps render blank on /setup/adapters for FIRMS and USGS because Leaflet computed zero dimensions before container layout settled. Fix: add setTimeout invalidateSize() after map creation. Bug B: No click-to-draw functionality - only drag corners. Fix: add L.Control.Draw for rectangle drawing with CREATED event handler to replace existing rectangle. Both fixes applied to: - setup_adapters.html (wizard inline JS) - _region_picker.html (standalone edit page) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gui): handle revisiting /setup/operator after operator created When an operator already exists, /setup/operator now shows a confirmation page instead of the create form. This prevents: - Unique constraint violations on duplicate username - Silent creation of duplicate operators GET /setup/operator: queries config.operators; if any exist, renders confirmation state with existing_operator context. POST /setup/operator: checks operator count before INSERT; if non-zero, renders confirmation state without inserting. Template updated with conditional to show "Operator Already Configured" message when existing_operator is set. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(csrf): replace fastapi-csrf-protect with session-bound CSRF Fixes CSRF race condition where every GET rotated the CSRF token, causing POST failures when users had multiple tabs or slow connections. Changes: - Remove fastapi-csrf-protect dependency - Add session-bound CSRF tokens stored in config.sessions table - Add pre-auth CSRF for unauthenticated routes (/login, /setup/operator) - Add csrf.py module for pre-auth token generation/validation - Update routes to use new CSRF token handling - Add migration 013 to add csrf_token column to sessions The session-bound approach ensures CSRF tokens remain stable for the duration of a session, eliminating the race condition. Note: Route tests (test_wizard.py, test_adapters.py, etc.) need refactoring to mock get_settings() instead of CsrfProtect dependency. Core auth/CSRF handler tests pass (74 tests). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(csrf): update test suite for session-bound CSRF tokens - Add CSRF fixtures to conftest.py for pre-auth and session CSRF - Update test_wizard.py: use bypass_pre_auth_csrf and patch_route_settings - Update test_adapters.py: set request.state.csrf_token and form mock data - Update test_api_keys.py: add CSRF token to form data for POST routes - Update test_streams.py: change return_value to side_effect for CSRF support - Update test_region_picker.py: add CSRF token handling - Update test_config_store.py: set CENTRAL_CSRF_SECRET env var in fixture All 285 tests now pass with session-bound CSRF validation. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Matt Johnson <mj@k7zvx.com>
2026-05-17 22:06:22 -06:00
# CSRF fixtures for route tests
@pytest.fixture
def bypass_pre_auth_csrf():
"""Patch pre-auth CSRF validation to always pass.
Use for tests of pre-auth routes: /login, /setup/operator
"""
with patch("central.gui.routes.validate_pre_auth_csrf", return_value=True):
with patch("central.gui.routes.generate_pre_auth_csrf", return_value=("test_csrf_token", "test_signed_token")):
yield
@pytest.fixture
def bypass_session_csrf():
"""Create a mock request with session CSRF properly configured.
Use for tests of authenticated routes that check request.state.csrf_token.
Returns a configured mock_request.
"""
request = MagicMock()
request.state.csrf_token = "test_csrf_token_12345"
request.state.operator = MagicMock()
request.state.operator.id = 1
request.state.operator.username = "testuser"
# Mock form() to return dict with matching CSRF token
form_data = {"csrf_token": "test_csrf_token_12345"}
async def mock_form():
return form_data
request.form = mock_form
request._form_data = form_data # Allow tests to modify form data
return request
@pytest.fixture
def patch_route_settings():
"""Patch get_settings in routes module."""
with patch("central.gui.routes.get_settings") as mock:
mock.return_value.csrf_secret = "test-csrf-secret-for-testing-only-32chars"
yield mock