meshai/tests/test_adapter_config_api.py
Matt Johnson (via Claude) 566b06de06 feat(v0.6-tail): close 5 v0.6-phase1-complete.md follow-ups
(1) Auto-call refresh-toggles on PUT /api/config/notifications
    meshai/dashboard/api/config_routes.py adds register_config_routes_hooks(app)
    which registers a FastAPI HTTP middleware: on any 2xx PUT whose path
    matches /api/config/notifications or /api/config, the middleware
    invokes _refresh_toggle_filter(app) which reaches into app.state.bus._
    pipeline_components["toggle_filter"] and calls .refresh(app.state.config).
    The dashboard no longer has to remember to ping POST /api/notifications/
    refresh-toggles after a toggle change. The explicit endpoint stays for
    backwards-compat.

(2) env_reporter block-size cap moved to adapter_config
    New registry row pipeline.env_reporter_block_chars (int, default 3000).
    meshai/notifications/env_reporter.py replaces the hardcoded
    _BLOCK_MAX_CHARS = 3000 with _DEFAULT_BLOCK_MAX_CHARS (the fallback) +
    a _block_cap() helper that reads from adapter_config on every slice.
    Mutating the row via PUT /api/adapter-config takes effect on the next
    env_reporter call -- no restart.

(3) Bulk-import endpoint for gauge_sites
    meshai/dashboard/api/gauge_sites_import.py adds
    POST /api/gauge-sites/import with two paths:
      format=csv      -- expects "data" (CSV text with header row matching
                          gauge_sites columns: site_id, gauge_name, lat, lon,
                          and optionally action_ft/flood_minor_ft/
                          flood_moderate_ft/flood_major_ft/enabled). UPSERT
                          via ON CONFLICT(site_id) DO UPDATE. Returns
                          {inserted, updated, skipped}.
      format=nws-ahps -- expects "wfo" (list of WFO codes). Fetches
                          water.weather.gov/ahps2/index.php?wfo=<WFO> for each,
                          regex-parses gauge links, then fetches up to 50
                          gauge detail pages per request and regex-parses
                          lat/lon + four threshold values. Best-effort; rows
                          stored under "AHPS-<gauge_id>" so they dont collide
                          with USGS-* ids. Returns the same shape plus
                          detail_fetched + errors list.
    Frontend (dashboard-frontend/src/pages/GaugeSites.tsx) gains a
    Import button + modal with two tabs (Paste CSV / Scrape NWS-AHPS)
    rendered via an ImportModal component. CSV tab has a 48-row textarea
    with the column-header hint inline; AHPS tab has a comma-separated WFO
    input defaulting to BOI. Both submit via fetch() and show the JSON
    response inline. Invalidates the curation cache server-side on any
    successful insert/update so nwis_handler sees the new gauges on its
    next call.

(4) WFIGS tombstone column -- CORRECTNESS
    v12.sql adds fires.tombstoned_at REAL (nullable) + idx_fires_tombstoned_at.
    meshai/central/wfigs_handler.py: the tombstone branch
    (kind=="wfigs_tombstone") UPDATE fires SET tombstoned_at=COALESCE(
    tombstoned_at, ?) so the first tombstone-time wins (idempotent against
    repeated tombstone envelopes).
    meshai/notifications/reminders/__init__.py: the wfigs tombstone
    termination condition now checks row["tombstoned_at"] IS NOT NULL.
    Reminders correctly STOP for closed fires -- before this change the
    8h cadence would have kept Active: broadcasts going indefinitely past
    a WFIGS removal.
    SCHEMA_VERSION 11 -> 12.

(5) Delete INCIDENT_BROADCAST_HEARTBEAT_S
    meshai/central/incident_handler.py: removed the dead constant
    (v0.5.9 REVISED dropped the heartbeat path but left the constant
    imported-but-never-read).
    tests/test_incident_handler.py: removed the orphan
    test_i_8h_heartbeat_triggers_update test (asserted None, used the
    deleted constant for time arithmetic) and the stray import line.

Tests (tests/test_tail_followups.py, 16 cases):
  - middleware fires refresh on PUT /api/config/notifications (200), does
    NOT fire on PUT /api/config/llm
  - env_reporter _block_cap() default 3000; mutate via PUT, invalidate,
    next read returns the new cap
  - CSV import inserts new rows, updates existing, skips bad rows,
    rejects missing required columns, rejects bad format
  - AHPS index parser extracts (gauge_id, name) from realistic HTML
  - AHPS detail parser extracts lat/lon + four thresholds from realistic
    HTML
  - fires has tombstoned_at column after migrations
  - wfigs tombstone branch stamps tombstoned_at
  - ReminderScheduler skips a fire whose tombstoned_at is NOT NULL
  - ReminderScheduler still fires for a fire whose tombstoned_at IS NULL
  - INCIDENT_BROADCAST_HEARTBEAT_S no longer importable

Foundation/API test counts bumped:
  REGISTRY 58 -> 59 (+ env_reporter_block_chars)
  schema_meta v11 -> v12

Test count: 844 -> 859 (+16 new, -1 deleted dead test). 0 regressions.
2026-06-05 21:37:05 +00:00

344 lines
10 KiB
Python

"""v0.6-3c API tests for adapter_config + adapter_meta routes.
Uses FastAPI TestClient against a tmp DB seeded by the conftest autouse
fixture. Covers: GET (list, per-adapter, single), PUT (incl. type
validation), POST reset, GET/PUT meta, cache invalidation propagation
to the accessor.
"""
from __future__ import annotations
import json
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
from meshai.adapter_config import adapter_config, invalidate_cache
from meshai.dashboard.api.adapter_config_routes import router
@pytest.fixture
def client():
app = FastAPI()
app.include_router(router, prefix="/api")
return TestClient(app)
# ============================================================================
# GET /api/adapter-config (grouped)
# ============================================================================
def test_list_returns_all_59_keys(client):
r = client.get("/api/adapter-config")
assert r.status_code == 200
body = r.json()
# 14 adapters with at least one key (itd_511 has zero -- not in the
# grouped dict because the SQL only returns rows that exist).
total = sum(len(v) for v in body.values())
assert total == 59
def test_list_grouped_by_adapter(client):
r = client.get("/api/adapter-config")
body = r.json()
assert "wfigs" in body
keys = {row["key"] for row in body["wfigs"]}
assert keys == {"cooldown_seconds", "anchor_max_mi",
"broadcast_on_acres", "broadcast_on_contained"}
def test_list_includes_type_value_default_description(client):
r = client.get("/api/adapter-config")
body = r.json()
row = next(row for row in body["wfigs"] if row["key"] == "cooldown_seconds")
assert row["value"] == 28800
assert row["default"] == 28800
assert row["type"] == "int"
assert row["description"]
# ============================================================================
# GET /api/adapter-config/{adapter}
# ============================================================================
def test_per_adapter_list(client):
r = client.get("/api/adapter-config/usgs_quake")
assert r.status_code == 200
body = r.json()
assert isinstance(body, list)
keys = {row["key"] for row in body}
assert keys == {
"regional_centroid", "regional_radius_mi",
"broadcast_pager_alerts", "global_mag_floor",
"regional_mag_floor", "escalate_mag_floor",
}
def test_per_adapter_empty_for_itd_511(client):
"""itd_511 has zero config keys post-3a.1; returns empty list, not 404."""
r = client.get("/api/adapter-config/itd_511")
assert r.status_code == 200
assert r.json() == []
# ============================================================================
# GET /api/adapter-config/{adapter}/{key}
# ============================================================================
def test_get_single_key(client):
r = client.get("/api/adapter-config/usgs_quake/global_mag_floor")
assert r.status_code == 200
body = r.json()
assert body["value"] == 3.0
assert body["default"] == 3.0
assert body["type"] == "float"
def test_get_unknown_key_404(client):
r = client.get("/api/adapter-config/wfigs/no_such_key")
assert r.status_code == 404
# ============================================================================
# PUT /api/adapter-config/{adapter}/{key}
# ============================================================================
def test_put_updates_value(client):
r = client.put(
"/api/adapter-config/usgs_quake/global_mag_floor",
json={"value": 2.8},
)
assert r.status_code == 200
assert r.json()["value"] == 2.8
# GET reflects the new value.
g = client.get("/api/adapter-config/usgs_quake/global_mag_floor")
assert g.json()["value"] == 2.8
def test_put_invalidates_accessor_cache(client):
"""The handler-side accessor reads the new value WITHOUT a restart."""
# Prime the cache with the default.
assert adapter_config.usgs_quake.global_mag_floor == 3.0
# Mutate via API.
client.put(
"/api/adapter-config/usgs_quake/global_mag_floor",
json={"value": 2.5},
)
# Accessor returns the new value -- cache invalidation worked.
assert adapter_config.usgs_quake.global_mag_floor == 2.5
def test_put_int_validation(client):
"""int field rejects string body."""
r = client.put(
"/api/adapter-config/wfigs/cooldown_seconds",
json={"value": "two hours"},
)
assert r.status_code == 400
def test_put_int_rejects_float_with_fraction(client):
r = client.put(
"/api/adapter-config/wfigs/cooldown_seconds",
json={"value": 3600.5},
)
assert r.status_code == 400
def test_put_int_accepts_float_with_integer_value(client):
r = client.put(
"/api/adapter-config/wfigs/cooldown_seconds",
json={"value": 3600.0},
)
assert r.status_code == 200
assert r.json()["value"] == 3600
def test_put_float_accepts_int(client):
r = client.put(
"/api/adapter-config/usgs_quake/global_mag_floor",
json={"value": 4},
)
assert r.status_code == 200
assert r.json()["value"] == 4.0
def test_put_bool_rejects_int(client):
r = client.put(
"/api/adapter-config/wfigs/broadcast_on_acres",
json={"value": 1},
)
assert r.status_code == 400
def test_put_str_validation(client):
r = client.put(
"/api/adapter-config/geocoder/photon_url",
json={"value": 42},
)
assert r.status_code == 400
def test_put_json_accepts_list(client):
r = client.put(
"/api/adapter-config/nws/broadcast_severities",
json={"value": ["Extreme"]},
)
assert r.status_code == 200
assert r.json()["value"] == ["Extreme"]
def test_put_json_accepts_dict(client):
r = client.put(
"/api/adapter-config/central/severity_thresholds",
json={"value": {"routine_max": 0, "priority_max": 1, "immediate_min": 2}},
)
assert r.status_code == 200
def test_put_json_accepts_none(client):
r = client.put(
"/api/adapter-config/firms/bbox",
json={"value": None},
)
assert r.status_code == 200
assert r.json()["value"] is None
def test_put_unknown_key_404(client):
r = client.put(
"/api/adapter-config/wfigs/no_such_key",
json={"value": 1},
)
assert r.status_code == 404
def test_put_missing_value_field(client):
r = client.put(
"/api/adapter-config/wfigs/cooldown_seconds",
json={},
)
assert r.status_code == 400
# ============================================================================
# POST /api/adapter-config/{adapter}/{key}/reset
# ============================================================================
def test_reset_restores_default(client):
# Mutate.
client.put(
"/api/adapter-config/usgs_quake/global_mag_floor",
json={"value": 2.8},
)
assert client.get("/api/adapter-config/usgs_quake/global_mag_floor").json()["value"] == 2.8
# Reset.
r = client.post("/api/adapter-config/usgs_quake/global_mag_floor/reset")
assert r.status_code == 200
assert r.json()["value"] == 3.0
# Accessor too.
invalidate_cache()
assert adapter_config.usgs_quake.global_mag_floor == 3.0
def test_reset_invalidates_cache(client):
"""Reset must invalidate the accessor cache the same as PUT."""
client.put(
"/api/adapter-config/usgs_quake/global_mag_floor",
json={"value": 2.8},
)
# Prime cache with the post-PUT value.
assert adapter_config.usgs_quake.global_mag_floor == 2.8
client.post("/api/adapter-config/usgs_quake/global_mag_floor/reset")
# Cache cleared -- next read returns the default.
assert adapter_config.usgs_quake.global_mag_floor == 3.0
def test_reset_unknown_key_404(client):
r = client.post("/api/adapter-config/wfigs/no_such_key/reset")
assert r.status_code == 404
# ============================================================================
# GET /api/adapter-meta
# ============================================================================
def test_list_meta(client):
r = client.get("/api/adapter-meta")
assert r.status_code == 200
body = r.json()
assert "wfigs" in body
assert body["wfigs"]["include_in_llm_context"] is True
# central / geocoder default to False
assert body["central"]["include_in_llm_context"] is False
assert body["geocoder"]["include_in_llm_context"] is False
# ============================================================================
# PUT /api/adapter-meta/{adapter}
# ============================================================================
def test_put_meta_toggles_llm_context(client):
r = client.put(
"/api/adapter-meta/itd_511",
json={"include_in_llm_context": False},
)
assert r.status_code == 200
assert r.json()["include_in_llm_context"] is False
# GET reflects the change.
g = client.get("/api/adapter-meta").json()
assert g["itd_511"]["include_in_llm_context"] is False
def test_put_meta_updates_display_name(client):
r = client.put(
"/api/adapter-meta/wfigs",
json={"display_name": "Active wildfires (WFIGS)"},
)
assert r.status_code == 200
assert r.json()["display_name"] == "Active wildfires (WFIGS)"
def test_put_meta_partial_update(client):
"""Only the fields in the body change; others survive."""
original = client.get("/api/adapter-meta").json()["wfigs"]
r = client.put(
"/api/adapter-meta/wfigs",
json={"include_in_llm_context": False},
)
assert r.status_code == 200
body = r.json()
assert body["display_name"] == original["display_name"] # unchanged
assert body["include_in_llm_context"] is False
def test_put_meta_rejects_bad_bool(client):
r = client.put(
"/api/adapter-meta/wfigs",
json={"include_in_llm_context": "yes"},
)
assert r.status_code == 400
def test_put_meta_rejects_empty_display_name(client):
r = client.put(
"/api/adapter-meta/wfigs",
json={"display_name": " "},
)
assert r.status_code == 400
def test_put_meta_unknown_adapter_404(client):
r = client.put(
"/api/adapter-meta/nonexistent_adapter",
json={"include_in_llm_context": False},
)
assert r.status_code == 404