meshai/meshai/dashboard/api/config_routes.py
Matt Johnson (via Claude) 566b06de06 feat(v0.6-tail): close 5 v0.6-phase1-complete.md follow-ups
(1) Auto-call refresh-toggles on PUT /api/config/notifications
    meshai/dashboard/api/config_routes.py adds register_config_routes_hooks(app)
    which registers a FastAPI HTTP middleware: on any 2xx PUT whose path
    matches /api/config/notifications or /api/config, the middleware
    invokes _refresh_toggle_filter(app) which reaches into app.state.bus._
    pipeline_components["toggle_filter"] and calls .refresh(app.state.config).
    The dashboard no longer has to remember to ping POST /api/notifications/
    refresh-toggles after a toggle change. The explicit endpoint stays for
    backwards-compat.

(2) env_reporter block-size cap moved to adapter_config
    New registry row pipeline.env_reporter_block_chars (int, default 3000).
    meshai/notifications/env_reporter.py replaces the hardcoded
    _BLOCK_MAX_CHARS = 3000 with _DEFAULT_BLOCK_MAX_CHARS (the fallback) +
    a _block_cap() helper that reads from adapter_config on every slice.
    Mutating the row via PUT /api/adapter-config takes effect on the next
    env_reporter call -- no restart.

(3) Bulk-import endpoint for gauge_sites
    meshai/dashboard/api/gauge_sites_import.py adds
    POST /api/gauge-sites/import with two paths:
      format=csv      -- expects "data" (CSV text with header row matching
                          gauge_sites columns: site_id, gauge_name, lat, lon,
                          and optionally action_ft/flood_minor_ft/
                          flood_moderate_ft/flood_major_ft/enabled). UPSERT
                          via ON CONFLICT(site_id) DO UPDATE. Returns
                          {inserted, updated, skipped}.
      format=nws-ahps -- expects "wfo" (list of WFO codes). Fetches
                          water.weather.gov/ahps2/index.php?wfo=<WFO> for each,
                          regex-parses gauge links, then fetches up to 50
                          gauge detail pages per request and regex-parses
                          lat/lon + four threshold values. Best-effort; rows
                          stored under "AHPS-<gauge_id>" so they dont collide
                          with USGS-* ids. Returns the same shape plus
                          detail_fetched + errors list.
    Frontend (dashboard-frontend/src/pages/GaugeSites.tsx) gains a
    Import button + modal with two tabs (Paste CSV / Scrape NWS-AHPS)
    rendered via an ImportModal component. CSV tab has a 48-row textarea
    with the column-header hint inline; AHPS tab has a comma-separated WFO
    input defaulting to BOI. Both submit via fetch() and show the JSON
    response inline. Invalidates the curation cache server-side on any
    successful insert/update so nwis_handler sees the new gauges on its
    next call.

(4) WFIGS tombstone column -- CORRECTNESS
    v12.sql adds fires.tombstoned_at REAL (nullable) + idx_fires_tombstoned_at.
    meshai/central/wfigs_handler.py: the tombstone branch
    (kind=="wfigs_tombstone") UPDATE fires SET tombstoned_at=COALESCE(
    tombstoned_at, ?) so the first tombstone-time wins (idempotent against
    repeated tombstone envelopes).
    meshai/notifications/reminders/__init__.py: the wfigs tombstone
    termination condition now checks row["tombstoned_at"] IS NOT NULL.
    Reminders correctly STOP for closed fires -- before this change the
    8h cadence would have kept Active: broadcasts going indefinitely past
    a WFIGS removal.
    SCHEMA_VERSION 11 -> 12.

(5) Delete INCIDENT_BROADCAST_HEARTBEAT_S
    meshai/central/incident_handler.py: removed the dead constant
    (v0.5.9 REVISED dropped the heartbeat path but left the constant
    imported-but-never-read).
    tests/test_incident_handler.py: removed the orphan
    test_i_8h_heartbeat_triggers_update test (asserted None, used the
    deleted constant for time arithmetic) and the stray import line.

Tests (tests/test_tail_followups.py, 16 cases):
  - middleware fires refresh on PUT /api/config/notifications (200), does
    NOT fire on PUT /api/config/llm
  - env_reporter _block_cap() default 3000; mutate via PUT, invalidate,
    next read returns the new cap
  - CSV import inserts new rows, updates existing, skips bad rows,
    rejects missing required columns, rejects bad format
  - AHPS index parser extracts (gauge_id, name) from realistic HTML
  - AHPS detail parser extracts lat/lon + four thresholds from realistic
    HTML
  - fires has tombstoned_at column after migrations
  - wfigs tombstone branch stamps tombstoned_at
  - ReminderScheduler skips a fire whose tombstoned_at is NOT NULL
  - ReminderScheduler still fires for a fire whose tombstoned_at IS NULL
  - INCIDENT_BROADCAST_HEARTBEAT_S no longer importable

Foundation/API test counts bumped:
  REGISTRY 58 -> 59 (+ env_reporter_block_chars)
  schema_meta v11 -> v12

Test count: 844 -> 859 (+16 new, -1 deleted dead test). 0 regressions.
2026-06-05 21:37:05 +00:00

252 lines
8.4 KiB
Python

"""Configuration API routes."""
import logging
from fastapi import APIRouter, HTTPException, Request
from meshai.config import (
Config,
_dataclass_to_dict,
_dict_to_dataclass,
load_config,
save_config,
)
from meshai.config_loader import save_section, get_config_dir_from_path
logger = logging.getLogger(__name__)
router = APIRouter(tags=["config"])
# Sections that require restart when changed
RESTART_REQUIRED_SECTIONS = {
"connection",
"llm",
"mesh_sources",
"meshmonitor",
"dashboard",
}
# Valid config section names
VALID_SECTIONS = {
"notifications",
"environmental",
"bot",
"connection",
"response",
"history",
"memory",
"context",
"commands",
"llm",
"weather",
"meshmonitor",
"knowledge",
"mesh_sources",
"mesh_intelligence",
"dashboard",
}
@router.get("/config")
async def get_full_config(request: Request):
"""Get full configuration."""
config = request.app.state.config
return _dataclass_to_dict(config)
@router.get("/config/{section}")
async def get_config_section(section: str, request: Request):
"""Get a specific configuration section."""
if section not in VALID_SECTIONS:
raise HTTPException(
status_code=404,
detail=f"Section '{section}' not found. Valid sections: {', '.join(sorted(VALID_SECTIONS))}"
)
config = request.app.state.config
if not hasattr(config, section):
raise HTTPException(status_code=404, detail=f"Section '{section}' not found")
section_data = getattr(config, section)
# Handle list types (mesh_sources)
if isinstance(section_data, list):
return [
_dataclass_to_dict(item) if hasattr(item, "__dataclass_fields__") else item
for item in section_data
]
# Handle dataclass types
if hasattr(section_data, "__dataclass_fields__"):
return _dataclass_to_dict(section_data)
return section_data
@router.put("/config/{section}")
async def update_config_section(section: str, request: Request):
"""Update a configuration section."""
if section not in VALID_SECTIONS:
raise HTTPException(
status_code=404,
detail=f"Section '{section}' not found. Valid sections: {', '.join(sorted(VALID_SECTIONS))}"
)
config_path = request.app.state.config_path
if not config_path:
raise HTTPException(status_code=500, detail="Config path not set")
try:
body = await request.json()
except Exception as e:
raise HTTPException(status_code=422, detail=f"Invalid JSON: {e}")
try:
# Get the section's dataclass type
field_info = Config.__dataclass_fields__.get(section)
if not field_info:
raise HTTPException(status_code=404, detail=f"Section '{section}' not found")
field_type = field_info.type
# Validate by coercing to the dataclass (runs __post_init__ validators),
# then persist via the multi-file / !include-aware save_section. The
# monolithic save_config cannot parse the !include orchestrator and blew
# up on every save in the prod layout (v0.4 C.2.1 fix).
if section == "mesh_sources":
from meshai.config import MeshSourceConfig
new_value = [
_dict_to_dataclass(MeshSourceConfig, item) if isinstance(item, dict) else item
for item in body
]
data_to_save = [
_dataclass_to_dict(v) if hasattr(v, "__dataclass_fields__") else v
for v in new_value
]
elif hasattr(field_type, "__dataclass_fields__"):
new_value = _dict_to_dataclass(field_type, body)
data_to_save = _dataclass_to_dict(new_value)
else:
new_value = body
data_to_save = body
config_dir = get_config_dir_from_path(config_path)
save_section(section, data_to_save, config_dir)
# Determine if restart is required
restart_required = section in RESTART_REQUIRED_SECTIONS
# Keep the live config in sync (no disk reload needed) when no restart is required
if not restart_required and getattr(request.app.state, "config", None) is not None:
try:
setattr(request.app.state.config, section, new_value)
except Exception:
pass
logger.info(f"Config section '{section}' updated, restart_required={restart_required}")
return {"saved": True, "restart_required": restart_required}
except ValueError as e:
raise HTTPException(status_code=422, detail=str(e))
except Exception as e:
logger.error(f"Config update error: {e}")
raise HTTPException(status_code=500, detail=str(e))
@router.post("/config/test-llm")
async def test_llm_connection(request: Request):
"""Test LLM backend connection."""
config = request.app.state.config
try:
# Create LLM backend based on config
api_key = config.resolve_api_key()
if not api_key:
return {"success": False, "error": "No API key configured"}
backend_name = config.llm.backend.lower()
if backend_name == "openai":
from meshai.backends import OpenAIBackend
backend = OpenAIBackend(config.llm, api_key, 0, 0)
elif backend_name == "anthropic":
from meshai.backends import AnthropicBackend
backend = AnthropicBackend(config.llm, api_key, 0, 0)
elif backend_name == "google":
from meshai.backends import GoogleBackend
backend = GoogleBackend(config.llm, api_key, 0, 0)
else:
return {"success": False, "error": f"Unknown backend: {backend_name}"}
# Send test prompt
response = await backend.generate("Reply with 'OK' if you can read this.", [])
await backend.close()
return {"success": True, "response": response}
except Exception as e:
logger.error(f"LLM test error: {e}")
return {"success": False, "error": str(e)}
# v0.6-6 -- live ToggleFilter refresh endpoint.
# Called by the frontend after PUT /api/config/notifications so the
# Inhibitor + Grouper + Dispatcher pick up the new enabled toggle set
# on the next event without a container restart.
def _refresh_toggle_filter(app) -> bool:
"""Best-effort live refresh of the running ToggleFilter. Returns True
when the refresh actually fired, False if the pipeline isn t up yet
(typical during tests / early startup). Never raises."""
try:
bus = getattr(app.state, "bus", None)
config = getattr(app.state, "config", None)
if bus is None or config is None:
return False
components = getattr(bus, "_pipeline_components", {}) or {}
tf = components.get("toggle_filter")
if tf is None:
return False
tf.refresh(config)
return True
except Exception:
logger.exception("toggle_filter refresh failed")
return False
@router.post("/notifications/refresh-toggles")
async def refresh_toggles(request: Request):
"""Explicit refresh endpoint (kept for backwards-compat with the
dashboard's manual ping path)."""
bus = getattr(request.app.state, "bus", None)
config = getattr(request.app.state, "config", None)
if bus is None or config is None:
raise HTTPException(503, "pipeline bus not yet initialized")
components = getattr(bus, "_pipeline_components", {}) or {}
tf = components.get("toggle_filter")
if tf is None:
raise HTTPException(503, "toggle_filter not on pipeline bus")
tf.refresh(config)
return {"ok": True}
# v0.6-tail item 1: auto-refresh the ToggleFilter after any successful
# config PUT that touches notifications. Registered from server.py at
# startup via register_config_routes_hooks(app).
def register_config_routes_hooks(app):
@app.middleware("http")
async def _auto_refresh_toggle_filter(request, call_next):
response = await call_next(request)
try:
method = request.method.upper()
path = request.url.path
if (method == "PUT"
and 200 <= response.status_code < 300
and ("/api/config/notifications" in path
or path.rstrip("/").endswith("/api/config"))):
_refresh_toggle_filter(request.app)
except Exception:
logger.exception("auto-refresh middleware failed")
return response