central/tests/test_enrichment_config_plumbing.py
Matt Johnson b694fc0c9d fix(3-L.5): per-backend settings schemas (fixes build_enrichers TypeError)
Surfaced during the 2026-05-20 NaviBackend activation: toggling
config.enrichment.backend_class to NoOpBackend while backend_settings still
held {"base_url": ...} crashed _rebuild_enrichers with
`TypeError: NoOpBackend() takes no arguments`, BEFORE invalidate() ran. Fixed
by mirroring the SourceAdapter.settings_schema pattern: each backend declares a
Pydantic settings_schema; validation happens at write-time (GUI POST) and
read-time (supervisor). A mismatch is now a clean ValidationError, never a
constructor TypeError.

Backends — each gets a `<Name>BackendSettings(BaseModel, extra="forbid")` +
`settings_schema` class attr, mirroring __init__ defaults EXACTLY (note:
timeout_s stays 10.0 — the brief's "5.0" was a transcription slip; preserve the
production default):
  NoOpBackend     -> NoOpBackendSettings        (no fields)
  NaviBackend     -> NaviBackendSettings        (base_url, timeout_s, headers, warmup)
  PhotonBackend   -> PhotonBackendSettings       (base_url, timeout_s, headers)
  NominatimBackend-> NominatimBackendSettings   (base_url, user_agent, rate_limit_per_sec, timeout_s)

GeocoderBackend Protocol (in geocoder.py, where the base actually lives — not
base.py, which only has Enricher) gains `settings_schema: type[BaseModel]`.

supervisor:
- build_enrichers validates backend_cls.settings_schema.model_validate(
  backend_settings) before instantiating, and constructs from the validated
  .model_dump(). ValidationError (not TypeError) on mismatch.
- _rebuild_enrichers builds into locals and commits to instance state only on
  success — a ValidationError leaves the previously-active enrichers/config/
  cache untouched.
- _handle_enrichment_change wraps the rebuild in try/except ValidationError:
  logs and returns, keeping the previous backend running (supervisor stays up;
  operator fixes the row; next NOTIFY applies cleanly). No cache invalidation
  on a failed change.

GUI /enrichment:
- GET skips the outer EnrichmentConfig.backend_settings field and renders a
  separate <fieldset> from describe_fields(backend_cls.settings_schema, ...)
  for the row's current backend_class. Backend fields namespaced bs_<name>.
- POST reassembles bs_<name> inputs into a backend_settings dict, validates it
  against the SUBMITTED backend_class's schema (so errors attach to the right
  fields when an operator is mid-switch), then validates the outer
  EnrichmentConfig. DB row written only if both pass; otherwise re-renders with
  field-level errors against the submitted backend.
- backend_class stays a plain text field (no <select>, no client-side reshape).

form_descriptors: generic `float -> "number"` widget (2 lines, mirrors K.5's
`dict -> "json"`), needed because backend schemas have float fields
(timeout_s, rate_limit_per_sec). Benefits any float field codebase-wide.

DB schema unchanged: backend_settings stays JSONB; validation moved to
use-site. _BACKEND_REGISTRY / _ENRICHER_REGISTRY unchanged beyond schema lookup.

Tests (test_backend_settings_schema.py, 11): schemas exist + extra='forbid';
Navi schema preserves defaults (timeout_s == 10.0); NoOp has zero fields;
build_enrichers raises ValidationError-not-TypeError for the exact 2026-05-20
case; supervisor keeps previous backend on a bad NOTIFY (the incident
scenario); valid NoOp-with-empty-settings applies + invalidates; GUI POST
rejects bad backend_settings without writing + re-renders against submitted
backend; GUI POST writes on valid settings. test_enrichment_config_plumbing
updated for the new context shape (outer_fields/backend_fields).

Verification: full pytest 546 passed, 1 skipped (was 535; +11). grep
subject_for_event/_ADAPTER_REGISTRY and grep 100.64.0./192.168.1. in src both
empty.

Does NOT touch PR L scope (events tab, remaining adapter enrichment_locations),
the DB schema, or the registries.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 23:10:10 +00:00

213 lines
8.1 KiB
Python

"""Tests for operator-settable EnrichmentConfig plumbing (PR K.5).
Covers: ConfigStore DB read/upsert, supervisor startup read + hot-reload
rebuild, cache invalidation on backend change (but not on TTL-only change),
EnrichmentCache.invalidate, the generic json widget for backend_settings, and
the /enrichment GUI render. No real DB / NATS — pool, config_source, and the
EnrichmentCache class are mocked.
"""
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from central.config_models import EnrichmentConfig
from central.enrichment.cache import EnrichmentCache
from central.enrichment.backends.navi import NaviBackend
from central.enrichment.backends.no_op import NoOpBackend
from central.gui.form_descriptors import describe_fields
# --- mock pool/conn helpers -------------------------------------------------
def _mock_pool(conn: MagicMock) -> MagicMock:
pool = MagicMock()
acquire_cm = MagicMock()
acquire_cm.__aenter__ = AsyncMock(return_value=conn)
acquire_cm.__aexit__ = AsyncMock(return_value=None)
pool.acquire = MagicMock(return_value=acquire_cm)
return pool
# --- ConfigStore --------------------------------------------------------------
@pytest.mark.asyncio
async def test_config_store_reads_enrichment_row():
from central.config_store import ConfigStore
conn = MagicMock()
conn.fetchrow = AsyncMock(return_value={
"enricher_class": "GeocoderEnricher",
"backend_class": "NaviBackend",
"backend_settings": {"base_url": "http://example.test:8440"},
"cache_ttl_s": 3600,
})
store = ConfigStore(_mock_pool(conn))
cfg = await store.get_enrichment_config()
assert isinstance(cfg, EnrichmentConfig)
assert cfg.backend_class == "NaviBackend"
assert cfg.backend_settings == {"base_url": "http://example.test:8440"}
assert cfg.cache_ttl_s == 3600
@pytest.mark.asyncio
async def test_config_store_falls_back_to_defaults_when_row_absent():
from central.config_store import ConfigStore
conn = MagicMock()
conn.fetchrow = AsyncMock(return_value=None)
store = ConfigStore(_mock_pool(conn))
cfg = await store.get_enrichment_config()
assert cfg == EnrichmentConfig() # framework defaults
assert cfg.backend_class == "NoOpBackend"
@pytest.mark.asyncio
async def test_config_store_upsert_passes_dict_settings():
from central.config_store import ConfigStore
conn = MagicMock()
conn.execute = AsyncMock()
store = ConfigStore(_mock_pool(conn))
cfg = EnrichmentConfig(backend_class="NaviBackend", backend_settings={"base_url": "x"})
await store.upsert_enrichment_config(cfg)
args = conn.execute.call_args.args
assert "INSERT INTO config.enrichment" in args[0]
# backend_settings passed as a dict (pool codec encodes to jsonb), not a str.
assert {"base_url": "x"} in args
# --- EnrichmentCache.invalidate ----------------------------------------------
@pytest.mark.asyncio
async def test_cache_invalidate_all(tmp_path):
cache = EnrichmentCache(tmp_path / "c.db", ttl_s=3600)
await cache.set("geocoder", 1.0, 2.0, {"name": "x"})
await cache.set("geocoder", 3.0, 4.0, {"name": "y"})
deleted = await cache.invalidate()
assert deleted == 2
assert await cache.get("geocoder", 1.0, 2.0) is None
@pytest.mark.asyncio
async def test_cache_invalidate_scoped_to_enricher(tmp_path):
cache = EnrichmentCache(tmp_path / "c.db", ttl_s=3600)
await cache.set("geocoder", 1.0, 2.0, {"name": "x"})
await cache.set("other", 1.0, 2.0, {"name": "z"})
deleted = await cache.invalidate("geocoder")
assert deleted == 1
assert await cache.get("geocoder", 1.0, 2.0) is None
assert await cache.get("other", 1.0, 2.0) == {"name": "z"}
# --- Supervisor startup read + hot-reload ------------------------------------
def _supervisor_with(enrichment_cfg: EnrichmentConfig):
"""Build a Supervisor with mocked deps and a mocked EnrichmentCache class
(so no real /var/lib cache file is touched)."""
from central import supervisor as sup_mod
config_source = MagicMock()
config_source.get_enrichment_config = AsyncMock(return_value=enrichment_cfg)
config_store = MagicMock()
sup = sup_mod.Supervisor(
config_source=config_source,
config_store=config_store,
nats_url="nats://localhost:4222",
)
return sup
@pytest.mark.asyncio
async def test_supervisor_builds_navi_from_config():
"""Given a config naming NaviBackend, the supervisor's enricher set wraps a
NaviBackend — proves the registry resolution end-to-end."""
with patch("central.supervisor.EnrichmentCache") as cache_cls:
cache_cls.return_value = MagicMock(invalidate=AsyncMock(return_value=0))
sup = _supervisor_with(
EnrichmentConfig(backend_class="NaviBackend",
backend_settings={"base_url": "http://x:8440", "warmup": False})
)
cfg = await sup._config_source.get_enrichment_config()
sup._rebuild_enrichers(cfg)
assert isinstance(sup._enrichers[0]._backend, NaviBackend)
@pytest.mark.asyncio
async def test_hot_reload_rebuilds_and_invalidates_on_backend_change():
from central import supervisor as sup_mod
with patch("central.supervisor.EnrichmentCache") as cache_cls:
invalidate = AsyncMock(return_value=5)
cache_cls.return_value = MagicMock(invalidate=invalidate)
# Start at NoOp.
sup = _supervisor_with(EnrichmentConfig())
sup._rebuild_enrichers(EnrichmentConfig())
assert isinstance(sup._enrichers[0]._backend, NoOpBackend)
# Config flips to Navi.
sup._config_source.get_enrichment_config = AsyncMock(
return_value=EnrichmentConfig(
backend_class="NaviBackend",
backend_settings={"base_url": "http://x:8440", "warmup": False},
)
)
await sup._handle_enrichment_change()
assert isinstance(sup._enrichers[0]._backend, NaviBackend)
invalidate.assert_awaited() # backend changed -> cache wiped
@pytest.mark.asyncio
async def test_hot_reload_does_not_invalidate_on_ttl_only_change():
with patch("central.supervisor.EnrichmentCache") as cache_cls:
invalidate = AsyncMock(return_value=0)
cache_cls.return_value = MagicMock(invalidate=invalidate)
sup = _supervisor_with(EnrichmentConfig())
sup._rebuild_enrichers(EnrichmentConfig())
# Same backend, only TTL changes.
sup._config_source.get_enrichment_config = AsyncMock(
return_value=EnrichmentConfig(cache_ttl_s=3600)
)
await sup._handle_enrichment_change()
invalidate.assert_not_awaited()
# --- generic json widget + GUI render ----------------------------------------
def test_describe_fields_renders_dict_as_json_widget():
fields = {f.name: f.widget for f in describe_fields(EnrichmentConfig, {})}
assert fields["backend_settings"] == "json"
assert fields["enricher_class"] == "text"
assert fields["cache_ttl_s"] == "number"
@pytest.mark.asyncio
async def test_enrichment_form_renders():
from central.gui.routes import enrichment_form
request = MagicMock()
request.state.operator = MagicMock(username="op")
request.state.csrf_token = "tok"
conn = MagicMock()
conn.fetchrow = AsyncMock(return_value={
"enricher_class": "GeocoderEnricher",
"backend_class": "NoOpBackend",
"backend_settings": {},
"cache_ttl_s": 86400,
})
templates = MagicMock()
templates.TemplateResponse.return_value = MagicMock()
with patch("central.gui.routes._get_templates", return_value=templates), \
patch("central.gui.routes.get_pool", return_value=_mock_pool(conn)):
await enrichment_form(request)
ctx = templates.TemplateResponse.call_args.kwargs["context"]
# PR L.5: outer fields exclude backend_settings (now a per-backend fieldset);
# NoOpBackend's fieldset has zero fields.
outer = {f.name for f in ctx["outer_fields"]}
assert "backend_settings" not in outer
assert "backend_class" in outer
assert ctx["backend_class"] == "NoOpBackend"
assert ctx["backend_fields"] == []
assert ctx["csrf_token"] == "tok"