mirror of
https://github.com/zvx-echo6/central.git
synced 2026-06-11 12:24:37 +02:00
feat(3-K.5): operator-settable EnrichmentConfig (config plumbing)
Bridge PR for v0.5.0. PR J wired the supervisor with a hardcoded EnrichmentConfig() default; PR K added real backends to the registry but left no operator path to select one. K.5 closes that gap by mirroring the config.adapters storage + LISTEN/NOTIFY hot-reload pattern. config.enrichment (migration 024): single-row table (id BOOLEAN PK CHECK (id = true), mirroring config.system). Columns enricher_class, backend_class, backend_settings JSONB, cache_ttl_s, updated_at. Reuses the existing config.set_updated_at + config.notify_config_change triggers (the NOTIFY function's ELSE branch emits 'enrichment:' for this keyless single-row table). Seeds framework DEFAULTS ONLY — GeocoderEnricher + NoOpBackend, empty backend_settings, 24h TTL. NO URLs/IPs/auth in the seed; a fresh deploy runs NoOp out of the box. Idempotent (CREATE IF NOT EXISTS / DROP TRIGGER IF EXISTS / INSERT ON CONFLICT DO NOTHING). Supervisor: - Reads config.enrichment at startup (start() -> config_source .get_enrichment_config()), overriding the constructor default. - Hot-reloads via _on_config_change(table == "enrichment"): re-reads the row, rebuilds the enricher set, and invalidates the enrichment cache when the enricher/backend/settings changed (a new backend must not keep serving the old backend's cached bundles until TTL). TTL-only changes retain the cache. - build_enrichers now takes an explicit EnrichmentCache (the supervisor owns it so it can invalidate); cache no longer built inside build_enrichers. ConfigStore / ConfigSource: get_enrichment_config() (falls back to defaults if the row is somehow absent) + upsert_enrichment_config(). Mirrors the adapter accessors. cache.py: EnrichmentCache.invalidate(enricher_name=None) — DELETE all or enricher-scoped; returns rows deleted. GUI /enrichment: GET renders the EnrichmentConfig form via the generic describe_fields machinery (no enrichment-specific Jinja); POST validates via Pydantic, writes config.enrichment, and lets the NOTIFY trigger propagate the hot-reload. New enrichment.html + a nav link. backend_settings (a dict field) needed a generic "json" widget in describe_fields + the template — usable by any dict-typed settings field, not enrichment-specific. Necessary deviation (surfaced): PR K shipped a deployment-specific default DEFAULT_BASE_URL = "http://192.168.1.130:8440" in navi.py. Bar (b) forbids deployer IPs in src, and operator-settable base_url is exactly K.5's purpose, so the default is changed to http://localhost:8440 (matching Photon/Nominatim defaults). The live integration smoke (tests/, env-gated, skipped) now reads the endpoint from NAVI_BASE_URL — no IP anywhere in src. Tests (test_enrichment_config_plumbing.py, 10): ConfigStore read / default fallback / upsert-passes-dict; cache invalidate all + scoped; supervisor builds NaviBackend from config; hot-reload rebuilds + invalidates on backend change; no-invalidate on TTL-only change; describe_fields json widget; /enrichment GET render. test_firms updated for the build_enrichers signature change. Hot-reload mechanism mirrored: Postgres LISTEN/NOTIFY on channel 'config_changed' (payload 'table:key'), same path adapters/streams use; the supervisor's existing _on_config_change dispatch gains an "enrichment" branch. Verification: full pytest 535 passed, 1 skipped (was 525; +10). Migration applied cleanly on the live prod schema; SELECT * FROM config.enrichment returns the NoOp default row. grep subject_for_event/_ADAPTER_REGISTRY and grep 100.64.0./192.168.1. in src both empty. Does NOT activate NaviBackend (ships NoOp default; operator action) and does NOT declare enrichment_locations on other adapters (PR L scope). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
54238093a5
commit
04c1d07b3f
13 changed files with 604 additions and 15 deletions
|
|
@ -1990,6 +1990,142 @@ async def streams_update(
|
|||
return RedirectResponse(url="/streams", status_code=302)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Enrichment config route
|
||||
# =============================================================================
|
||||
|
||||
|
||||
def _enrichment_fields(current: dict) -> list[FieldDescriptor]:
|
||||
"""Field descriptors for the single-row EnrichmentConfig form (generic
|
||||
machinery — same describe_fields used by adapter pages)."""
|
||||
from central.config_models import EnrichmentConfig
|
||||
|
||||
return describe_fields(EnrichmentConfig, current)
|
||||
|
||||
|
||||
async def _read_enrichment_row(conn) -> dict:
|
||||
row = await conn.fetchrow(
|
||||
"""
|
||||
SELECT enricher_class, backend_class, backend_settings, cache_ttl_s
|
||||
FROM config.enrichment WHERE id = true
|
||||
"""
|
||||
)
|
||||
return dict(row) if row is not None else {}
|
||||
|
||||
|
||||
@router.get("/enrichment", response_class=HTMLResponse)
|
||||
async def enrichment_form(request: Request) -> HTMLResponse:
|
||||
"""Render the enrichment config form."""
|
||||
templates = _get_templates()
|
||||
pool = get_pool()
|
||||
operator = request.state.operator
|
||||
|
||||
async with pool.acquire() as conn:
|
||||
current = await _read_enrichment_row(conn)
|
||||
|
||||
response = templates.TemplateResponse(
|
||||
request=request,
|
||||
name="enrichment.html",
|
||||
context={
|
||||
"operator": operator,
|
||||
"csrf_token": request.state.csrf_token,
|
||||
"fields": _enrichment_fields(current),
|
||||
"errors": None,
|
||||
"form_data": None,
|
||||
},
|
||||
)
|
||||
return response
|
||||
|
||||
|
||||
@router.post("/enrichment")
|
||||
async def enrichment_update(request: Request) -> Response:
|
||||
"""Validate + persist the enrichment config. Hot-reload picks it up via
|
||||
the config.enrichment NOTIFY trigger."""
|
||||
from central.config_models import EnrichmentConfig
|
||||
|
||||
templates = _get_templates()
|
||||
pool = get_pool()
|
||||
operator = request.state.operator
|
||||
|
||||
form = await request.form()
|
||||
if not form.get("csrf_token") or form.get("csrf_token") != request.state.csrf_token:
|
||||
raise CsrfValidationError("Invalid CSRF token")
|
||||
|
||||
errors: dict[str, str] = {}
|
||||
form_data: dict[str, Any] = {}
|
||||
parsed: dict[str, Any] = {}
|
||||
|
||||
for field in _enrichment_fields({}):
|
||||
raw = form.get(field.name, "")
|
||||
form_data[field.name] = raw
|
||||
if field.widget == "number":
|
||||
try:
|
||||
parsed[field.name] = int(raw) if raw else None
|
||||
except ValueError:
|
||||
errors[field.name] = f"{field.label} must be a number"
|
||||
elif field.widget == "json":
|
||||
if not raw or not raw.strip():
|
||||
parsed[field.name] = {}
|
||||
else:
|
||||
try:
|
||||
loaded = json.loads(raw)
|
||||
if not isinstance(loaded, dict):
|
||||
errors[field.name] = f"{field.label} must be a JSON object"
|
||||
else:
|
||||
parsed[field.name] = loaded
|
||||
except json.JSONDecodeError as e:
|
||||
errors[field.name] = f"{field.label} is not valid JSON: {e}"
|
||||
else: # text
|
||||
parsed[field.name] = raw.strip() if raw else None
|
||||
|
||||
if not errors:
|
||||
try:
|
||||
validated = EnrichmentConfig(
|
||||
**{k: v for k, v in parsed.items() if v is not None}
|
||||
)
|
||||
except ValidationError as e:
|
||||
for err in e.errors():
|
||||
loc = err["loc"][0] if err["loc"] else "unknown"
|
||||
errors[str(loc)] = err["msg"]
|
||||
|
||||
if errors:
|
||||
async with pool.acquire() as conn:
|
||||
current = await _read_enrichment_row(conn)
|
||||
response = templates.TemplateResponse(
|
||||
request=request,
|
||||
name="enrichment.html",
|
||||
context={
|
||||
"operator": operator,
|
||||
"csrf_token": request.state.csrf_token,
|
||||
"fields": _enrichment_fields(current),
|
||||
"errors": errors,
|
||||
"form_data": form_data,
|
||||
},
|
||||
status_code=200,
|
||||
)
|
||||
return response
|
||||
|
||||
async with pool.acquire() as conn:
|
||||
await conn.execute(
|
||||
"""
|
||||
INSERT INTO config.enrichment
|
||||
(id, enricher_class, backend_class, backend_settings, cache_ttl_s)
|
||||
VALUES (true, $1, $2, $3, $4)
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
enricher_class = EXCLUDED.enricher_class,
|
||||
backend_class = EXCLUDED.backend_class,
|
||||
backend_settings = EXCLUDED.backend_settings,
|
||||
cache_ttl_s = EXCLUDED.cache_ttl_s
|
||||
""",
|
||||
validated.enricher_class,
|
||||
validated.backend_class,
|
||||
validated.backend_settings, # encoded as jsonb by the pool codec
|
||||
validated.cache_ttl_s,
|
||||
)
|
||||
|
||||
return RedirectResponse(url="/enrichment", status_code=302)
|
||||
|
||||
|
||||
# Alias validation regex
|
||||
ALIAS_REGEX = re.compile(r'^[a-zA-Z0-9_]+$')
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue