meshai/tests/test_adapter_config_foundation.py
Matt Johnson (via Claude) 3a410d5087 feat(v0.6-phase3): reminder system + schema split + NWS dedup relaxation
Third broadcast type Active: clock-driven re-broadcasts of still-live
events at human-scale cadences. WFIGS fires 8h, itd_511 work zones daily
8 AM Mountain, SWPC G-storms 8h. NWS is NOT a clock reminder -- instead
the per-CAP-id dedup is relaxed to allow re-broadcast if >3h since last.
Schema split first_broadcast_at + last_broadcast_at on all reminder-
eligible tables. Wire prefix logic: New (first sight), Update (WFIGS
material change), Active (clock reminder). All cadences, channels, day-
of-week patterns, timezones, and termination conditions GUI-editable
from day one via the existing adapter_config editor. Termination:
tombstone OR containment_100 OR end_date_passed (no max-count). Quiet
hours not respected -- ripped out in Phase 2.

Schema (v11.sql):
  - ALTER TABLE fires|nws_alerts|traffic_events|quake_events|swpc_events|
    gauge_readings ADD COLUMN first_broadcast_at REAL
  - Backfill: UPDATE ... SET first_broadcast_at = last_broadcast_at
    WHERE last_broadcast_at IS NOT NULL
  - ALTER TABLE adapter_meta ADD COLUMN reminder_enabled INTEGER NOT NULL
    DEFAULT 0
  - UPDATE adapter_meta SET reminder_enabled=1 WHERE adapter IN
    ('wfigs', 'swpc') -- itd_511_work_zone is a new meta row seeded
    with reminder_enabled=1
  - SCHEMA_VERSION 10 -> 11

Handler commit-callbacks (wfigs/nws/quake/swpc/incident):
  - UPDATE ... SET last_broadcast_at=?, first_broadcast_at=COALESCE(
    first_broadcast_at, ?) -- first_broadcast_at stamped once, never
    overwritten

NWS handler (meshai/central/nws_handler.py):
  - _render() gains a prefix kwarg
  - After-first-broadcast branch: when (now - last_broadcast_at) >=
    adapter_config.nws.duplicate_allowed_after_seconds (default 10800
    = 3h), allow the re-broadcast with prefix=Active. Under the
    window, suppress as before. The commit callback continues to
    update last_broadcast_at.

ReminderScheduler (meshai/notifications/reminders/__init__.py):
  - Async loop, ticks every 60s
  - Each tick: SELECT adapter FROM adapter_meta WHERE reminder_enabled=1
  - Per adapter, load reminders_<adapter> config from adapter_config
    (cadence_kind, cadence_value, channels, terminate_when, dow_mask,
    timezone)
  - Interval cadence: rows where last_broadcast_at <= now - cadence_value
  - Clock cadence: localizes now to configured tz, finds slots that
    just passed in the last tick window, gated by dow_mask
  - Termination conditions checked per adapter:
      wfigs.containment_100      -> current_contained_pct >= 100
      wfigs.last_event_age_24h   -> last_event_at older than 24h
      swpc.end_date_passed       -> payload_json end_time in past
      itd_511_work_zone.end_date_passed -> traffic_events.end_at in past
  - Active: prefix on every emitted wire; dispatcher.dispatch_scheduled_
    broadcast() honors cold-start grace, bypasses toggle path
  - On success, last_broadcast_at = now; first_broadcast_at preserved

Launched from notifications/pipeline/__init__.py:start_pipeline()
alongside BandConditionsScheduler.

adapter_config registry (+15 new keys, 43 -> 58):
  - reminders_wfigs.cadence_kind/cadence_value/channels/terminate_when
  - reminders_swpc.cadence_kind/cadence_value/channels/terminate_when
  - reminders_itd_511_work_zone.cadence_kind/cadence_value/channels/
    dow_mask/timezone/terminate_when
  - nws.duplicate_allowed_after_seconds

adapter_meta (+4 rows, 15 -> 19):
  - reminders_wfigs, reminders_swpc, reminders_itd_511_work_zone
    (pseudo-adapters carrying the reminder config)
  - itd_511_work_zone (reminder target row; reminder_enabled=1)
  - reminder_enabled flag added to wfigs/swpc (existing rows updated by
    v11.sql) and to itd_511_work_zone seed.

Tests (tests/test_reminders.py, 10 cases):
  - wfigs reminder fires past 8h cadence, stamps last_broadcast_at,
    preserves first_broadcast_at
  - reminder skipped within cadence
  - reminder skipped when containment_100, last_event_age_24h
  - swpc reminder fires (interval)
  - work_zone clock reminder fires at 08:00 Mountain on enabled DOW
  - work_zone reminder skipped when end_date_passed
  - work_zone reminder skipped outside slot window
  - reminder_enabled=0 suppresses all reminders for that adapter

tests/test_nws_dedup_relaxation.py (5 cases):
  - First sighting renders without Active: prefix
  - Re-broadcast within 3h suppressed
  - Re-broadcast after 3h allowed with Active: prefix
  - adapter_config.nws.duplicate_allowed_after_seconds override takes
    effect (1h window verified)
  - First sighting stamps first_broadcast_at=committed_at,
    last_broadcast_at=committed_at; 4h later broadcast stamps
    last_broadcast_at only, first_broadcast_at preserved

Test count: 829 -> 844 (+15 new, 0 regressions). Foundation tests
updated for new counts (REGISTRY=58, ADAPTER_META=19, schema=v11).
2026-06-05 21:11:32 +00:00

347 lines
11 KiB
Python

"""v0.6-3a foundation tests: migration, seed, accessor, orphan prune.
v0.6-3a.1: trimmed registry to 43 keys per Matt's CONFIG-vs-CODE rule.
prune_orphans cleans up rows that were in the v0.6-3a draft but no longer
in the trimmed REGISTRY. Tests now assert the 43-key count and exercise
the prune path.
"""
from __future__ import annotations
import json
import logging
from unittest.mock import patch
import pytest
from meshai.adapter_config import (
adapter_config,
invalidate_cache,
seed_defaults,
prune_orphans,
REGISTRY,
ADAPTER_META,
)
from meshai.adapter_config import _accessor as accessor_mod
from meshai.persistence import close_thread_connection, init_db
from meshai.persistence import db as persistence_db
# ---------- fixtures ------------------------------------------------------
@pytest.fixture
def fresh_db(tmp_path, monkeypatch):
p = str(tmp_path / "ac-test.sqlite")
monkeypatch.setenv("MESHAI_DB_PATH", p)
persistence_db._initialised.clear()
close_thread_connection()
invalidate_cache()
conn = init_db()
yield conn
close_thread_connection()
persistence_db._initialised.discard(p)
invalidate_cache()
# ---------- schema --------------------------------------------------------
def test_v6_tables_exist(fresh_db):
tables = {r["name"] for r in fresh_db.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
).fetchall()}
assert "adapter_config" in tables
assert "adapter_meta" in tables
def test_schema_meta_at_v11(fresh_db):
v = fresh_db.execute(
"SELECT value FROM schema_meta WHERE key='version'"
).fetchone()["value"]
assert int(v) == 11
def test_adapter_config_type_check_constrains_vocabulary(fresh_db):
with pytest.raises(Exception):
fresh_db.execute(
"INSERT INTO adapter_config(adapter, key, value_json, default_json, "
"type, description, updated_at) VALUES (?,?,?,?,?,?,?)",
("x", "y", "1", "1", "integer", "", 0.0),
)
# ---------- registry shape -----------------------------------------------
def test_registry_at_58_entries():
"""v0.6-3a.1 trim: 43 CONFIG-only keys (was 77 in v0.6-3a draft)."""
assert len(REGISTRY) == 58, (
f"REGISTRY should have 43 entries after CONFIG-vs-CODE trim; got {len(REGISTRY)}. "
f"If a sentence template / emoji / heuristic snuck in, it belongs in CODE not config."
)
def test_adapter_meta_at_19(fresh_db):
assert len(ADAPTER_META) == 19
# ---------- seed ----------------------------------------------------------
def test_seed_populates_every_registry_row(fresh_db):
rows = fresh_db.execute("SELECT adapter, key FROM adapter_config").fetchall()
db_keys = {(r["adapter"], r["key"]) for r in rows}
assert db_keys == set(REGISTRY.keys())
def test_seed_value_matches_registry_default(fresh_db):
for (adapter, key), spec in REGISTRY.items():
row = fresh_db.execute(
"SELECT value_json, default_json, type FROM adapter_config "
"WHERE adapter=? AND key=?",
(adapter, key),
).fetchone()
expected = json.dumps(spec["default"])
assert row["value_json"] == expected, f"{adapter}.{key} value drift"
assert row["default_json"] == expected, f"{adapter}.{key} default drift"
assert row["type"] == spec["type"]
def test_seed_populates_every_adapter_meta_row(fresh_db):
rows = fresh_db.execute("SELECT adapter, include_in_llm_context FROM adapter_meta").fetchall()
db_adapters = {r["adapter"] for r in rows}
assert db_adapters == set(ADAPTER_META.keys())
def test_seed_is_idempotent(fresh_db):
a, b = seed_defaults(fresh_db)
assert a == 0 and b == 0
def test_seed_does_not_overwrite_user_edits(fresh_db):
fresh_db.execute(
"UPDATE adapter_config SET value_json=? WHERE adapter=? AND key=?",
("999", "wfigs", "cooldown_seconds"),
)
seed_defaults(fresh_db)
row = fresh_db.execute(
"SELECT value_json FROM adapter_config "
"WHERE adapter='wfigs' AND key='cooldown_seconds'"
).fetchone()
assert row["value_json"] == "999"
# ---------- prune_orphans -------------------------------------------------
def test_prune_orphans_removes_unknown_keys(fresh_db, caplog):
"""A row whose (adapter, key) is no longer in REGISTRY is deleted on
the next prune_orphans, and the delete is logged at INFO."""
fresh_db.execute(
"INSERT INTO adapter_config(adapter, key, value_json, default_json, "
"type, description, updated_at) VALUES (?,?,?,?,?,?,?)",
("wfigs", "deprecated_legacy_key", "\"old\"", "\"old\"",
"str", "", 0.0),
)
caplog.set_level(logging.INFO, logger="meshai.adapter_config")
removed = prune_orphans(fresh_db)
assert removed == 1
msgs = [r.getMessage() for r in caplog.records
if r.name.startswith("meshai.adapter_config")]
assert any(
"adapter_config orphan removed: wfigs.deprecated_legacy_key" in m
for m in msgs
), f"expected orphan-removed log line; got: {msgs}"
# Row gone.
assert fresh_db.execute(
"SELECT 1 FROM adapter_config WHERE adapter=? AND key=?",
("wfigs", "deprecated_legacy_key"),
).fetchone() is None
def test_prune_orphans_idempotent(fresh_db):
assert prune_orphans(fresh_db) == 0
assert prune_orphans(fresh_db) == 0
def test_prune_orphans_does_not_touch_known_keys(fresh_db):
"""Every REGISTRY row survives the prune."""
before = {(r["adapter"], r["key"]) for r in fresh_db.execute(
"SELECT adapter, key FROM adapter_config"
).fetchall()}
prune_orphans(fresh_db)
after = {(r["adapter"], r["key"]) for r in fresh_db.execute(
"SELECT adapter, key FROM adapter_config"
).fetchall()}
assert before == after == set(REGISTRY.keys())
def test_prune_orphans_does_not_touch_adapter_meta(fresh_db):
"""A previously-known adapter whose config keys all moved to CODE
keeps its adapter_meta row (for the include_in_llm_context toggle)."""
before = fresh_db.execute("SELECT COUNT(*) FROM adapter_meta").fetchone()[0]
prune_orphans(fresh_db)
after = fresh_db.execute("SELECT COUNT(*) FROM adapter_meta").fetchone()[0]
assert before == after == len(ADAPTER_META)
def test_prune_orphans_invalidates_cache(fresh_db):
"""If a key disappears, any cached read of it should NOT linger."""
invalidate_cache()
# Prime cache with a key that will become orphan.
fresh_db.execute(
"INSERT INTO adapter_config(adapter, key, value_json, default_json, "
"type, description, updated_at) VALUES (?,?,?,?,?,?,?)",
("wfigs", "ghost", "42", "42", "int", "", 0.0),
)
# We can't read it via accessor (not in REGISTRY -> no fallback) so
# we just verify the cache is empty after prune.
prune_orphans(fresh_db)
assert accessor_mod._cache == {}, "cache should be cleared after orphan prune"
# ---------- accessor ------------------------------------------------------
def test_accessor_returns_int(fresh_db):
invalidate_cache()
assert adapter_config.wfigs.cooldown_seconds == 28800
def test_accessor_returns_float(fresh_db):
invalidate_cache()
assert adapter_config.usgs_quake.global_mag_floor == 3.0
def test_accessor_returns_str(fresh_db):
invalidate_cache()
assert adapter_config.geocoder.photon_url == "http://100.64.0.24:2322"
def test_accessor_returns_bool(fresh_db):
invalidate_cache()
assert adapter_config.tomtom_incidents.drop_zero_magnitude is True
def test_accessor_returns_json_list(fresh_db):
invalidate_cache()
assert adapter_config.nws.broadcast_severities == ["Extreme", "Severe"]
def test_accessor_returns_json_dict(fresh_db):
invalidate_cache()
v = adapter_config.central.severity_thresholds
assert v == {"routine_max": 1, "priority_max": 2, "immediate_min": 3}
def test_accessor_returns_json_none(fresh_db):
invalidate_cache()
assert adapter_config.firms.bbox is None
def test_firms_dedup_distance_m_default(fresh_db):
"""v0.6-3a.1 Matt's call: user-facing unit is meters, default 5."""
invalidate_cache()
v = adapter_config.firms.dedup_distance_m
assert isinstance(v, int)
assert v == 5
# ---------- cache --------------------------------------------------------
def test_cache_hits_second_read(fresh_db):
invalidate_cache()
_ = adapter_config.wfigs.cooldown_seconds
with patch.object(accessor_mod, "_load_from_db",
side_effect=AssertionError("cache miss")):
v = adapter_config.wfigs.cooldown_seconds
assert v == 28800
def test_invalidate_forces_reload(fresh_db):
invalidate_cache()
_ = adapter_config.wfigs.cooldown_seconds
fresh_db.execute(
"UPDATE adapter_config SET value_json=? WHERE adapter='wfigs' AND key='cooldown_seconds'",
("3600",),
)
assert adapter_config.wfigs.cooldown_seconds == 28800 # still cached
invalidate_cache()
assert adapter_config.wfigs.cooldown_seconds == 3600
# ---------- defensive fallback paths -------------------------------------
def test_registry_fallback_when_db_row_missing(fresh_db, caplog):
invalidate_cache()
fresh_db.execute(
"DELETE FROM adapter_config WHERE adapter='wfigs' AND key='cooldown_seconds'"
)
caplog.set_level(logging.WARNING, logger="meshai.adapter_config._accessor")
v = adapter_config.wfigs.cooldown_seconds
assert v == 28800
assert any("missing from DB" in r.message for r in caplog.records)
def test_unknown_key_raises(fresh_db):
invalidate_cache()
with pytest.raises(AttributeError):
_ = adapter_config.wfigs.no_such_key
def test_setattr_blocked(fresh_db):
invalidate_cache()
with pytest.raises(AttributeError):
adapter_config.wfigs.cooldown_seconds = 999
# ---------- registry sanity ----------------------------------------------
def test_every_registry_default_round_trips_through_json():
for (adapter, key), spec in REGISTRY.items():
encoded = json.dumps(spec["default"])
decoded = json.loads(encoded)
assert decoded == spec["default"], f"{adapter}.{key}: JSON round-trip drift"
def test_every_registry_type_is_in_vocabulary():
valid = {"int", "float", "str", "bool", "json"}
for (adapter, key), spec in REGISTRY.items():
assert spec["type"] in valid, f"{adapter}.{key}: invalid type {spec['type']!r}"
def test_adapter_meta_includes_every_registry_adapter():
reg_adapters = {a for a, _ in REGISTRY}
meta_adapters = set(ADAPTER_META)
missing = reg_adapters - meta_adapters
assert not missing, f"adapters in REGISTRY but missing ADAPTER_META: {missing}"
# ---------- guard against CODE leaking back into the registry -----------
def test_no_emoji_keys_in_registry():
"""Emoji choices are CODE, not config (Matt's locked rule)."""
for (adapter, key) in REGISTRY:
assert "emoji" not in key, (
f"{adapter}.{key} looks like an emoji setting; emojis are CODE"
)
def test_no_template_keys_in_registry():
"""Sentence templates are CODE."""
for (adapter, key) in REGISTRY:
assert "template" not in key and "prefix" not in key, (
f"{adapter}.{key} looks like a sentence template / prefix; sentences are CODE"
)
def test_no_map_keys_in_registry():
"""Translation maps are CODE (TomTom icon_map, ITD sub_type_map, etc.)."""
for (adapter, key) in REGISTRY:
assert not key.endswith("_map"), (
f"{adapter}.{key} looks like a translation map; mapping functions are CODE"
)