meshai/tests/conftest.py

52 lines
2 KiB
Python
Raw Normal View History

feat(v0.6-2): dispatcher state persistence -- cold-start, cooldowns, dedup LRU to SQLite Closes Rule-20 dispatcher gap from audit doc v0.6-phase1-audit.md finding #1. Pre-this-commit the cold-start anchor, 4 drop counters, per-toggle cooldown map, and dedup OrderedDict all lived in Dispatcher instance memory and were lost on every container restart. v5.sql adds three tables: - dispatcher_state (singleton id=1): cold_start_anchor + 4 drop counters - dispatcher_cooldowns ((toggle,category,region) keyed): last_fired_at - dispatcher_dedup ((source,event_id) keyed): seen_at Dispatcher refactor: - __init__ calls _restore_from_db -- counters, cold-start anchor, cooldown map, and dedup LRU (most-recent 10k by seen_at) all rehydrated from the three new tables - write-through on every mutation: _persist_state for counter/anchor, _persist_cooldown for cooldown UPSERT + 2*cooldown_s prune, _persist_dedup for dedup INSERT OR REPLACE + 7-day cleanup - in-memory caches stay authoritative on the fast read path - cumulative-since-install counters (NOT since-boot); LLM will be able to answer "we have dropped 47 stale events this week" after commit #5 (env_reporter) lands - graceful degrade: missing v5 tables / persistence outage falls back to fresh in-memory state without crashing the constructor Tests: - tests/test_dispatcher_persistence.py (17 tests): state restore on init, counter+cooldown+dedup survival across simulated restart, cooldown rearm within 2x window, dedup LRU rebuild caps at 10k, 7-day cleanup on insert, INSERT OR REPLACE on duplicate source+event_id, v5 migration idempotent, synthetic storm (50 events) -> restart -> replay (5 incl 1 duplicate) with the duplicate dedup-rejected and counters NOT resetting - tests/conftest.py (new): autouse MESHAI_DB_PATH redirection to per-test tmp file, so the dispatcher_* tables on production /data dont get polluted by tests that construct Dispatcher() without an explicit fixture - tests/test_notification_toggles.py: _dispatch helper wipes dedup/cooldown/ state tables between calls (per-call independence preserved; pre-v0.6-2 in-memory-only Dispatcher reset naturally per instance) Test count: 680 -> 697 (+17 new, 0 regressions). Refs audit doc v0.6-phase1-audit.md finding #1.
2026-06-05 16:35:40 +00:00
"""Pytest fixture isolation for meshai persistence (v0.6-2).
Before v0.6-2 the dispatcher held all state in instance memory, so tests
that constructed `Dispatcher(...)` were inert w.r.t. SQLite. v0.6-2 made
`Dispatcher.__init__` read/restore from the persistence layer, which by
default points at `/data/meshai.sqlite`. Without isolation every test
would now read+write production state, polluting across tests and across
pytest invocations.
This autouse fixture redirects `MESHAI_DB_PATH` to a per-test tmp file
and clears the persistence-layer threading.local caches around each test.
Existing tests that don't reference any fixture get isolation for free;
tests that explicitly use a `db_path` (or similar) fixture can still
override the env var inside their own fixture body -- last setenv wins.
"""
import pytest
from meshai.persistence import close_thread_connection
from meshai.persistence import db as _persistence_db
@pytest.fixture(autouse=True)
def _isolate_meshai_db(tmp_path, monkeypatch):
feat(v0.6-3a): adapter_config foundation -- migration + defaults registry + typed accessor Closes the foundation slice of audit doc Section A (Rule 17). Lands two SQLite tables, the seed routine that populates them from a Python defaults registry, and a typed accessor that handler code will read in v0.6-3b. No handler changes in this commit -- ZERO behavior risk, every existing test still passes (721 / 69 skipped / 0 failed). v6.sql tables: - adapter_config(adapter, key, value_json, default_json, type, description, updated_at) PRIMARY KEY(adapter, key) -- JSON-encoded values flow through a single column uniformly. CHECK constraint on `type` closes the vocab (int/float/str/ bool/json). - adapter_meta(adapter PK, display_name, include_in_llm_context, description, updated_at) -- per-adapter metadata + the user-scopable LLM-context toggle (Matt refinement #5). meshai/adapter_config/ package: - defaults.py: REGISTRY dict mapping (adapter, key) -> {default, type, description}. Covers audit doc sections A.1-A.12: wfigs, nws, usgs_quake, swpc, usgs_nwis, incident family (tomtom_incidents, state_511_atis, itd_511, shared "incident"), central consumer, dispatcher, band_conditions, geocoder, firms, pipeline (Inhibitor + Grouper). ~85 keys total. ADAPTER_META covers 15 adapters with display_name + include_in_llm_context defaulting to True. Per Matt refinement #3, every default matches the current handler constant EXACTLY -- first deploy behavior is unchanged. - _accessor.py: AdapterConfig class with `adapter_config.<adapter>.<key>` syntax. Read pipeline: in-memory cache hit -> SQL -> registry fallback (with WARNING) -> AttributeError. Process-wide cache; PUT via v0.6-3c REST API calls invalidate_cache() to drop the cache. GIL-atomic dict reads on the fast path (handlers call this hot). - __init__.py: seed_defaults(conn) -- INSERT OR IGNOREs one row per registry entry. Idempotent, never overwrites user edits. Wiring: - meshai/persistence/db.py: SCHEMA_VERSION 5 -> 6, and init_db() now calls seed_defaults() after migrations apply. - meshai/main.py: _init_components() now calls init_db() FIRST (per commit #1 lessons-learned: a startup-time migration is required when handlers will rely on the new schema; lazy-on-first-handler is fine for v4/v5 but not for v6 where handler reads start in v0.6-3b). - tests/conftest.py: autouse fixture now calls init_db() + clears the accessor cache around each test, so every test gets the v6 seed AND a clean cache without per-test boilerplate. Tests (tests/test_adapter_config_foundation.py, 24 cases): - v6 tables exist + schema_meta at 6 + type-vocabulary CHECK enforced - seed populates every REGISTRY + ADAPTER_META row, value_json == default_json on first seed, type matches - seed is idempotent + does not overwrite user edits - accessor returns correctly typed values for int/float/str/bool/ json list/json dict/json None - cache hit: second read does not touch the DB (patched _load_from_db raises, accessor still succeeds) - invalidate_cache forces a re-read; mutated DB value wins - registry fallback path triggers when a row is missing (with WARNING) - unknown key raises AttributeError - setattr blocked (writes go via the REST API in 3c) - every default JSON round-trips cleanly; every type is in vocabulary - ADAPTER_META covers every adapter in REGISTRY Test count: 697 -> 721 (+24 new, 0 regressions). v0.6-3b will wire handlers one at a time (wfigs, nws, quake, swpc, nwis, incident, central, dispatcher, band_conditions, geocoder, firms). Per the audit lock, defaults match exactly so each wiring step is a pure refactor -- bisect-safe. v0.6-3c lands the /api/adapter-config CRUD + the AdapterConfig.tsx dashboard editor + cache invalidation on PUT. Refs audit doc v0.6-phase1-audit.md Section A + finding #4.
2026-06-05 17:06:51 +00:00
"""Point MESHAI_DB_PATH at a tmp file per test + run init_db so the
v6 adapter_config seed lands before any test code runs.
v0.6-3a: init_db() applies pending migrations AND seeds
adapter_config from the defaults registry. Tests that read via
need the seed in place.
"""
feat(v0.6-2): dispatcher state persistence -- cold-start, cooldowns, dedup LRU to SQLite Closes Rule-20 dispatcher gap from audit doc v0.6-phase1-audit.md finding #1. Pre-this-commit the cold-start anchor, 4 drop counters, per-toggle cooldown map, and dedup OrderedDict all lived in Dispatcher instance memory and were lost on every container restart. v5.sql adds three tables: - dispatcher_state (singleton id=1): cold_start_anchor + 4 drop counters - dispatcher_cooldowns ((toggle,category,region) keyed): last_fired_at - dispatcher_dedup ((source,event_id) keyed): seen_at Dispatcher refactor: - __init__ calls _restore_from_db -- counters, cold-start anchor, cooldown map, and dedup LRU (most-recent 10k by seen_at) all rehydrated from the three new tables - write-through on every mutation: _persist_state for counter/anchor, _persist_cooldown for cooldown UPSERT + 2*cooldown_s prune, _persist_dedup for dedup INSERT OR REPLACE + 7-day cleanup - in-memory caches stay authoritative on the fast read path - cumulative-since-install counters (NOT since-boot); LLM will be able to answer "we have dropped 47 stale events this week" after commit #5 (env_reporter) lands - graceful degrade: missing v5 tables / persistence outage falls back to fresh in-memory state without crashing the constructor Tests: - tests/test_dispatcher_persistence.py (17 tests): state restore on init, counter+cooldown+dedup survival across simulated restart, cooldown rearm within 2x window, dedup LRU rebuild caps at 10k, 7-day cleanup on insert, INSERT OR REPLACE on duplicate source+event_id, v5 migration idempotent, synthetic storm (50 events) -> restart -> replay (5 incl 1 duplicate) with the duplicate dedup-rejected and counters NOT resetting - tests/conftest.py (new): autouse MESHAI_DB_PATH redirection to per-test tmp file, so the dispatcher_* tables on production /data dont get polluted by tests that construct Dispatcher() without an explicit fixture - tests/test_notification_toggles.py: _dispatch helper wipes dedup/cooldown/ state tables between calls (per-call independence preserved; pre-v0.6-2 in-memory-only Dispatcher reset naturally per instance) Test count: 680 -> 697 (+17 new, 0 regressions). Refs audit doc v0.6-phase1-audit.md finding #1.
2026-06-05 16:35:40 +00:00
p = str(tmp_path / "meshai-test-isolated.sqlite")
monkeypatch.setenv("MESHAI_DB_PATH", p)
_persistence_db._initialised.clear()
close_thread_connection()
feat(v0.6-3a): adapter_config foundation -- migration + defaults registry + typed accessor Closes the foundation slice of audit doc Section A (Rule 17). Lands two SQLite tables, the seed routine that populates them from a Python defaults registry, and a typed accessor that handler code will read in v0.6-3b. No handler changes in this commit -- ZERO behavior risk, every existing test still passes (721 / 69 skipped / 0 failed). v6.sql tables: - adapter_config(adapter, key, value_json, default_json, type, description, updated_at) PRIMARY KEY(adapter, key) -- JSON-encoded values flow through a single column uniformly. CHECK constraint on `type` closes the vocab (int/float/str/ bool/json). - adapter_meta(adapter PK, display_name, include_in_llm_context, description, updated_at) -- per-adapter metadata + the user-scopable LLM-context toggle (Matt refinement #5). meshai/adapter_config/ package: - defaults.py: REGISTRY dict mapping (adapter, key) -> {default, type, description}. Covers audit doc sections A.1-A.12: wfigs, nws, usgs_quake, swpc, usgs_nwis, incident family (tomtom_incidents, state_511_atis, itd_511, shared "incident"), central consumer, dispatcher, band_conditions, geocoder, firms, pipeline (Inhibitor + Grouper). ~85 keys total. ADAPTER_META covers 15 adapters with display_name + include_in_llm_context defaulting to True. Per Matt refinement #3, every default matches the current handler constant EXACTLY -- first deploy behavior is unchanged. - _accessor.py: AdapterConfig class with `adapter_config.<adapter>.<key>` syntax. Read pipeline: in-memory cache hit -> SQL -> registry fallback (with WARNING) -> AttributeError. Process-wide cache; PUT via v0.6-3c REST API calls invalidate_cache() to drop the cache. GIL-atomic dict reads on the fast path (handlers call this hot). - __init__.py: seed_defaults(conn) -- INSERT OR IGNOREs one row per registry entry. Idempotent, never overwrites user edits. Wiring: - meshai/persistence/db.py: SCHEMA_VERSION 5 -> 6, and init_db() now calls seed_defaults() after migrations apply. - meshai/main.py: _init_components() now calls init_db() FIRST (per commit #1 lessons-learned: a startup-time migration is required when handlers will rely on the new schema; lazy-on-first-handler is fine for v4/v5 but not for v6 where handler reads start in v0.6-3b). - tests/conftest.py: autouse fixture now calls init_db() + clears the accessor cache around each test, so every test gets the v6 seed AND a clean cache without per-test boilerplate. Tests (tests/test_adapter_config_foundation.py, 24 cases): - v6 tables exist + schema_meta at 6 + type-vocabulary CHECK enforced - seed populates every REGISTRY + ADAPTER_META row, value_json == default_json on first seed, type matches - seed is idempotent + does not overwrite user edits - accessor returns correctly typed values for int/float/str/bool/ json list/json dict/json None - cache hit: second read does not touch the DB (patched _load_from_db raises, accessor still succeeds) - invalidate_cache forces a re-read; mutated DB value wins - registry fallback path triggers when a row is missing (with WARNING) - unknown key raises AttributeError - setattr blocked (writes go via the REST API in 3c) - every default JSON round-trips cleanly; every type is in vocabulary - ADAPTER_META covers every adapter in REGISTRY Test count: 697 -> 721 (+24 new, 0 regressions). v0.6-3b will wire handlers one at a time (wfigs, nws, quake, swpc, nwis, incident, central, dispatcher, band_conditions, geocoder, firms). Per the audit lock, defaults match exactly so each wiring step is a pure refactor -- bisect-safe. v0.6-3c lands the /api/adapter-config CRUD + the AdapterConfig.tsx dashboard editor + cache invalidation on PUT. Refs audit doc v0.6-phase1-audit.md Section A + finding #4.
2026-06-05 17:06:51 +00:00
# Clear the adapter_config accessor cache so the prior tests
# in-memory cache does not leak into this test.
try:
from meshai.adapter_config import invalidate_cache
invalidate_cache()
except Exception:
pass
# Eager init: applies v1..v6 migrations + seeds adapter_config.
from meshai.persistence import init_db
init_db()
feat(v0.6-2): dispatcher state persistence -- cold-start, cooldowns, dedup LRU to SQLite Closes Rule-20 dispatcher gap from audit doc v0.6-phase1-audit.md finding #1. Pre-this-commit the cold-start anchor, 4 drop counters, per-toggle cooldown map, and dedup OrderedDict all lived in Dispatcher instance memory and were lost on every container restart. v5.sql adds three tables: - dispatcher_state (singleton id=1): cold_start_anchor + 4 drop counters - dispatcher_cooldowns ((toggle,category,region) keyed): last_fired_at - dispatcher_dedup ((source,event_id) keyed): seen_at Dispatcher refactor: - __init__ calls _restore_from_db -- counters, cold-start anchor, cooldown map, and dedup LRU (most-recent 10k by seen_at) all rehydrated from the three new tables - write-through on every mutation: _persist_state for counter/anchor, _persist_cooldown for cooldown UPSERT + 2*cooldown_s prune, _persist_dedup for dedup INSERT OR REPLACE + 7-day cleanup - in-memory caches stay authoritative on the fast read path - cumulative-since-install counters (NOT since-boot); LLM will be able to answer "we have dropped 47 stale events this week" after commit #5 (env_reporter) lands - graceful degrade: missing v5 tables / persistence outage falls back to fresh in-memory state without crashing the constructor Tests: - tests/test_dispatcher_persistence.py (17 tests): state restore on init, counter+cooldown+dedup survival across simulated restart, cooldown rearm within 2x window, dedup LRU rebuild caps at 10k, 7-day cleanup on insert, INSERT OR REPLACE on duplicate source+event_id, v5 migration idempotent, synthetic storm (50 events) -> restart -> replay (5 incl 1 duplicate) with the duplicate dedup-rejected and counters NOT resetting - tests/conftest.py (new): autouse MESHAI_DB_PATH redirection to per-test tmp file, so the dispatcher_* tables on production /data dont get polluted by tests that construct Dispatcher() without an explicit fixture - tests/test_notification_toggles.py: _dispatch helper wipes dedup/cooldown/ state tables between calls (per-call independence preserved; pre-v0.6-2 in-memory-only Dispatcher reset naturally per instance) Test count: 680 -> 697 (+17 new, 0 regressions). Refs audit doc v0.6-phase1-audit.md finding #1.
2026-06-05 16:35:40 +00:00
yield p
close_thread_connection()
_persistence_db._initialised.discard(p)
feat(v0.6-3a): adapter_config foundation -- migration + defaults registry + typed accessor Closes the foundation slice of audit doc Section A (Rule 17). Lands two SQLite tables, the seed routine that populates them from a Python defaults registry, and a typed accessor that handler code will read in v0.6-3b. No handler changes in this commit -- ZERO behavior risk, every existing test still passes (721 / 69 skipped / 0 failed). v6.sql tables: - adapter_config(adapter, key, value_json, default_json, type, description, updated_at) PRIMARY KEY(adapter, key) -- JSON-encoded values flow through a single column uniformly. CHECK constraint on `type` closes the vocab (int/float/str/ bool/json). - adapter_meta(adapter PK, display_name, include_in_llm_context, description, updated_at) -- per-adapter metadata + the user-scopable LLM-context toggle (Matt refinement #5). meshai/adapter_config/ package: - defaults.py: REGISTRY dict mapping (adapter, key) -> {default, type, description}. Covers audit doc sections A.1-A.12: wfigs, nws, usgs_quake, swpc, usgs_nwis, incident family (tomtom_incidents, state_511_atis, itd_511, shared "incident"), central consumer, dispatcher, band_conditions, geocoder, firms, pipeline (Inhibitor + Grouper). ~85 keys total. ADAPTER_META covers 15 adapters with display_name + include_in_llm_context defaulting to True. Per Matt refinement #3, every default matches the current handler constant EXACTLY -- first deploy behavior is unchanged. - _accessor.py: AdapterConfig class with `adapter_config.<adapter>.<key>` syntax. Read pipeline: in-memory cache hit -> SQL -> registry fallback (with WARNING) -> AttributeError. Process-wide cache; PUT via v0.6-3c REST API calls invalidate_cache() to drop the cache. GIL-atomic dict reads on the fast path (handlers call this hot). - __init__.py: seed_defaults(conn) -- INSERT OR IGNOREs one row per registry entry. Idempotent, never overwrites user edits. Wiring: - meshai/persistence/db.py: SCHEMA_VERSION 5 -> 6, and init_db() now calls seed_defaults() after migrations apply. - meshai/main.py: _init_components() now calls init_db() FIRST (per commit #1 lessons-learned: a startup-time migration is required when handlers will rely on the new schema; lazy-on-first-handler is fine for v4/v5 but not for v6 where handler reads start in v0.6-3b). - tests/conftest.py: autouse fixture now calls init_db() + clears the accessor cache around each test, so every test gets the v6 seed AND a clean cache without per-test boilerplate. Tests (tests/test_adapter_config_foundation.py, 24 cases): - v6 tables exist + schema_meta at 6 + type-vocabulary CHECK enforced - seed populates every REGISTRY + ADAPTER_META row, value_json == default_json on first seed, type matches - seed is idempotent + does not overwrite user edits - accessor returns correctly typed values for int/float/str/bool/ json list/json dict/json None - cache hit: second read does not touch the DB (patched _load_from_db raises, accessor still succeeds) - invalidate_cache forces a re-read; mutated DB value wins - registry fallback path triggers when a row is missing (with WARNING) - unknown key raises AttributeError - setattr blocked (writes go via the REST API in 3c) - every default JSON round-trips cleanly; every type is in vocabulary - ADAPTER_META covers every adapter in REGISTRY Test count: 697 -> 721 (+24 new, 0 regressions). v0.6-3b will wire handlers one at a time (wfigs, nws, quake, swpc, nwis, incident, central, dispatcher, band_conditions, geocoder, firms). Per the audit lock, defaults match exactly so each wiring step is a pure refactor -- bisect-safe. v0.6-3c lands the /api/adapter-config CRUD + the AdapterConfig.tsx dashboard editor + cache invalidation on PUT. Refs audit doc v0.6-phase1-audit.md Section A + finding #4.
2026-06-05 17:06:51 +00:00
try:
from meshai.adapter_config import invalidate_cache
invalidate_cache()
except Exception:
pass