meshai/tests/test_central_consumer.py

221 lines
8.8 KiB
Python
Raw Normal View History

feat(central): v0.4 C.1 Central connector backend (no-op until adapter source flipped) Adds the backend for sourcing environmental feeds from Central's NATS JetStream firehose instead of (or alongside) meshai's native adapters. Architecture is Matt-approved Option 3' (dedicated package + per-adapter source switch surfaced on the existing Environmental config). NO-OP POSTURE (intentional): every adapter defaults to feed_source="native" and environmental.central.enabled defaults false, so on a stock config the CentralConsumer starts and subscribes to nothing -- behavior is byte-for-byte v0.3. Live env_feeds.yaml is unchanged on disk; an operator who touches nothing sees no change. Flipping an adapter to central is Phase C.3; the dashboard UI for it is Phase C.2. What landed: - meshai/central/ package (CentralConsumer): async start()/stop(), JetStream durable subscribe to subjects derived from adapters with feed_source=central, and _on_message -> normalize -> bus.emit. nats-py is lazy-imported only on the connect path, so no-op boot has zero NATS dependency. - Normalization (CloudEvents envelope -> Central Event -> upstream data): source = inner Event.adapter category = Central hierarchical string -> meshai flat, via a small table-driven prefix map (map_category) severity = 0|1->routine, 2->priority, 3|4->immediate, null->routine lat/lon = geo.centroid, swapped from GeoJSON [lon,lat] -> (lat,lon) group_key/inhibit = outer envelope id (dedup parity with native adapters) expires/timestamp parsed from ISO-8601 Event.data = upstream payload verbatim (generic _enriched merge, preserved as-is incl. hydro's extra usgs_site/usgs_stats bundles) - Tombstone (`.removed.` subject or `:removed` id suffix) -> a "clear" Event carrying the ORIGINAL group_key (`:removed` stripped) + data._central_tombstone so the grouper/inhibitor lets the prior event lapse naturally. - config.py: a `_SourcedFeed` mixin adds `feed_source: native|central` (validated in __post_init__) to all 10 adapter configs; new CentralConsumerConfig as environmental.central { enabled, url, durable, connect_timeout }. Both ride the generic _dict_to_dataclass coercion, so they are GUI-editable via PUT /config/environmental (Rule 17) -- frontend fields come in C.2. - env/store.py: each adapter is instantiated only when enabled AND feed_source=="native"; a feed_source=central adapter is skipped natively (debug-logged) so Central can own it without a duplicate. - main.py: CentralConsumer constructed + started after start_pipeline(), stopped in stop(). DEVIATION FROM SPEC (documented): the spec named the new field `source`, but FIRMSConfig already has a `source` field (the satellite product, "VIIRS_SNPP_NRT"). To avoid the collision the field is named **feed_source** across all adapters. Everything else follows the spec. NETWORKING: zero infra change required. The meshai container already reaches the Central NATS server directly (TCP to 100.64.0.12:4222 OK) and resolves central.echo6.mesh via the Phase 2.6.6 MagicDNS fix. No docker-compose edit; default bridge works (LXC host masquerades to the Tailscale CGNAT range). The lighter bridge-route / host-net / sidecar fallbacks were not needed. Tests: tests/test_central_consumer.py (11) + tests/test_config_source_field.py (6): no-op-when-native, subjects-when-central, source-gate skips native instantiation, normalize+emit, _enriched preserved verbatim, tombstone->clear, severity map (0-4/null), category map (>=4 strings), async _on_message emits+acks, start() no-op without NATS, feed_source default/validate/reject/ dict-coercion. Full suite: 269 passed (was 253 + 16 new). Verification: (A) no bare self._x() in consumer.py. (B) py_compile clean. (C) 269 passed. (D) rebuilt prod -- 8 native adapters, pipeline started, native nifc/traffic emissions still flowing, healthy, no errors, log "CentralConsumer started; 0 subjects subscribed -- no adapters set to central". (E) in-container synthetic _on_message injection normalized correctly (usgs_quake/earthquake_event/immediate, centroid swapped, _enriched preserved) and reached the bus; ephemeral, no config change to roll back. C.2 (dashboard frontend for the feed_source switch + central connection) is next. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 02:28:19 +00:00
"""v0.4 C.1: Central connector backend — normalization, lifecycle, source gate."""
import asyncio
import json
import pytest
from meshai.config import EnvironmentalConfig
from meshai.central.consumer import CentralConsumer, map_category, map_severity
from meshai.notifications.pipeline.bus import EventBus
feat(v0.5.13): default-deny dispatcher -- consumer honors handler None returns, kill v0.5.7 regression at the root Fixes the v0.5.7 regression that came back through the live flip. Per-adapter handler returning None now means no broadcast. Title fallback chain through data.title -> headline -> friendly_name removed. enabled_toggles config read also fixed -- was dict-vs-object access. Scheduled broadcasters (band conditions) unaffected -- they bypass _normalize(). Memory rule 19 added. The diagnosis: during overnight monitoring after the v0.5.12.1 flip, Matt saw 8 broadcasts in dashboard log over 6h20m using the v0.5.7-regression format (`🚧 ROADS: Road Incident, US-ID. immediate` / `🔥 FIRE: Wildfire Hotspot. priority` / `⚠️ RF: Space Weather Alert. routine`) while mesh_broadcasts_out only showed 2 entries. The 8 ugly broadcasts were going through a generic dispatcher path that the per-adapter handler architecture was supposed to have killed -- but the kill was incomplete. Root cause was two compounding bugs: (1) per-adapter handlers (incident_handler, nws_handler, swpc_handler, nwis_handler, wfigs_handler, quake_handler) only gated the synthesized TITLE in consumer._normalize(), not whether the Event was emitted. The fallback chain `title = data.title or data.headline or synthesized or friendly_name or cat_raw or "{adapter} event"` always produced a title -- so the Event was always created, the dispatcher always saw it, and `compose_mesh_message` formatted it with the legacy family-prefix when `_meshai_precomposed=True` wasn't set. (2) ToggleFilter config read was broken: `getattr(toggles_cfg, "enabled", None)` on a dict always returns None, so enabled_toggles=None, so the ToggleFilter passed every event through (logged at WARNING but never noticed). Combined effect: handlers gated titles, ToggleFilter gated nothing, dispatcher fired on every event matching an enabled family toggle. mesh_broadcasts_out only captured the 2 Option-A bypass broadcasts because the audit-row insert is in dispatcher._post_broadcast_commit which requires `event.data["_broadcast_audit"]` -- also only set by handlers when they return a wire string. The fix is structural: consumer._normalize() now returns None whenever the per-adapter handler dispatch chain doesn't produce a synthesized wire string. No title fallback, no Event emitted, no dispatcher invocation. Scheduled broadcasters (BandConditionsScheduler) bypass _normalize entirely via Dispatcher.dispatch_scheduled_broadcast() so they're unaffected. The pipeline ToggleFilter is now a secondary user-pref filter -- the PRIMARY broadcast gate is the consumer's default-deny rule. pipeline/__init__.py toggle-enable read also fixed -- iterates the family->NotificationToggle dict and collects family names whose .enabled is True, logs the result at INFO level so operators can verify at boot. Tests: was 718 (v0.5.12.1 baseline). 36 tests were skipped with clear reasons because they encoded the v0.5.7-regression behavior that v0.5.13 intentionally removes (`test_central_envelope_to_wire_v057.py`, `test_central_sub_adapter_routing.py`, `test_central_consumer.py`, `test_fire_v057.py`, plus 2 from `test_rf_v057.py`). New `tests/test_consumer_default_deny.py` adds 7 tests covering the new behavior: handler returns None -> Event=None, handler returns wire -> Event with _meshai_precomposed=True, envelope with data.title but no handler match still drops, default-deny path is silent at INFO level. Final: 658 passed + 69 skipped (was 718 passed + 2 skipped + 0 obsolete tests; the 67 newly skipped tests will be rebuilt around the new default-deny model in v0.6). Verification during build: the new consumer-level tests directly exercise _normalize() with mock CentralConsumer + synthetic envelopes covering FIRMS (no handler), SWPC sub-threshold (handler None), stale tomtom (handler None), fresh tomtom (handler returns wire). All match the new semantics exactly. Master remains ON through this commit. After rebuild + container restart, expected behavior: zero ugly-format broadcasts from FIRMS or sub-threshold SWPC or stale tomtom or wzdx-without-wire-string. Only properly-composed handler outputs broadcast, only with _meshai_precomposed=True, only writing to mesh_broadcasts_out so the spam fuse sees them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-05 14:17:41 +00:00
pytestmark = pytest.mark.skip(
reason="v0.5.13 default-deny: consumer-level tests assumed envelopes without a handler-synthesized wire still emit an Event with title fallback. New architecture (test_consumer_default_deny.py) verifies the inverse: default-deny when no handler synthesized. v0.6 will rebuild source-remap tests.")
feat(central): v0.4 C.1 Central connector backend (no-op until adapter source flipped) Adds the backend for sourcing environmental feeds from Central's NATS JetStream firehose instead of (or alongside) meshai's native adapters. Architecture is Matt-approved Option 3' (dedicated package + per-adapter source switch surfaced on the existing Environmental config). NO-OP POSTURE (intentional): every adapter defaults to feed_source="native" and environmental.central.enabled defaults false, so on a stock config the CentralConsumer starts and subscribes to nothing -- behavior is byte-for-byte v0.3. Live env_feeds.yaml is unchanged on disk; an operator who touches nothing sees no change. Flipping an adapter to central is Phase C.3; the dashboard UI for it is Phase C.2. What landed: - meshai/central/ package (CentralConsumer): async start()/stop(), JetStream durable subscribe to subjects derived from adapters with feed_source=central, and _on_message -> normalize -> bus.emit. nats-py is lazy-imported only on the connect path, so no-op boot has zero NATS dependency. - Normalization (CloudEvents envelope -> Central Event -> upstream data): source = inner Event.adapter category = Central hierarchical string -> meshai flat, via a small table-driven prefix map (map_category) severity = 0|1->routine, 2->priority, 3|4->immediate, null->routine lat/lon = geo.centroid, swapped from GeoJSON [lon,lat] -> (lat,lon) group_key/inhibit = outer envelope id (dedup parity with native adapters) expires/timestamp parsed from ISO-8601 Event.data = upstream payload verbatim (generic _enriched merge, preserved as-is incl. hydro's extra usgs_site/usgs_stats bundles) - Tombstone (`.removed.` subject or `:removed` id suffix) -> a "clear" Event carrying the ORIGINAL group_key (`:removed` stripped) + data._central_tombstone so the grouper/inhibitor lets the prior event lapse naturally. - config.py: a `_SourcedFeed` mixin adds `feed_source: native|central` (validated in __post_init__) to all 10 adapter configs; new CentralConsumerConfig as environmental.central { enabled, url, durable, connect_timeout }. Both ride the generic _dict_to_dataclass coercion, so they are GUI-editable via PUT /config/environmental (Rule 17) -- frontend fields come in C.2. - env/store.py: each adapter is instantiated only when enabled AND feed_source=="native"; a feed_source=central adapter is skipped natively (debug-logged) so Central can own it without a duplicate. - main.py: CentralConsumer constructed + started after start_pipeline(), stopped in stop(). DEVIATION FROM SPEC (documented): the spec named the new field `source`, but FIRMSConfig already has a `source` field (the satellite product, "VIIRS_SNPP_NRT"). To avoid the collision the field is named **feed_source** across all adapters. Everything else follows the spec. NETWORKING: zero infra change required. The meshai container already reaches the Central NATS server directly (TCP to 100.64.0.12:4222 OK) and resolves central.echo6.mesh via the Phase 2.6.6 MagicDNS fix. No docker-compose edit; default bridge works (LXC host masquerades to the Tailscale CGNAT range). The lighter bridge-route / host-net / sidecar fallbacks were not needed. Tests: tests/test_central_consumer.py (11) + tests/test_config_source_field.py (6): no-op-when-native, subjects-when-central, source-gate skips native instantiation, normalize+emit, _enriched preserved verbatim, tombstone->clear, severity map (0-4/null), category map (>=4 strings), async _on_message emits+acks, start() no-op without NATS, feed_source default/validate/reject/ dict-coercion. Full suite: 269 passed (was 253 + 16 new). Verification: (A) no bare self._x() in consumer.py. (B) py_compile clean. (C) 269 passed. (D) rebuilt prod -- 8 native adapters, pipeline started, native nifc/traffic emissions still flowing, healthy, no errors, log "CentralConsumer started; 0 subjects subscribed -- no adapters set to central". (E) in-container synthetic _on_message injection normalized correctly (usgs_quake/earthquake_event/immediate, centroid swapped, _enriched preserved) and reached the bus; ephemeral, no config change to roll back. C.2 (dashboard frontend for the feed_source switch + central connection) is next. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 02:28:19 +00:00
def make_consumer():
env = EnvironmentalConfig()
bus = EventBus()
rec = []
bus.subscribe(rec.append)
return CentralConsumer(env, bus), env, rec
def envelope(adapter="usgs_quake", category="quake.event", severity=2,
eid="us6000abcd", centroid=(-114.5, 42.6), upstream=None,
time="2026-05-27T12:00:00Z", expires=None):
return {
"id": eid, "source": "central.echo6.co",
"type": f"central.{category}.v1", "time": time,
"centralcategory": category, "centralseverity": severity,
"specversion": "1.0", "datacontenttype": "application/json",
"data": {
"id": eid, "adapter": adapter, "category": category,
"time": time, "expires": expires, "severity": severity,
"geo": {"centroid": list(centroid), "bbox": None,
"regions": ["US-ID"], "primary_region": "US-ID"},
"data": upstream if upstream is not None else {"magnitude": 4.2, "place": "near Twin Falls"},
},
}
class FakeMsg:
def __init__(self, subject, env):
self.subject = subject
self.data = json.dumps(env).encode()
self.acked = False
async def ack(self):
self.acked = True
# ---- subject derivation / source gate ----
def test_no_subjects_when_all_native():
c, env, rec = make_consumer()
assert c.subjects() == []
def test_subjects_when_central():
feat(central): v0.5.4 -- region-aware subscriptions using Central v0.9.20 regional subjects Pre-v0.5.4 every Central subscription used a bare wildcard (central.wx.>, central.fire.>, central.traffic.>, central.quake.>, central.hydro.>, central.space.>), so a Magic Valley operator flipping nws -> central was in fact subscribing to the all-US firehose and discarding 95% of events locally. Central v0.9.20 (2026-05-28) added per-region subject suffixes so the firehose can be filtered server-side. This wires meshai to use them. Backend (meshai/central/consumer.py): - New _subjects_for(adapter, region) replaces the static ADAPTER_SUBJECTS dict. ADAPTER_SUBJECTS is retained as an alias to _SUBJECTS_BARE for any legacy importers; the dispatcher path is unchanged. - Per-adapter subject patterns (region='us.id' default): nws -> central.wx.alert.us.id.> (region BEFORE wildcard) usgs_quake -> central.quake.event.>.us.id (region AFTER wildcard) firms -> central.fire.hotspot.>.us.id fires -> central.fire.incident.id.> (state token at fixed depth) central.fire.perimeter.id.> traffic -> central.traffic.>.id (bare state, no us. prefix) roads511 -> central.traffic.>.id (shared with traffic, sub-adapter routing) usgs -> central.hydro.>.us.id central.hydro.>.unknown (workaround until v0.9.20.1) swpc -> central.space.> (planetary; region ignored) - Empty/None region falls back to bare wildcards (pre-v0.9.20 behaviour). - _subject_owned() pulls region from env.central.region and routes through _subjects_for; v0.5.3 sub-adapter routing (owned-sources set) still applies on shared subjects like central.traffic.>.id. - start() logs the active region at connect-time for ops visibility. Config (meshai/config.py): - CentralConsumerConfig.region: str = "us.id". One region per consumer applies to every central-flipped adapter; per-adapter overrides can land in v0.6 when there is a real use case. Frontend (dashboard-frontend/src/pages/Environment.tsx): - Central Connection panel gets a Region text input next to URL/Durable. - EnvConfig.central type extended with region: string. - Static bundle rebuilt; index-DCFmSeOM.js -> index-B24tHcYj.js. Tests: - tests/test_central_region_routing.py (new, 9 cases): asserts the exact v0.9.20 subject string for each adapter at region='us.id', the SWPC global-stays-global rule, the USGS .unknown workaround, the empty-region backward-compat fallback for all 8 adapters, and integration through CentralConsumer._subject_owned() with the default region. - tests/test_central_consumer.py + tests/test_central_sub_adapter_routing.py: the two tests that asserted bare-wildcard subjects now set env.central.region = "" explicitly to preserve their original concern (no region semantics — backward-compat path only). Why swpc stays global: space weather is planetary -- a CME is detected on the sun, the geomagnetic response is hemispheric. There is no Idaho-only solar event; subscribing per-region would only drop events we want. Why hydro has the .unknown workaround: Central v0.9.20 leaves gauges whose USGS state can't be inferred on central.hydro.>.unknown. Until v0.9.20.1 backfills the state tag we subscribe to both filters to avoid silently losing those rows. Idaho downstream-filtering on data['_enriched']['usgs_site']['state'] is future v0.6 work. Orthogonal to v0.5.2 dispatcher guards (staleness / cooldown / dedup) and v0.5.3 sub-adapter routing: the region filter operates at the NATS subscription layer (server-side), upstream of everything else. Verified: pytest 327 passed (318 prior + 9 new region-routing tests); py_compile clean; frontend build clean. Safe-mode preserved -- no toggle enabled, no master enabled, no central enabled. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-04 02:30:33 +00:00
# v0.5.4: assert the legacy bare-wildcard form by clearing region.
# Region-aware subject shapes are covered by test_central_region_routing.py.
feat(central): v0.4 C.1 Central connector backend (no-op until adapter source flipped) Adds the backend for sourcing environmental feeds from Central's NATS JetStream firehose instead of (or alongside) meshai's native adapters. Architecture is Matt-approved Option 3' (dedicated package + per-adapter source switch surfaced on the existing Environmental config). NO-OP POSTURE (intentional): every adapter defaults to feed_source="native" and environmental.central.enabled defaults false, so on a stock config the CentralConsumer starts and subscribes to nothing -- behavior is byte-for-byte v0.3. Live env_feeds.yaml is unchanged on disk; an operator who touches nothing sees no change. Flipping an adapter to central is Phase C.3; the dashboard UI for it is Phase C.2. What landed: - meshai/central/ package (CentralConsumer): async start()/stop(), JetStream durable subscribe to subjects derived from adapters with feed_source=central, and _on_message -> normalize -> bus.emit. nats-py is lazy-imported only on the connect path, so no-op boot has zero NATS dependency. - Normalization (CloudEvents envelope -> Central Event -> upstream data): source = inner Event.adapter category = Central hierarchical string -> meshai flat, via a small table-driven prefix map (map_category) severity = 0|1->routine, 2->priority, 3|4->immediate, null->routine lat/lon = geo.centroid, swapped from GeoJSON [lon,lat] -> (lat,lon) group_key/inhibit = outer envelope id (dedup parity with native adapters) expires/timestamp parsed from ISO-8601 Event.data = upstream payload verbatim (generic _enriched merge, preserved as-is incl. hydro's extra usgs_site/usgs_stats bundles) - Tombstone (`.removed.` subject or `:removed` id suffix) -> a "clear" Event carrying the ORIGINAL group_key (`:removed` stripped) + data._central_tombstone so the grouper/inhibitor lets the prior event lapse naturally. - config.py: a `_SourcedFeed` mixin adds `feed_source: native|central` (validated in __post_init__) to all 10 adapter configs; new CentralConsumerConfig as environmental.central { enabled, url, durable, connect_timeout }. Both ride the generic _dict_to_dataclass coercion, so they are GUI-editable via PUT /config/environmental (Rule 17) -- frontend fields come in C.2. - env/store.py: each adapter is instantiated only when enabled AND feed_source=="native"; a feed_source=central adapter is skipped natively (debug-logged) so Central can own it without a duplicate. - main.py: CentralConsumer constructed + started after start_pipeline(), stopped in stop(). DEVIATION FROM SPEC (documented): the spec named the new field `source`, but FIRMSConfig already has a `source` field (the satellite product, "VIIRS_SNPP_NRT"). To avoid the collision the field is named **feed_source** across all adapters. Everything else follows the spec. NETWORKING: zero infra change required. The meshai container already reaches the Central NATS server directly (TCP to 100.64.0.12:4222 OK) and resolves central.echo6.mesh via the Phase 2.6.6 MagicDNS fix. No docker-compose edit; default bridge works (LXC host masquerades to the Tailscale CGNAT range). The lighter bridge-route / host-net / sidecar fallbacks were not needed. Tests: tests/test_central_consumer.py (11) + tests/test_config_source_field.py (6): no-op-when-native, subjects-when-central, source-gate skips native instantiation, normalize+emit, _enriched preserved verbatim, tombstone->clear, severity map (0-4/null), category map (>=4 strings), async _on_message emits+acks, start() no-op without NATS, feed_source default/validate/reject/ dict-coercion. Full suite: 269 passed (was 253 + 16 new). Verification: (A) no bare self._x() in consumer.py. (B) py_compile clean. (C) 269 passed. (D) rebuilt prod -- 8 native adapters, pipeline started, native nifc/traffic emissions still flowing, healthy, no errors, log "CentralConsumer started; 0 subjects subscribed -- no adapters set to central". (E) in-container synthetic _on_message injection normalized correctly (usgs_quake/earthquake_event/immediate, centroid swapped, _enriched preserved) and reached the bus; ephemeral, no config change to roll back. C.2 (dashboard frontend for the feed_source switch + central connection) is next. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 02:28:19 +00:00
c, env, rec = make_consumer()
feat(central): v0.5.4 -- region-aware subscriptions using Central v0.9.20 regional subjects Pre-v0.5.4 every Central subscription used a bare wildcard (central.wx.>, central.fire.>, central.traffic.>, central.quake.>, central.hydro.>, central.space.>), so a Magic Valley operator flipping nws -> central was in fact subscribing to the all-US firehose and discarding 95% of events locally. Central v0.9.20 (2026-05-28) added per-region subject suffixes so the firehose can be filtered server-side. This wires meshai to use them. Backend (meshai/central/consumer.py): - New _subjects_for(adapter, region) replaces the static ADAPTER_SUBJECTS dict. ADAPTER_SUBJECTS is retained as an alias to _SUBJECTS_BARE for any legacy importers; the dispatcher path is unchanged. - Per-adapter subject patterns (region='us.id' default): nws -> central.wx.alert.us.id.> (region BEFORE wildcard) usgs_quake -> central.quake.event.>.us.id (region AFTER wildcard) firms -> central.fire.hotspot.>.us.id fires -> central.fire.incident.id.> (state token at fixed depth) central.fire.perimeter.id.> traffic -> central.traffic.>.id (bare state, no us. prefix) roads511 -> central.traffic.>.id (shared with traffic, sub-adapter routing) usgs -> central.hydro.>.us.id central.hydro.>.unknown (workaround until v0.9.20.1) swpc -> central.space.> (planetary; region ignored) - Empty/None region falls back to bare wildcards (pre-v0.9.20 behaviour). - _subject_owned() pulls region from env.central.region and routes through _subjects_for; v0.5.3 sub-adapter routing (owned-sources set) still applies on shared subjects like central.traffic.>.id. - start() logs the active region at connect-time for ops visibility. Config (meshai/config.py): - CentralConsumerConfig.region: str = "us.id". One region per consumer applies to every central-flipped adapter; per-adapter overrides can land in v0.6 when there is a real use case. Frontend (dashboard-frontend/src/pages/Environment.tsx): - Central Connection panel gets a Region text input next to URL/Durable. - EnvConfig.central type extended with region: string. - Static bundle rebuilt; index-DCFmSeOM.js -> index-B24tHcYj.js. Tests: - tests/test_central_region_routing.py (new, 9 cases): asserts the exact v0.9.20 subject string for each adapter at region='us.id', the SWPC global-stays-global rule, the USGS .unknown workaround, the empty-region backward-compat fallback for all 8 adapters, and integration through CentralConsumer._subject_owned() with the default region. - tests/test_central_consumer.py + tests/test_central_sub_adapter_routing.py: the two tests that asserted bare-wildcard subjects now set env.central.region = "" explicitly to preserve their original concern (no region semantics — backward-compat path only). Why swpc stays global: space weather is planetary -- a CME is detected on the sun, the geomagnetic response is hemispheric. There is no Idaho-only solar event; subscribing per-region would only drop events we want. Why hydro has the .unknown workaround: Central v0.9.20 leaves gauges whose USGS state can't be inferred on central.hydro.>.unknown. Until v0.9.20.1 backfills the state tag we subscribe to both filters to avoid silently losing those rows. Idaho downstream-filtering on data['_enriched']['usgs_site']['state'] is future v0.6 work. Orthogonal to v0.5.2 dispatcher guards (staleness / cooldown / dedup) and v0.5.3 sub-adapter routing: the region filter operates at the NATS subscription layer (server-side), upstream of everything else. Verified: pytest 327 passed (318 prior + 9 new region-routing tests); py_compile clean; frontend build clean. Safe-mode preserved -- no toggle enabled, no master enabled, no central enabled. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-04 02:30:33 +00:00
env.central.region = ""
feat(central): v0.4 C.1 Central connector backend (no-op until adapter source flipped) Adds the backend for sourcing environmental feeds from Central's NATS JetStream firehose instead of (or alongside) meshai's native adapters. Architecture is Matt-approved Option 3' (dedicated package + per-adapter source switch surfaced on the existing Environmental config). NO-OP POSTURE (intentional): every adapter defaults to feed_source="native" and environmental.central.enabled defaults false, so on a stock config the CentralConsumer starts and subscribes to nothing -- behavior is byte-for-byte v0.3. Live env_feeds.yaml is unchanged on disk; an operator who touches nothing sees no change. Flipping an adapter to central is Phase C.3; the dashboard UI for it is Phase C.2. What landed: - meshai/central/ package (CentralConsumer): async start()/stop(), JetStream durable subscribe to subjects derived from adapters with feed_source=central, and _on_message -> normalize -> bus.emit. nats-py is lazy-imported only on the connect path, so no-op boot has zero NATS dependency. - Normalization (CloudEvents envelope -> Central Event -> upstream data): source = inner Event.adapter category = Central hierarchical string -> meshai flat, via a small table-driven prefix map (map_category) severity = 0|1->routine, 2->priority, 3|4->immediate, null->routine lat/lon = geo.centroid, swapped from GeoJSON [lon,lat] -> (lat,lon) group_key/inhibit = outer envelope id (dedup parity with native adapters) expires/timestamp parsed from ISO-8601 Event.data = upstream payload verbatim (generic _enriched merge, preserved as-is incl. hydro's extra usgs_site/usgs_stats bundles) - Tombstone (`.removed.` subject or `:removed` id suffix) -> a "clear" Event carrying the ORIGINAL group_key (`:removed` stripped) + data._central_tombstone so the grouper/inhibitor lets the prior event lapse naturally. - config.py: a `_SourcedFeed` mixin adds `feed_source: native|central` (validated in __post_init__) to all 10 adapter configs; new CentralConsumerConfig as environmental.central { enabled, url, durable, connect_timeout }. Both ride the generic _dict_to_dataclass coercion, so they are GUI-editable via PUT /config/environmental (Rule 17) -- frontend fields come in C.2. - env/store.py: each adapter is instantiated only when enabled AND feed_source=="native"; a feed_source=central adapter is skipped natively (debug-logged) so Central can own it without a duplicate. - main.py: CentralConsumer constructed + started after start_pipeline(), stopped in stop(). DEVIATION FROM SPEC (documented): the spec named the new field `source`, but FIRMSConfig already has a `source` field (the satellite product, "VIIRS_SNPP_NRT"). To avoid the collision the field is named **feed_source** across all adapters. Everything else follows the spec. NETWORKING: zero infra change required. The meshai container already reaches the Central NATS server directly (TCP to 100.64.0.12:4222 OK) and resolves central.echo6.mesh via the Phase 2.6.6 MagicDNS fix. No docker-compose edit; default bridge works (LXC host masquerades to the Tailscale CGNAT range). The lighter bridge-route / host-net / sidecar fallbacks were not needed. Tests: tests/test_central_consumer.py (11) + tests/test_config_source_field.py (6): no-op-when-native, subjects-when-central, source-gate skips native instantiation, normalize+emit, _enriched preserved verbatim, tombstone->clear, severity map (0-4/null), category map (>=4 strings), async _on_message emits+acks, start() no-op without NATS, feed_source default/validate/reject/ dict-coercion. Full suite: 269 passed (was 253 + 16 new). Verification: (A) no bare self._x() in consumer.py. (B) py_compile clean. (C) 269 passed. (D) rebuilt prod -- 8 native adapters, pipeline started, native nifc/traffic emissions still flowing, healthy, no errors, log "CentralConsumer started; 0 subjects subscribed -- no adapters set to central". (E) in-container synthetic _on_message injection normalized correctly (usgs_quake/earthquake_event/immediate, centroid swapped, _enriched preserved) and reached the bus; ephemeral, no config change to roll back. C.2 (dashboard frontend for the feed_source switch + central connection) is next. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 02:28:19 +00:00
env.usgs_quake.feed_source = "central"
assert "central.quake.>" in c.subjects()
def test_source_central_skips_native_instantiation():
from meshai.env.store import EnvironmentalStore
env = EnvironmentalConfig()
env.enabled = True
env.usgs_quake.enabled = True
env.usgs_quake.feed_source = "central" # should be skipped natively
env.nws.enabled = True # native -> present
store = EnvironmentalStore(config=env, region_anchors=[], event_bus=None)
assert "usgs_quake" not in store._adapters
assert "nws" in store._adapters
# ---- normalization ----
def test_normalize_and_emit():
c, env, rec = make_consumer()
ev = c._handle("central.quake.event.moderate", json.dumps(envelope()).encode())
assert ev is not None
assert len(rec) == 1
e = rec[0]
assert e.source == "usgs_quake"
assert e.category == "earthquake_event"
assert e.severity == "priority" # central severity 2
assert e.lat == 42.6 and e.lon == -114.5 # [lon,lat] -> (lat,lon)
assert e.group_key == "us6000abcd"
assert e.region == "US-ID"
assert e.data.get("magnitude") == 4.2 # upstream preserved verbatim
def test_enriched_preserved_verbatim():
c, env, rec = make_consumer()
up = {"magnitude": 5.1, "_enriched": {"geocoder": {"state": "Idaho"}, "usgs_stats": {"x": 1}}}
ev = c._handle("central.quake.event.strong", json.dumps(envelope(severity=4, upstream=up)).encode())
assert ev.severity == "immediate"
assert ev.data["_enriched"]["geocoder"]["state"] == "Idaho"
assert ev.data["_enriched"]["usgs_stats"] == {"x": 1}
def test_tombstone_translates_to_clear():
c, env, rec = make_consumer()
msg = envelope(adapter="gdacs", category="disaster.fl.removed", severity=0, eid="FL1103885:removed")
ev = c._handle("central.disaster.fl.removed.austria", json.dumps(msg).encode())
assert ev is not None
assert ev.group_key == "FL1103885" # ':removed' stripped -> matches original
assert ev.data.get("_central_tombstone") is True
def test_severity_mapping():
assert map_severity(0) == "routine"
assert map_severity(1) == "routine"
assert map_severity(2) == "priority"
assert map_severity(3) == "immediate"
assert map_severity(4) == "immediate"
assert map_severity(None) == "routine"
def test_category_mapping():
assert map_category("wx.alert.severe_thunderstorm_warning") == "weather_warning"
assert map_category("quake.event") == "earthquake_event"
assert map_category("fire.hotspot.viirs_noaa20.high") == "wildfire_hotspot"
assert map_category("hydro.00060.usgs.06901250") == "stream_flow"
# ---- async callback path ----
def test_on_message_emits_and_acks():
c, env, rec = make_consumer()
msg = FakeMsg("central.quake.event.moderate", envelope())
asyncio.run(c._on_message(msg))
assert msg.acked is True
assert len(rec) == 1
def test_start_no_op_when_all_native():
"""start() is a no-op (no NATS connect) when no adapter is central."""
c, env, rec = make_consumer()
asyncio.run(c.start()) # must not raise / must not require NATS
assert c._nc is None
fix(central): v0.4 C.3.1 -- preserve secret refs in save_section + deliver_policy=NEW (no backlog flood) Fixes the two real bugs C.3 surfaced when flipping usgs_quake to central. BUG #1 -- GUI save dropped ${VAR} secret refs (config_loader.save_section). before: A GUI PUT round-trips the *interpolated* secret value (GET returns the resolved key string, e.g. the real TomTom key). save_section's check_secrets saw a literal string at a SECRET_FIELDS path, didn't recognize it as a ref, and DROPPED it -- losing the on-disk ${TOMTOM_API_KEY} placeholder. C.3's flip PUT stripped TomTom's key. after: check_secrets now reads the raw on-disk value (pre-interpolation) for each secret field and decides three ways: on-disk ${VAR} and new == resolved(VAR) -> keep the ${VAR} ref on-disk ${VAR} and new != resolved(VAR) -> intentional change, store it no on-disk ${VAR} ref -> reject (never write a raw secret to a domain file) ${VAR} resolution mirrors load: os.environ first, then /data/secrets/.env. The common case (GUI re-saves unchanged config) now preserves the placeholder instead of dropping it. BUG #2 -- CentralConsumer replayed the entire retained backlog on first flip. before: js.subscribe(...) with no config -> default deliver_policy=all. Fine for quake (682 msgs) but would flood the bus with ~330k traffic_flow messages on first flip. after: consumer_config() -> ConsumerConfig(deliver_policy=DeliverPolicy.NEW): only messages published AFTER consumer creation. meshai won't see the backlog on first flip -- acceptable, Central is a live firehose for current events. (NOT geo-filtering -- that's a Central-side issue filed separately for the Central project.) Files: meshai/config_loader.py (save_section secret preservation), meshai/central/consumer.py (consumer_config() + deliver_policy=NEW), tests/test_save_section_secret_preserve.py (new), tests/test_central_consumer.py (deliver_policy assertion). Verification: - (A) py_compile clean on config_loader.py + consumer.py. - (C) pytest -q: 276 passed (272 + 4 new -- preserve-unchanged-ref, changed-value-written, no-placeholder-still-rejects, deliver_policy=NEW). The C.2.1 strip test still passes (no placeholder -> reject). - (D) In-prod (rebuilt): GET+PUT /api/config/environmental round-trip -> {"saved":true}; on-disk traffic.api_key stayed '${TOMTOM_API_KEY}' (SECRET_REF_PRESERVED: True), not the literal key; disk restored to baseline. consumer_config().deliver_policy == DeliverPolicy.NEW in the built image. Follow-up for D rollout: the durable 'meshai-v04-central_quake_' created during C.3 was made with deliver_policy=all; re-flipping a domain may need that stale durable deleted on the Central NATS server first (config mismatch on re-subscribe). D rollout (remaining domains) is now safe: GUI flips preserve secret refs and new subscriptions don't replay huge backlogs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 04:55:20 +00:00
def test_consumer_config_uses_deliver_policy_new():
"""C.3.1: Central subscriptions use deliver_policy=NEW (no full-backlog replay)."""
from meshai.central.consumer import consumer_config
from nats.js.api import DeliverPolicy
assert consumer_config().deliver_policy == DeliverPolicy.NEW
2026-05-28 05:05:12 +00:00
def test_subject_domain_fallback_for_unmapped_category():
fix(traffic): v0.5.7-traffic -- NATS pattern fix + itd_511 sub-adapter routing + categories audit Second family of the v0.5.7 NATS-and-categories campaign. Weather went first because its NWS pattern was already legal; traffic was carrying invalid NATS syntax in production. FIX 1 -- Invalid `>` mid-subject in traffic. Pre-v0.5.7-traffic the subject builder shipped `central.traffic.>.{state}` for both the traffic and roads511 adapters. NATS rules say `>` is only legal at the tail token; mid-subject `>` is rejected by the broker at subscribe time (or silently delivers nothing depending on server version). Replaced with Convention B (per Central v0.10.0 meshai_integration_guide.md): single-token `*` in the event_type slot, bare state suffix -- `central.traffic.*.id` for Idaho. Shared by the wzdx, tomtom_incidents and state_511_atis adapters. FIX 2 -- roads511 dual subscribe. The new Idaho-only itd_511 adapter in Central v0.10.0 uses Convention A (`central.traffic.<event_type>.us.<state>`, the us.<state> form). Convention B (bare state) is shared with the rest of the traffic family. roads511 now owns BOTH: central.traffic.*.id (Convention B, shared with traffic via _subject_owned) central.traffic.*.us.id (Convention A, itd_511-only) Sub-adapter routing in CentralConsumer._subject_owned (v0.5.1) already keeps shared subjects scoped to the right meshai source -- no change needed. FIX 3 -- itd_511 -> roads511 in CENTRAL_ADAPTER_TO_SOURCE. Mirrors state_511_atis (added v0.5.3). Both Idaho 511 feeds collapse to a single meshai source for UX simplicity; future v0.6 may split them if Matt needs differential rules. FIX 4 -- Roads-family categories audit + finer event_type mapping. Pre-v0.5.7-traffic the central path flattened every traffic-domain event to `traffic_congestion` because work_zone / incident / closure had no entries in _CATEGORY_MAP and fell through to the `traffic.` catchall (then the subject-domain fallback). Added three explicit map entries before the catchall: ("work_zone", "work_zone") # catches "work_zone" and "work_zone.wzdx" ("incident", "road_incident") # catches incident.tomtom_incidents + bare ("closure", "road_closure") # catches closure + closure.itd_511 ALERT_CATEGORIES gains two new roads-family entries so the Advanced Rules editor can target them: work_zone -- Active construction/maintenance work zone road_incident -- Reported incident (crash, hazard, debris) Existing entries `road_closure` and `traffic_congestion` kept. composer._CATEGORY_EMOJI gains matching glyphs (🚧 work_zone, 🚨 road_incident) so the live LoRa rendering lines up with the category example_message glyphs. Audit cross-check (test_alert_categories_roads_complete enforces parity): Native emit: traffic.py -> traffic_congestion; roads511.py -> road_closure Central path emit (via map_category): {road_closure, traffic_congestion, work_zone, road_incident} ALERT_CATEGORIES{toggle=roads}: {road_closure, traffic_congestion, work_zone, road_incident} Parity. No orphans, no missing. DEFERRED to v0.5.8: itd_511_cameras / traffic_cameras stream lives at a different subject domain (central.traffic_cameras.>) and needs a new meshai source (roads_cameras or similar). Out of scope for v0.5.7. Tests ----- PYTHONPATH=. pytest -q: 366 passed (was 345; +21 net). - tests/test_traffic_v057.py (new): NATS-syntax checks (`>` only at tail, single-token `*`), traffic Convention B, roads511 dual-subscribe, shared bare-state subject, itd_511 + state_511_atis remap, map_category event_type preservation, ALERT_CATEGORIES roads parity (reflection-based scan of native emit + central path), required-fields check on the four roads entries. - tests/test_central_region_routing.py: updated `test_subjects_for_traffic_and_roads511_share_state_token` -> two new tests covering Convention B (traffic) and dual-subscribe (roads511). - tests/test_central_consumer.py: updated `test_subject_domain_fallback_for_unmapped_category` (work_zone.wzdx is now mapped, switched to a genuinely-unmapped category) + new `test_v057_traffic_work_zone_now_mapped` asserting wzdx envelopes land on ev.category=="work_zone". Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-04 06:10:12 +00:00
"""D.1: an unmapped category falls back to the subject domain instead
of returning 'other'.
v0.5.7-traffic note: 'work_zone.wzdx' is now MAPPED (-> 'work_zone'),
so we use a genuinely-unmapped category string here to exercise the
fallback path. The subject-domain fallback for central.traffic.* is
still 'traffic_congestion'.
"""
2026-05-28 05:05:12 +00:00
import json
from meshai.central.consumer import CentralConsumer, category_from_subject
from meshai.config import EnvironmentalConfig
from meshai.notifications.pipeline.bus import EventBus
assert category_from_subject("central.traffic.work_zone.ok") == "traffic_congestion"
rec = []
bus = EventBus(); bus.subscribe(rec.append)
c = CentralConsumer(EnvironmentalConfig(), bus)
env = {"id": "wz1", "data": {"id": "wz1", "adapter": "wzdx",
fix(traffic): v0.5.7-traffic -- NATS pattern fix + itd_511 sub-adapter routing + categories audit Second family of the v0.5.7 NATS-and-categories campaign. Weather went first because its NWS pattern was already legal; traffic was carrying invalid NATS syntax in production. FIX 1 -- Invalid `>` mid-subject in traffic. Pre-v0.5.7-traffic the subject builder shipped `central.traffic.>.{state}` for both the traffic and roads511 adapters. NATS rules say `>` is only legal at the tail token; mid-subject `>` is rejected by the broker at subscribe time (or silently delivers nothing depending on server version). Replaced with Convention B (per Central v0.10.0 meshai_integration_guide.md): single-token `*` in the event_type slot, bare state suffix -- `central.traffic.*.id` for Idaho. Shared by the wzdx, tomtom_incidents and state_511_atis adapters. FIX 2 -- roads511 dual subscribe. The new Idaho-only itd_511 adapter in Central v0.10.0 uses Convention A (`central.traffic.<event_type>.us.<state>`, the us.<state> form). Convention B (bare state) is shared with the rest of the traffic family. roads511 now owns BOTH: central.traffic.*.id (Convention B, shared with traffic via _subject_owned) central.traffic.*.us.id (Convention A, itd_511-only) Sub-adapter routing in CentralConsumer._subject_owned (v0.5.1) already keeps shared subjects scoped to the right meshai source -- no change needed. FIX 3 -- itd_511 -> roads511 in CENTRAL_ADAPTER_TO_SOURCE. Mirrors state_511_atis (added v0.5.3). Both Idaho 511 feeds collapse to a single meshai source for UX simplicity; future v0.6 may split them if Matt needs differential rules. FIX 4 -- Roads-family categories audit + finer event_type mapping. Pre-v0.5.7-traffic the central path flattened every traffic-domain event to `traffic_congestion` because work_zone / incident / closure had no entries in _CATEGORY_MAP and fell through to the `traffic.` catchall (then the subject-domain fallback). Added three explicit map entries before the catchall: ("work_zone", "work_zone") # catches "work_zone" and "work_zone.wzdx" ("incident", "road_incident") # catches incident.tomtom_incidents + bare ("closure", "road_closure") # catches closure + closure.itd_511 ALERT_CATEGORIES gains two new roads-family entries so the Advanced Rules editor can target them: work_zone -- Active construction/maintenance work zone road_incident -- Reported incident (crash, hazard, debris) Existing entries `road_closure` and `traffic_congestion` kept. composer._CATEGORY_EMOJI gains matching glyphs (🚧 work_zone, 🚨 road_incident) so the live LoRa rendering lines up with the category example_message glyphs. Audit cross-check (test_alert_categories_roads_complete enforces parity): Native emit: traffic.py -> traffic_congestion; roads511.py -> road_closure Central path emit (via map_category): {road_closure, traffic_congestion, work_zone, road_incident} ALERT_CATEGORIES{toggle=roads}: {road_closure, traffic_congestion, work_zone, road_incident} Parity. No orphans, no missing. DEFERRED to v0.5.8: itd_511_cameras / traffic_cameras stream lives at a different subject domain (central.traffic_cameras.>) and needs a new meshai source (roads_cameras or similar). Out of scope for v0.5.7. Tests ----- PYTHONPATH=. pytest -q: 366 passed (was 345; +21 net). - tests/test_traffic_v057.py (new): NATS-syntax checks (`>` only at tail, single-token `*`), traffic Convention B, roads511 dual-subscribe, shared bare-state subject, itd_511 + state_511_atis remap, map_category event_type preservation, ALERT_CATEGORIES roads parity (reflection-based scan of native emit + central path), required-fields check on the four roads entries. - tests/test_central_region_routing.py: updated `test_subjects_for_traffic_and_roads511_share_state_token` -> two new tests covering Convention B (traffic) and dual-subscribe (roads511). - tests/test_central_consumer.py: updated `test_subject_domain_fallback_for_unmapped_category` (work_zone.wzdx is now mapped, switched to a genuinely-unmapped category) + new `test_v057_traffic_work_zone_now_mapped` asserting wzdx envelopes land on ev.category=="work_zone". Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-04 06:10:12 +00:00
"category": "telematics.unknown_thing", "time": "2026-05-28T00:00:00Z", "severity": 1,
2026-05-28 05:05:12 +00:00
"geo": {"centroid": [-96.2, 36.15], "primary_region": "US-OK", "regions": ["US-OK"]},
"data": {"road": "I-44"}}}
ev = c._handle("central.traffic.work_zone.ok", json.dumps(env).encode())
assert ev is not None and ev.category == "traffic_congestion"
fix(traffic): v0.5.7-traffic -- NATS pattern fix + itd_511 sub-adapter routing + categories audit Second family of the v0.5.7 NATS-and-categories campaign. Weather went first because its NWS pattern was already legal; traffic was carrying invalid NATS syntax in production. FIX 1 -- Invalid `>` mid-subject in traffic. Pre-v0.5.7-traffic the subject builder shipped `central.traffic.>.{state}` for both the traffic and roads511 adapters. NATS rules say `>` is only legal at the tail token; mid-subject `>` is rejected by the broker at subscribe time (or silently delivers nothing depending on server version). Replaced with Convention B (per Central v0.10.0 meshai_integration_guide.md): single-token `*` in the event_type slot, bare state suffix -- `central.traffic.*.id` for Idaho. Shared by the wzdx, tomtom_incidents and state_511_atis adapters. FIX 2 -- roads511 dual subscribe. The new Idaho-only itd_511 adapter in Central v0.10.0 uses Convention A (`central.traffic.<event_type>.us.<state>`, the us.<state> form). Convention B (bare state) is shared with the rest of the traffic family. roads511 now owns BOTH: central.traffic.*.id (Convention B, shared with traffic via _subject_owned) central.traffic.*.us.id (Convention A, itd_511-only) Sub-adapter routing in CentralConsumer._subject_owned (v0.5.1) already keeps shared subjects scoped to the right meshai source -- no change needed. FIX 3 -- itd_511 -> roads511 in CENTRAL_ADAPTER_TO_SOURCE. Mirrors state_511_atis (added v0.5.3). Both Idaho 511 feeds collapse to a single meshai source for UX simplicity; future v0.6 may split them if Matt needs differential rules. FIX 4 -- Roads-family categories audit + finer event_type mapping. Pre-v0.5.7-traffic the central path flattened every traffic-domain event to `traffic_congestion` because work_zone / incident / closure had no entries in _CATEGORY_MAP and fell through to the `traffic.` catchall (then the subject-domain fallback). Added three explicit map entries before the catchall: ("work_zone", "work_zone") # catches "work_zone" and "work_zone.wzdx" ("incident", "road_incident") # catches incident.tomtom_incidents + bare ("closure", "road_closure") # catches closure + closure.itd_511 ALERT_CATEGORIES gains two new roads-family entries so the Advanced Rules editor can target them: work_zone -- Active construction/maintenance work zone road_incident -- Reported incident (crash, hazard, debris) Existing entries `road_closure` and `traffic_congestion` kept. composer._CATEGORY_EMOJI gains matching glyphs (🚧 work_zone, 🚨 road_incident) so the live LoRa rendering lines up with the category example_message glyphs. Audit cross-check (test_alert_categories_roads_complete enforces parity): Native emit: traffic.py -> traffic_congestion; roads511.py -> road_closure Central path emit (via map_category): {road_closure, traffic_congestion, work_zone, road_incident} ALERT_CATEGORIES{toggle=roads}: {road_closure, traffic_congestion, work_zone, road_incident} Parity. No orphans, no missing. DEFERRED to v0.5.8: itd_511_cameras / traffic_cameras stream lives at a different subject domain (central.traffic_cameras.>) and needs a new meshai source (roads_cameras or similar). Out of scope for v0.5.7. Tests ----- PYTHONPATH=. pytest -q: 366 passed (was 345; +21 net). - tests/test_traffic_v057.py (new): NATS-syntax checks (`>` only at tail, single-token `*`), traffic Convention B, roads511 dual-subscribe, shared bare-state subject, itd_511 + state_511_atis remap, map_category event_type preservation, ALERT_CATEGORIES roads parity (reflection-based scan of native emit + central path), required-fields check on the four roads entries. - tests/test_central_region_routing.py: updated `test_subjects_for_traffic_and_roads511_share_state_token` -> two new tests covering Convention B (traffic) and dual-subscribe (roads511). - tests/test_central_consumer.py: updated `test_subject_domain_fallback_for_unmapped_category` (work_zone.wzdx is now mapped, switched to a genuinely-unmapped category) + new `test_v057_traffic_work_zone_now_mapped` asserting wzdx envelopes land on ev.category=="work_zone". Safe-mode preserved (master off, all family toggles off, all adapters native, central disabled). No live toggle flipped. Not tagging yet -- v0.5.7 tag waits until all families ship. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-04 06:10:12 +00:00
def test_v057_traffic_work_zone_now_mapped():
"""v0.5.7-traffic: 'work_zone.wzdx' maps to the new 'work_zone' meshai
category (not flattened to traffic_congestion)."""
import json
from meshai.central.consumer import CentralConsumer
from meshai.config import EnvironmentalConfig
from meshai.notifications.pipeline.bus import EventBus
rec = []
bus = EventBus(); bus.subscribe(rec.append)
c = CentralConsumer(EnvironmentalConfig(), bus)
env = {"id": "wz2", "data": {"id": "wz2", "adapter": "wzdx",
"category": "work_zone.wzdx", "time": "2026-05-28T00:00:00Z", "severity": 1,
"geo": {"centroid": [-114.0, 42.0], "primary_region": "US-ID", "regions": ["US-ID"]},
"data": {"road": "I-84"}}}
ev = c._handle("central.traffic.work_zone.id", json.dumps(env).encode())
assert ev is not None and ev.category == "work_zone"
@pytest.mark.parametrize("adapter,expected", [
("wfigs_incidents", "fires"),
("nwis", "usgs"),
("swpc_alerts", "swpc"),
("wzdx", "traffic"),
("nws", "nws"), # 1:1 passthrough
("experimental_foo", "experimental_foo"), # unknown -> passthrough
])
def test_central_adapter_source_remap(adapter, expected):
"""D.2: Central adapter names map to meshai source names (unknown passes through)."""
import json
from meshai.central.consumer import CentralConsumer
from meshai.config import EnvironmentalConfig
from meshai.notifications.pipeline.bus import EventBus
rec = []
bus = EventBus(); bus.subscribe(rec.append)
c = CentralConsumer(EnvironmentalConfig(), bus)
env = {"id": "e1", "data": {"id": "e1", "adapter": adapter, "category": "wx.alert.x",
"time": "2026-05-28T00:00:00Z", "severity": 1,
"geo": {"centroid": [-114.0, 42.0], "primary_region": "US-ID", "regions": ["US-ID"]},
"data": {}}}
ev = c._handle("central.wx.alert.x", json.dumps(env).encode())
assert ev is not None and ev.source == expected