feat(central): v0.4 C.1 Central connector backend (no-op until adapter source flipped)

Adds the backend for sourcing environmental feeds from Central's NATS
JetStream firehose instead of (or alongside) meshai's native adapters.
Architecture is Matt-approved Option 3' (dedicated package + per-adapter
source switch surfaced on the existing Environmental config).

NO-OP POSTURE (intentional): every adapter defaults to feed_source="native"
and environmental.central.enabled defaults false, so on a stock config the
CentralConsumer starts and subscribes to nothing -- behavior is byte-for-byte
v0.3. Live env_feeds.yaml is unchanged on disk; an operator who touches
nothing sees no change. Flipping an adapter to central is Phase C.3; the
dashboard UI for it is Phase C.2.

What landed:
- meshai/central/ package (CentralConsumer): async start()/stop(), JetStream
  durable subscribe to subjects derived from adapters with feed_source=central,
  and _on_message -> normalize -> bus.emit. nats-py is lazy-imported only on
  the connect path, so no-op boot has zero NATS dependency.
- Normalization (CloudEvents envelope -> Central Event -> upstream data):
    source   = inner Event.adapter
    category = Central hierarchical string -> meshai flat, via a small
               table-driven prefix map (map_category)
    severity = 0|1->routine, 2->priority, 3|4->immediate, null->routine
    lat/lon  = geo.centroid, swapped from GeoJSON [lon,lat] -> (lat,lon)
    group_key/inhibit = outer envelope id (dedup parity with native adapters)
    expires/timestamp parsed from ISO-8601
    Event.data = upstream payload verbatim (generic _enriched merge, preserved
                 as-is incl. hydro's extra usgs_site/usgs_stats bundles)
- Tombstone (`.removed.` subject or `:removed` id suffix) -> a "clear" Event
  carrying the ORIGINAL group_key (`:removed` stripped) + data._central_tombstone
  so the grouper/inhibitor lets the prior event lapse naturally.
- config.py: a `_SourcedFeed` mixin adds `feed_source: native|central`
  (validated in __post_init__) to all 10 adapter configs; new
  CentralConsumerConfig as environmental.central { enabled, url, durable,
  connect_timeout }. Both ride the generic _dict_to_dataclass coercion, so
  they are GUI-editable via PUT /config/environmental (Rule 17) -- frontend
  fields come in C.2.
- env/store.py: each adapter is instantiated only when
  enabled AND feed_source=="native"; a feed_source=central adapter is skipped
  natively (debug-logged) so Central can own it without a duplicate.
- main.py: CentralConsumer constructed + started after start_pipeline(),
  stopped in stop().

DEVIATION FROM SPEC (documented): the spec named the new field `source`, but
FIRMSConfig already has a `source` field (the satellite product,
"VIIRS_SNPP_NRT"). To avoid the collision the field is named **feed_source**
across all adapters. Everything else follows the spec.

NETWORKING: zero infra change required. The meshai container already reaches
the Central NATS server directly (TCP to 100.64.0.12:4222 OK) and resolves
central.echo6.mesh via the Phase 2.6.6 MagicDNS fix. No docker-compose edit;
default bridge works (LXC host masquerades to the Tailscale CGNAT range). The
lighter bridge-route / host-net / sidecar fallbacks were not needed.

Tests: tests/test_central_consumer.py (11) + tests/test_config_source_field.py
(6): no-op-when-native, subjects-when-central, source-gate skips native
instantiation, normalize+emit, _enriched preserved verbatim, tombstone->clear,
severity map (0-4/null), category map (>=4 strings), async _on_message
emits+acks, start() no-op without NATS, feed_source default/validate/reject/
dict-coercion. Full suite: 269 passed (was 253 + 16 new).

Verification: (A) no bare self._x() in consumer.py. (B) py_compile clean.
(C) 269 passed. (D) rebuilt prod -- 8 native adapters, pipeline started,
native nifc/traffic emissions still flowing, healthy, no errors, log
"CentralConsumer started; 0 subjects subscribed -- no adapters set to central".
(E) in-container synthetic _on_message injection normalized correctly
(usgs_quake/earthquake_event/immediate, centroid swapped, _enriched preserved)
and reached the bus; ephemeral, no config change to roll back.

C.2 (dashboard frontend for the feed_source switch + central connection) is next.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
K7ZVX 2026-05-28 02:28:19 +00:00
commit 73c007d227
8 changed files with 500 additions and 20 deletions

View file

@ -297,7 +297,19 @@ class MeshIntelligenceConfig:
# Environmental feed configs
@dataclass
class NWSConfig:
class _SourcedFeed:
"""Mixin: an environmental feed is sourced 'native' (local adapter) or
'central' (Central NATS firehose). Default 'native' preserves v0.3 behavior."""
feed_source: str = "native"
def __post_init__(self):
if self.feed_source not in ("native", "central"):
raise ValueError(f"feed_source must be 'native' or 'central', got {self.feed_source!r}")
@dataclass
class NWSConfig(_SourcedFeed):
"""NWS weather alerts settings."""
enabled: bool = True
@ -316,7 +328,7 @@ class NWSConfig:
@dataclass
class SWPCConfig:
class SWPCConfig(_SourcedFeed):
"""NOAA Space Weather settings."""
enabled: bool = True
@ -331,7 +343,7 @@ class SWPCConfig:
@dataclass
class DuctingConfig:
class DuctingConfig(_SourcedFeed):
"""Tropospheric ducting settings."""
enabled: bool = True
@ -349,7 +361,7 @@ class DuctingConfig:
@dataclass
class NICFFiresConfig:
class NICFFiresConfig(_SourcedFeed):
"""NIFC fire perimeters settings (Phase 2)."""
enabled: bool = False
@ -358,7 +370,7 @@ class NICFFiresConfig:
@dataclass
class AvalancheConfig:
class AvalancheConfig(_SourcedFeed):
"""Avalanche advisory settings (Phase 2)."""
enabled: bool = False
@ -368,7 +380,7 @@ class AvalancheConfig:
@dataclass
class USGSConfig:
class USGSConfig(_SourcedFeed):
"""USGS stream gauge settings."""
enabled: bool = False
@ -378,7 +390,7 @@ class USGSConfig:
@dataclass
class USGSQuakeConfig:
class USGSQuakeConfig(_SourcedFeed):
"""USGS earthquake feed settings (Phase 2.14)."""
enabled: bool = False
@ -391,7 +403,7 @@ class USGSQuakeConfig:
@dataclass
class TomTomConfig:
class TomTomConfig(_SourcedFeed):
"""TomTom traffic flow settings."""
enabled: bool = False
@ -401,7 +413,7 @@ class TomTomConfig:
@dataclass
class Roads511Config:
class Roads511Config(_SourcedFeed):
"""511 road conditions settings."""
enabled: bool = False
@ -413,7 +425,7 @@ class Roads511Config:
@dataclass
class FIRMSConfig:
class FIRMSConfig(_SourcedFeed):
"""NASA FIRMS satellite fire hotspot settings."""
enabled: bool = False
@ -426,6 +438,16 @@ class FIRMSConfig:
proximity_km: float = 10.0 # km to match known fire
@dataclass
class CentralConsumerConfig:
"""Connection settings for the Central NATS JetStream consumer (v0.4)."""
enabled: bool = False
url: str = "nats://central.echo6.mesh:4222"
durable: str = "meshai-consumer"
connect_timeout: float = 10.0
@dataclass
class EnvironmentalConfig:
"""Environmental feeds settings."""
@ -442,6 +464,7 @@ class EnvironmentalConfig:
traffic: TomTomConfig = field(default_factory=TomTomConfig)
roads511: Roads511Config = field(default_factory=Roads511Config)
firms: FIRMSConfig = field(default_factory=FIRMSConfig)
central: CentralConsumerConfig = field(default_factory=CentralConsumerConfig)
@dataclass