meshai/work/tests/test_firms_fusion_event_contract.py

108 lines
4.9 KiB
Python
Raw Normal View History

chore: excise the dead Central NATS consumer path (-11,328 LOC) (#144) * chore: excise the dead Central NATS consumer path Central was retired and its database dropped 2026-07-15; its NATS broker no longer exists. Verified against the live CT108 deployment: all 12 adapters run feed_source=native, zero on central, and central.enabled is False (default, never overridden). The consumer and its handlers were unreachable. Removed: - central/consumer.py and 6 dead handlers (nws, quake, swpc, nwis, avy, incident) -- their handle_* entrypoints were reachable only from the consumer's dispatch - the Central wiring in main.py (init, guarded start, retry loop, stop path) - the dead config surface: CentralConsumerConfig, EnvironmentalConfig.central, adapter_config ("central","severity_thresholds") and its display block - the nats-py dependency (consumer.py was its only importer) - 19 test files that exercised only the dead path KEPT -- these live under central/ but are imported directly by native adapters, and deleting them would break production: - wfigs_handler.py: firms_handler._handle_pass_boundary() calls its _render() on the live FIRMS growth-fire path (env/firms.py -> ingest_hotspot_pixel) - firms_handler, satpass_handler, tle_handler: split files whose handle_* entrypoints are dead but whose engines are live. Left intact; splitting them is separate work. - pass_predictor, budget, idaho_gauge_sites: fully live. The usgs_quake keys global_mag_floor / regional_mag_floor / regional_centroid / regional_radius_mi / broadcast_pager_alerts are NOT removed despite comments labelling them "CENTRAL-PATH ONLY" -- notifications/gating/quake.py reads them unconditionally in the native path. Those comments are corrected separately. Test-count note: the suite drops ~425 tests. Most were migration PARITY tests whose sole purpose was proving the native rewrite byte-matched the Central handler (golden byte-parity, cross-source identity, gate-sequence replay). With the handler deleted there is nothing left to compare against, so they cannot exist. Native-only tests were kept and reworked where a test reached for a central symbol incidentally. This is a real coverage loss, accepted deliberately: the parity harness proved the refactor faithful, and git history preserves the originals. Suite: 1984 passed, 6 failed -- the same 6 pre-existing failures as main (stale SCHEMA_VERSION x3, expired TLE fixtures x2, one order-dependent), all being fixed on fix/green-test-suite. No new failures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(nws): restore native-only golden coverage for the wire formatter Commit ca751fb5 deleted the Central nws_handler parity harness along with the handler itself, which took the ONLY tests that pinned formatters.nws .format()'s literal wire output. Gate-sequence and schema-conformance tests already survived natively; the formatter's actual rendered text did not have any native-only regression net. Add TestFormatterGolden to test_nws_refactor.py: 3 real-fixture cases plus 6 hand-built pathological cases mined from the deleted test_nws_handler.py (SVR path-sampling, the "no dangling separator" regression, TOR on-ground vs radar-indicated, FFW flood-cause detection). Every literal was verified by temporarily restoring the pre-excision central.nws_handler._render() from git history (ca751fb5^) in a throwaway, uncommitted script, confirming byte-identical output against the current native format() for all 37 real fixtures (nws/ + nws_last/) and all 9 pathological cases, then pinning the confirmed-matching string as the literal -- not a blind snapshot of current behavior. quake/swpc/avalanche/hydro/incident/fire were checked and already carry equivalent native-only golden coverage (added directly in ca751fb5), so no changes were needed there. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:03:40 -06:00
"""Regression tests for the FIRMS FirePacer contract (issue #119).
Originally three sections (A/B/C) guarding issues #117-#119 in the path
from firms_handler's growth/spotting/halt/cluster fusion decisions to the
actual meshai Event that reaches the dispatcher + pacer. Sections A and B,
plus one test in section C, drove that path exclusively through
`meshai.central.consumer.CentralConsumer._normalize()`/`._handle()` -- the
Central NATS-consumer bridge, which has been deleted (production runs the
native env/firms.py -> firms_handler.ingest_hotspot_pixel fusion path
exclusively; see meshai.env.firms.FirmsAdapter._make_fusion_event, which
independently applies the same `_severity_override`-over-`severity`
resolution). Deleting CentralConsumer makes those tests uncollectable, and
since the mechanism they guarded (issues #117/#118) lived entirely inside
the now-dead consumer, they can no longer exist as tests of live behavior
-- git history preserves them.
The #117/#118 category+severity contract they exercised at the data_patch
level is independently covered against the LIVE native gating path in
tests/test_firms_refactor.py (asserts `data_patch["category"]` /
`data_patch["_severity_override"]` for growth/spotting/halt/cluster) and at
the Event/category level in tests/test_firms_native_fusion.py (drives the
real adapter tick() -> to_event() chain and asserts `ev.category`).
What remains here (issue #119, section C) is two tests that exercise the
FirePacer class directly with no dependency on CentralConsumer -- these are
native, standalone FirePacer unit tests (head-of-line ordering, no-drop
guarantee) and survive unchanged. The third section-C test (routing a real
FIRMS growth broadcast into a mocked pacer via CentralConsumer._handle) is
deleted along with A/B for the same reason; the equivalent native-path
routing guarantee (store._emit_event() -> FirePacer, for source="firms")
is already covered end-to-end in tests/test_native_fire_pacer.py.
fix(firms): repair the FIRMS fire-fusion Event contract (issues #117-#119) (#120) Three independent bugs kept firms_handler's growth/spotting/halt/cluster fusion decisions from reaching a correct mesh Event: - #117: consumer._normalize() computed `category` from the raw Central category BEFORE the per-adapter handler ran and never re-read data["category"] afterward, so every firms_handler category stamp was a silent no-op. Now re-read post-dispatch, validated against the known category registry (unrecognized overrides are logged and ignored). - #118: consumer.py only ever honors data["_severity_override"], but firms_handler's halt/spotting/cluster sites stamped the plain data["severity"] key instead (only growth used the right key). Switched all three sites to `_severity_override` for one consistent contract. This is severity plumbing only -- it does not change which events fire. - #119: FirePacer's gate only matched source in ("fires","wfigs") at severity=="priority", so FIRMS fusion broadcasts (source="firms", growth/spotting at "immediate") never reached the pacer. Broadened the gate to cover "firms" + {"priority","immediate"}, and gave FirePacer head-of-line insertion so an "immediate" event is never stuck behind already-queued "priority" events. Still unbounded/never-drops. Cluster detection is left exactly as main ships it: live, always on, no toggle (PR #73's curated new-fire cluster broadcasts with cold-start silent-seeding). Only its severity-override key changes, under #118. Updated existing tests that asserted the old (buggy) data["severity"] contract, and added tests/test_firms_fusion_event_contract.py covering all three fixes end-to-end through consumer._normalize()/_handle() and FirePacer directly. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 17:41:40 -06:00
"""
from __future__ import annotations
import asyncio
from meshai.notifications.events import make_event
from meshai.notifications.pipeline.pacer import FirePacer
# ═════════════════════════════════════════════════════════════════════════════
chore: excise the dead Central NATS consumer path (-11,328 LOC) (#144) * chore: excise the dead Central NATS consumer path Central was retired and its database dropped 2026-07-15; its NATS broker no longer exists. Verified against the live CT108 deployment: all 12 adapters run feed_source=native, zero on central, and central.enabled is False (default, never overridden). The consumer and its handlers were unreachable. Removed: - central/consumer.py and 6 dead handlers (nws, quake, swpc, nwis, avy, incident) -- their handle_* entrypoints were reachable only from the consumer's dispatch - the Central wiring in main.py (init, guarded start, retry loop, stop path) - the dead config surface: CentralConsumerConfig, EnvironmentalConfig.central, adapter_config ("central","severity_thresholds") and its display block - the nats-py dependency (consumer.py was its only importer) - 19 test files that exercised only the dead path KEPT -- these live under central/ but are imported directly by native adapters, and deleting them would break production: - wfigs_handler.py: firms_handler._handle_pass_boundary() calls its _render() on the live FIRMS growth-fire path (env/firms.py -> ingest_hotspot_pixel) - firms_handler, satpass_handler, tle_handler: split files whose handle_* entrypoints are dead but whose engines are live. Left intact; splitting them is separate work. - pass_predictor, budget, idaho_gauge_sites: fully live. The usgs_quake keys global_mag_floor / regional_mag_floor / regional_centroid / regional_radius_mi / broadcast_pager_alerts are NOT removed despite comments labelling them "CENTRAL-PATH ONLY" -- notifications/gating/quake.py reads them unconditionally in the native path. Those comments are corrected separately. Test-count note: the suite drops ~425 tests. Most were migration PARITY tests whose sole purpose was proving the native rewrite byte-matched the Central handler (golden byte-parity, cross-source identity, gate-sequence replay). With the handler deleted there is nothing left to compare against, so they cannot exist. Native-only tests were kept and reworked where a test reached for a central symbol incidentally. This is a real coverage loss, accepted deliberately: the parity harness proved the refactor faithful, and git history preserves the originals. Suite: 1984 passed, 6 failed -- the same 6 pre-existing failures as main (stale SCHEMA_VERSION x3, expired TLE fixtures x2, one order-dependent), all being fixed on fix/green-test-suite. No new failures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(nws): restore native-only golden coverage for the wire formatter Commit ca751fb5 deleted the Central nws_handler parity harness along with the handler itself, which took the ONLY tests that pinned formatters.nws .format()'s literal wire output. Gate-sequence and schema-conformance tests already survived natively; the formatter's actual rendered text did not have any native-only regression net. Add TestFormatterGolden to test_nws_refactor.py: 3 real-fixture cases plus 6 hand-built pathological cases mined from the deleted test_nws_handler.py (SVR path-sampling, the "no dangling separator" regression, TOR on-ground vs radar-indicated, FFW flood-cause detection). Every literal was verified by temporarily restoring the pre-excision central.nws_handler._render() from git history (ca751fb5^) in a throwaway, uncommitted script, confirming byte-identical output against the current native format() for all 37 real fixtures (nws/ + nws_last/) and all 9 pathological cases, then pinning the confirmed-matching string as the literal -- not a blind snapshot of current behavior. quake/swpc/avalanche/hydro/incident/fire were checked and already carry equivalent native-only golden coverage (added directly in ca751fb5), so no changes were needed there. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:03:40 -06:00
# FirePacer covers FIRMS; immediate jumps the queue; nothing is dropped
# (issue #119)
fix(firms): repair the FIRMS fire-fusion Event contract (issues #117-#119) (#120) Three independent bugs kept firms_handler's growth/spotting/halt/cluster fusion decisions from reaching a correct mesh Event: - #117: consumer._normalize() computed `category` from the raw Central category BEFORE the per-adapter handler ran and never re-read data["category"] afterward, so every firms_handler category stamp was a silent no-op. Now re-read post-dispatch, validated against the known category registry (unrecognized overrides are logged and ignored). - #118: consumer.py only ever honors data["_severity_override"], but firms_handler's halt/spotting/cluster sites stamped the plain data["severity"] key instead (only growth used the right key). Switched all three sites to `_severity_override` for one consistent contract. This is severity plumbing only -- it does not change which events fire. - #119: FirePacer's gate only matched source in ("fires","wfigs") at severity=="priority", so FIRMS fusion broadcasts (source="firms", growth/spotting at "immediate") never reached the pacer. Broadened the gate to cover "firms" + {"priority","immediate"}, and gave FirePacer head-of-line insertion so an "immediate" event is never stuck behind already-queued "priority" events. Still unbounded/never-drops. Cluster detection is left exactly as main ships it: live, always on, no toggle (PR #73's curated new-fire cluster broadcasts with cold-start silent-seeding). Only its severity-override key changes, under #118. Updated existing tests that asserted the old (buggy) data["severity"] contract, and added tests/test_firms_fusion_event_contract.py covering all three fixes end-to-end through consumer._normalize()/_handle() and FirePacer directly. Co-authored-by: Matt Johnson <mj@k7zvx.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 17:41:40 -06:00
# ═════════════════════════════════════════════════════════════════════════════
class TestPacerCoversFirms:
def test_immediate_event_emitted_before_already_queued_priority_events(self):
"""Two 'priority' events are queued first; a later 'immediate' event
must still be emitted BEFORE them (head-of-line), not after."""
emitted = []
class _FakeBus:
def emit(self, event):
emitted.append(event)
pacer = FirePacer(_FakeBus(), interval_seconds=0.01)
p1 = make_event(source="fires", category="wildfire_incident",
severity="priority", title="priority-1")
p2 = make_event(source="fires", category="wildfire_incident",
severity="priority", title="priority-2")
imm = make_event(source="firms", category="wildfire_spotting",
severity="immediate", title="immediate-1")
pacer.enqueue(p1)
pacer.enqueue(p2)
pacer.enqueue(imm) # must jump ahead of p1/p2
async def _drive():
await pacer.start()
await asyncio.sleep(0.2)
await pacer.stop()
asyncio.run(_drive())
assert [e.title for e in emitted] == [
"immediate-1", "priority-1", "priority-2"]
def test_pacer_never_drops_events(self):
"""Rapid-fire enqueue of many events (mixed severities) -- the
unbounded FIFO must eventually deliver every single one."""
emitted = []
class _FakeBus:
def emit(self, event):
emitted.append(event)
pacer = FirePacer(_FakeBus(), interval_seconds=0.001)
total = 25
for i in range(total):
sev = "immediate" if i % 5 == 0 else "priority"
pacer.enqueue(make_event(
source="firms", category="wildfire_growth",
severity=sev, title=f"evt-{i}"))
assert pacer.pending_count() == total
async def _drive():
await pacer.start()
# Generous wait: interval is 1ms, 25 events, allow real margin.
await asyncio.sleep(1.0)
await pacer.stop()
asyncio.run(_drive())
assert len(emitted) == total, (
f"pacer must never drop events: expected {total}, got {len(emitted)}")
assert pacer.pending_count() == 0