central/tests/test_enrichment_locations_coverage.py

159 lines
6.4 KiB
Python
Raw Normal View History

feat(3-L.a): finish enrichment_locations across adapters First half of the split PR L (events-tab + map deferred to L-b). Only FIRMS declared enrichment_locations, so apply_enrichment silently bypassed every other adapter. This declares it for all 12. Pre-read finding (resolved per PM): apply_enrichment is a FLAT lookup (event.data.get(lat_path)); FIRMS/usgs_quake already carry top-level latitude/longitude in event.data, but the other point adapters kept coords only in Geo.centroid where the flat path can't reach them. Per PM (option b), the 5 centroid-only adapters now also write top-level latitude/longitude into event.data, mirroring their existing Geo.centroid (lon, lat) — 2-3 lines each, no framework refactor. Geo retained for existing rendering uses. Declarations (verbatim): firms [("latitude","longitude")] (unchanged) usgs_quake [("latitude","longitude")] (already top-level in data) nwis [("latitude","longitude")] + centroid mirror eonet [("latitude","longitude")] + centroid mirror gdacs [("latitude","longitude")] + centroid mirror wfigs_incidents [("latitude","longitude")] + centroid mirror (inline data) inciweb [("latitude","longitude")] + centroid mirror (inline data) wfigs_perimeters [] # polygons, no point nws [] # forecast zones/counties, no point swpc_alerts [] # space weather, no coordinate swpc_kindex [] # space weather, no coordinate swpc_protons [] # space weather, no coordinate Centroid mirror is `latitude = centroid[1]; longitude = centroid[0]` (centroid is GeoJSON (lon, lat)); guarded on centroid presence so coordinate-less events get no lat/lon keys (apply_enrichment then skips them). map_render_kind concept dropped — the existing /events map is already geometry-kind-agnostic (renders any row's data-geometry via L.geoJSON), so it was unnecessary. Events-tab enhancements are PR L-b. Tests (test_enrichment_locations_coverage.py, 6, all registry-derived): - every adapter explicitly declares enrichment_locations in its own class body - declarations are valid list[(str,str)] - point adapters all use the canonical ("latitude","longitude") paths - >=5 point adapters are non-empty (regression guard) - synthetic-event builders prove the keys resolve: usgs_quake._feature_to_event and nwis._build_event (the two adapters with isolated builders; the four inline-build adapters are covered by the post-merge live smoke). Verification: full pytest 552 passed, 1 skipped (was 546; +6). grep subject_for_event/_ADAPTER_REGISTRY and grep 100.64.0./192.168.1. in src empty. Follow-ups (NOT here): consumer-doc per-adapter _enriched.geocoder notes for the newly-enriched adapters belong in L-b's doc pass; live end-to-end smoke runs post-merge (USGS quake + one other) per the acceptance bar. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 01:48:23 +00:00
"""Coverage tests for adapter enrichment_locations declarations (PR L-a).
Every adapter must make a conscious enrichment_locations declaration a
non-empty [(lat_field, lon_field)] for point adapters, or an explicit [] for
adapters with no point coordinate. Registry-derived (iterates
discover_adapters()), no hardcoded adapter lists.
Plus synthetic-event tests for the two adapters with isolated event builders
(usgs_quake._feature_to_event, nwis._build_event) proving the declared
latitude/longitude paths actually resolve on event.data. The four inline-build
point adapters (eonet, gdacs, wfigs_incidents, inciweb) construct events only
inside poll() and are covered by the live end-to-end smoke instead.
"""
from datetime import datetime, timezone
from pathlib import Path
from unittest.mock import MagicMock
from central.adapter_discovery import discover_adapters
from central.config_models import AdapterConfig
def test_every_adapter_explicitly_declares_enrichment_locations():
"""Each adapter declares enrichment_locations in its OWN class body (not
just inheriting the SourceAdapter default) a conscious choice per adapter."""
missing = [
name for name, cls in discover_adapters().items()
if "enrichment_locations" not in cls.__dict__
]
assert not missing, f"adapters missing an explicit enrichment_locations: {missing}"
def test_enrichment_locations_shape_is_valid():
"""Each declaration is a list of (str, str) tuples (possibly empty)."""
for name, cls in discover_adapters().items():
locs = cls.enrichment_locations
assert isinstance(locs, list), f"{name}: not a list"
for tup in locs:
assert isinstance(tup, tuple) and len(tup) == 2, f"{name}: bad tuple {tup!r}"
assert all(isinstance(p, str) for p in tup), f"{name}: non-str path {tup!r}"
def test_point_adapters_use_canonical_lat_lon_paths():
"""Every point adapter (non-empty declaration) uses the same protocol keys
('latitude', 'longitude') the convention FIRMS established."""
for name, cls in discover_adapters().items():
if cls.enrichment_locations:
assert cls.enrichment_locations == [("latitude", "longitude")], (
f"{name} uses non-canonical paths: {cls.enrichment_locations}"
)
def test_at_least_the_known_point_adapters_are_non_empty():
"""Registry-derived sanity: the adapters that carry a point coordinate have
a non-empty declaration. Derived by probing enrichment_locations, not a
hardcoded list guards against a regression that blanks them all."""
non_empty = {
name for name, cls in discover_adapters().items() if cls.enrichment_locations
}
# firms is the original; there must be several point adapters now.
assert "firms" in non_empty
assert len(non_empty) >= 5, f"expected several point adapters, got {sorted(non_empty)}"
# --- synthetic-event tests for the two isolated builders --------------------
def test_usgs_quake_event_exposes_top_level_latlon():
from central.adapters.usgs_quake import USGSQuakeAdapter
config = AdapterConfig(
name="usgs_quake", enabled=True, cadence_s=60,
settings={}, updated_at=datetime.now(timezone.utc),
)
adapter = USGSQuakeAdapter(config, MagicMock(), Path("/tmp/never_used.db"))
feature = {
"type": "Feature", "id": "test_q1",
"properties": {"mag": 2.5, "place": "X", "time": 1715000000000,
"updated": 1715000000000},
"geometry": {"type": "Point", "coordinates": [-116.2, 43.7, 10.5]},
}
event = adapter._feature_to_event(feature)
assert event is not None
assert event.data["latitude"] == 43.7
assert event.data["longitude"] == -116.2
def test_nwis_event_mirrors_centroid_into_data():
from central.adapters.nwis import NWISAdapter
config = AdapterConfig(
name="nwis", enabled=True, cadence_s=900,
settings={}, updated_at=datetime.now(timezone.utc),
)
adapter = NWISAdapter(config, MagicMock(), Path("/tmp/never_used.db"))
feature = {
"geometry": {"type": "Point", "coordinates": [-90.25, 41.78]}, # (lon, lat)
"properties": {
"monitoring_location_id": "USGS-05420500",
"time": "2026-05-20T00:00:00Z",
"value": "123.0",
"unit_of_measure": "ft",
},
}
event = adapter._build_event(feature, "00060")
assert event is not None
# latitude = centroid[1], longitude = centroid[0]; no axis swap.
assert event.data["latitude"] == 41.78
assert event.data["longitude"] == -90.25
# Geo.centroid retained for existing rendering uses.
assert event.geo.centroid == (-90.25, 41.78)
feat(gui-bugs): fix eonet dashboard exception + out-of-range map bbox Kickoff of the v0.7.x GUI rework arc. Two operator-facing bugs confirmed live; production code, central-gui + central-supervisor restart required. Bug 1 (eonet exception leaking to /dashboard): The supervisor calls adapter.bump_last_seen on every dedup hit, but only 4 of 12 adapters defined it and the base class did not. Adapters that re-emit already-published events (eonet re-lists open natural events each poll) raised AttributeError; the supervisor published it as the adapter's status error, which /dashboard rendered as literal text in the Last Poll cell. Fix: add bump_last_seen to the SourceAdapter base class (guarded on getattr(self, "_db", None)); remove the 4 now-redundant identical overrides. Fixes all 8 affected adapters, not just eonet. Documents the method in PRODUCER-INTEGRATION.md 4.3 (producer-doc API guard). Bug 2 (map bbox out of valid range): applyViewportFilter serialized raw Leaflet getEast()/getWest(), which exceed [-180,180] when panned past the dateline at low zoom (e.g. region_east=411.3281, region_west=-608.2031), and _parse_events_params passed them straight to ST_MakeEnvelope. Fix (JS): normalize longitudes into [-180,180]; when the visible span exceeds ~350 deg, omit the bbox entirely. Fix (backend, defense in depth): _parse_events_params treats an out-of-range or inverted envelope as "no bbox" rather than erroring or querying a bogus envelope. Bugs 3 (FIRMS "duplicates") and 4 (missing expand buttons) from the planning walkthrough were investigated and refuted (FIRMS rows are distinct fire pixels, not satellite dupes -- dropping satellite collapses 0 rows; the expand button is present and functional on main), so they are not part of this PR. Tests: registry-derived guard that every adapter resolves bump_last_seen + base-method behavior test; 3 bbox-guard unit tests on _parse_events_params. Full suite: 634 passed, 1 skipped (central and unprivileged zvx). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 22:38:13 +00:00
# --- bump_last_seen contract (v0.7.0 Bug 1) ---------------------------------
def test_every_adapter_resolves_bump_last_seen():
"""The supervisor calls adapter.bump_last_seen on every dedup hit. Every
registered adapter must resolve a callable (inherited from SourceAdapter or
overridden) -- otherwise the AttributeError leaks to /dashboard (the eonet
bug). Registry-derived, no hardcoded list."""
missing = [
name for name, cls in discover_adapters().items()
if not callable(getattr(cls, "bump_last_seen", None))
]
assert not missing, f"adapters missing bump_last_seen: {missing}"
def test_base_bump_last_seen_updates_and_is_noop_without_db():
"""SourceAdapter.bump_last_seen updates published_ids.last_seen when a _db
handle is present, and is a safe no-op when it is not."""
import sqlite3
from central.adapter import SourceAdapter
class _StubAdapter(SourceAdapter):
name = "stub"
async def poll(self): ... # never called in this test
async def apply_config(self, new_config): ...
def subject_for(self, event): return ""
adapter = _StubAdapter()
adapter.bump_last_seen("e1") # no _db attribute -> must not raise
adapter._db = sqlite3.connect(":memory:")
adapter._db.execute(
"CREATE TABLE published_ids (adapter TEXT, event_id TEXT, "
"first_seen TIMESTAMP, last_seen TIMESTAMP)"
)
adapter._db.execute(
"INSERT INTO published_ids VALUES ('stub', 'e1', "
"'2020-01-01 00:00:00', '2020-01-01 00:00:00')"
)
adapter._db.commit()
adapter.bump_last_seen("e1")
row = adapter._db.execute(
"SELECT last_seen FROM published_ids WHERE event_id = 'e1'"
).fetchone()
assert row[0] != "2020-01-01 00:00:00" # bumped to CURRENT_TIMESTAMP