Final structural step. central_normalizer.py (973 LOC, misnamed — it lived
OUTSIDE central/ but its docstring described "the Central firehose") is DELETED.
No file or symbol named "central" survives (only immutable historical comments
in an already-applied migration .sql remain — those are not edited).
Split per the owner's rule ("next to what uses it"):
- Live geo/place-name utilities (nearest_town, _haversine_miles, _bearing_compass,
_compute_distance_bearing, _h3_cell, _photon_reverse_places, geocoder config)
→ the EXISTING meshai/geo.py ("Geographic utilities…"), same domain. NOT a
junk drawer. geo.py's nearest_city (hardcoded table) and central_normalizer's
nearest_town (live Photon+H3 reverse-geocode) are different implementations —
both kept, no false consolidation.
- WZDx work-zone parsers (_parse_wzdx_federal, _norm_wzdx_sub_type, _norm_direction,
_parse_mile_posts, _clean_description, _is_uninformative_road) → env/wzdx_parse.py,
next to their sole consumer env/wzdx.py.
- ~400 dead lines deleted: the Central-envelope machinery (normalize,
_parse_state_511_atis, _parse_wfigs_incidents, _parse_itd_511_work_zone,
should_skip_state_511_atis_id, is_incident_envelope_stale, normalize_road_name,
_norm_sub_type, _parse_ends_at) — only caller was the deleted Central consumer.
Consumers rewired: env/wzdx.py, env/fire_render.py, formatters/fire.py,
formatters/_anchor.py, main.py, persistence/curation.py, composer.py.
Tests: test_central_normalizer.py RENAMED → test_geo_wzdx_parse.py (live geo/wzdx
tests kept + rewired; dead-parser tests dropped). test_itd_511_work_zone.py
deleted (tested only the dead itd_511 parser — verified gone from source).
Other test files: import-path rewires. Live coverage for nearest_town /
_parse_wzdx_federal / bearing preserved across test_geo_wzdx_parse, test_adapter_wzdx,
test_wfigs_handler, test_incident_refactor.
Suite: 2010 passed, 0 failed. The 49-test drop is the dead Central-envelope
parser tests, correctly removed. Stale "Central firehose" docstrings on the
moved code updated to describe present behavior.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both were fully plumbed and read by nothing.
api_key -- self-documented as dead at config.py ("Keyless: api_key is
retained but unused"), yet wired end-to-end: a GUI ManagedSecret field, a
SECRET_FIELDS entry, an EXPECTED_SECRETS entry, a secrets_store mapping and
label, and a line in .env.example. env/wzdx.py assigned self._api_key and
never read it again. So an operator could go get an API key, paste it into
the secure secrets manager, and have it do precisely nothing -- the ritual
looked complete end to end, which is what made it worth removing rather
than leaving.
endpoints -- default ["/get/event"], exposed as an editable list in the
dashboard, never read. Copy-paste from Roads511Config.endpoints (which IS
read, at env/roads511.py; Roads511 is untouched here). WZDx discovers feeds
via the FHWA registry_url/states instead.
WZDx's actual fetch behavior is unchanged; this removes dead config only.
Note for existing installs: anyone with WZDX_API_KEY set in
/data/secrets/.env will simply have an ignored env var. Harmless -- it was
already ignored.
Suite: 2337 passed, 6 failed (the pre-existing set: stale SCHEMA_VERSION x3,
expired TLE fixtures x2, one order-dependent), 72 skipped -- exact baseline
match, no new failures.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1. WZDx work zones: replace per-event broadcasting with a once-a-day
per-region count summary. Coalesce the upstream per-direction /
per-schedule-day fan-out into one row per physical zone
(road + lat3 + lon3 + sub_type); work zones are stored in
traffic_events but no longer per-event broadcast, while 511 crash /
closure / hazard incidents still broadcast live. A WZDxSummaryScheduler
emits one count line per coverage region once a day (default 07:00
America/Boise), routed via the region_routes 'roads' cells; work-zone
details are DM-queryable (build_work_zones_detail). New config:
wzdx.summary_enabled / summary_time / summary_tz.
2. FIRMS cold-start is now restart-safe: gate the silent-seed on the
persisted firms_pixels baseline being empty (first-ever run) instead of
an in-memory per-boot flag, so a restart no longer silently absorbs a
genuinely-new hotspot cluster.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two bugs found running native adapters against real upstreams:
- env/wzdx.py: _select_feeds() called .strip() on the registry url field,
but Socrata "URL"-column values arrive as {"url": "..."} — crashed ALL
native WZDx discovery with AttributeError. Added _unwrap_url() (dict/str/
None-robust), applied to url/apiurl/feed_url.
- composer._context_segment: appended optional fields by key presence, so
cause: None (set by native road adapters) leaked literal "None" onto the
wire. Guard on value (cause/expires_at truthiness; containment_pct is-not-
None so 0% still renders). Legacy Mode-B path — golden tests unchanged.
+3 regression tests; golden/composer suites pass; full suite 10-failure
baseline (1685 passed).
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New env/wzdx.py native adapter so meshai has a LOCAL source for the
work_zone category instead of Central-only. Keyless: discovers per-state
WZDx feeds from the FHWA registry (datahub.transportation.gov), filtered to
configured states (default [ID]), fetches each WZDx v4 GeoJSON feed, and
parses road_events into canonical work_zone Event.data.
- reuses central_normalizer._parse_wzdx_federal for identical field mapping
→ renders through the existing Phase-2 formatters/gating incident path
(no new formatter/gating); verified end-to-end to a valid work-zone wire
- stable external_id = {data_source_id}:{feature_id} for incident dedup
- registry cached 6h; per-feed failures logged + skipped (never crash)
- WZDxConfig extended (registry_url, registry_ttl, states); registered in
env/store.py; enabled=false + feed_source=native default (dormant until
turned on at the coordinated flip)
19 new tests; suite at the 10-failure baseline (1640 passed). Only env/wzdx.py
(new), env/store.py, config.py, tests/test_adapter_wzdx.py touched.
Co-authored-by: Matt Johnson <mj@k7zvx.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
