mirror of
https://github.com/zvx-echo6/central.git
synced 2026-06-10 11:54:37 +02:00
feat(state_511_atis): Castle Rock 511 adapter — Idaho incidents/closures/road work (v0.9.2)
Second CENTRAL_TRAFFIC adapter. Production code; central-supervisor + central-gui
restart (new adapter class + ADAPTER_GROUPS). No new stream -> no archive restart;
migration 026 adds the adapter row only. Ships disabled.
Two-endpoint join per layer: GET /map/mapIcons/<Layer> (markers: itemId + coords)
joined on id with POST /List/GetData/<Layer> (DataTables detail: roadwayName,
description, county, severity). The marker feed has coords but no text; the List
feed has text but no coords.
Layers -> event_types (wzdx category/subject precedent): Incidents->incident,
Closures->closure, Construction (type "Roadwork")->work_zone. category is
"<event_type>.state_511_atis"; subject central.traffic.<event_type>.<state>.
Severity 3 if isFullClosure else 1. Cadence 300s. Dedup inherited from the
v0.9.1 SourceAdapter mixin. enrichment_locations canonical (latitude,longitude)
from the marker join; county/state come upstream.
Templatized per state via settings {"states":[{code,base_url}]} but ships
Idaho-only: cross-state spot-checks refuted the shared-URL hypothesis (Oregon
TripCheck is HTML, Wyoming wyoroad 404 -- neither is Castle Rock). Add states as
settings rows once each host is verified.
Also fixes a latent test bug: test_consumer_doc per-adapter heading regex was
[a-z_]+ (no digits); state_511_atis is the first adapter name with digits, so
widened to [a-z0-9_]+.
Full suite: 759 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
efb2a5799d
commit
30e25bf475
15 changed files with 493 additions and 2 deletions
|
|
@ -1476,6 +1476,47 @@ already running can disable those overlap categories via `EONETSettings.category
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### state_511_atis — State 511 incidents / closures / road work (Castle Rock ATIS)
|
||||||
|
|
||||||
|
State-DOT 511 traffic events from the Castle Rock ATIS platform. Each layer is a
|
||||||
|
two-endpoint join: `GET /map/mapIcons/<Layer>` (markers: `itemId` + `location`
|
||||||
|
`[lat,lon]`) joined on id with `POST /List/GetData/<Layer>` (DataTables detail:
|
||||||
|
road name, description, county, severity). Verified for Idaho only.
|
||||||
|
|
||||||
|
- **Stream:** `CENTRAL_TRAFFIC`
|
||||||
|
- **Layers / event_types:** Incidents -> `incident`, Closures -> `closure`,
|
||||||
|
Construction (`type":"Roadwork"`) -> `work_zone`. (Cameras are telemetry and
|
||||||
|
ship as a separate adapter later.)
|
||||||
|
- **Subject pattern:** `central.traffic.<event_type>.<state>` (e.g.
|
||||||
|
`central.traffic.incident.id`); `<state>` is the lowercased config `code`.
|
||||||
|
- **GUI event_type:** from `category = "<event_type>.state_511_atis"` (first
|
||||||
|
dotted segment). `incident` and `closure` are new event_types (query-derived;
|
||||||
|
no hardcoded enum); `work_zone` is shared with wzdx.
|
||||||
|
- **Cadence default:** 300s (5 min).
|
||||||
|
- **Dedup key shape:** composite `<state_code>:<layer>:<id>`
|
||||||
|
(e.g. `ID:Incidents:33579`); reused as the inner `Event.id`.
|
||||||
|
- **Event.data fields:**
|
||||||
|
|
||||||
|
| key | type | nullable | description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `roadway_name` | str | yes | Road name, e.g. `US-95` |
|
||||||
|
| `description` | str | yes | Operator-readable narrative |
|
||||||
|
| `event_sub_type` | str | yes | e.g. `roadwayBlocked`, `longTermRoadConstruction` |
|
||||||
|
| `direction` | str | yes | `Both` / `North` / `Unknown` … |
|
||||||
|
| `location_description` | str | yes | Cross-street / landmark, e.g. `Five Mile Creek \| US-20` |
|
||||||
|
| `county` / `state` | str | yes | Upstream-supplied; populate the Location column |
|
||||||
|
| `start_date` / `last_updated` | str | yes | US-format local strings (no TZ; parsed naive->UTC, approximate) |
|
||||||
|
| `is_full_closure` | bool | yes | Closures only; drives severity |
|
||||||
|
| `layer` / `state_code` | str | no | Source layer + 2-letter state code (subject routing) |
|
||||||
|
| `latitude` / `longitude` | float | yes | From the marker join (enrichment input) |
|
||||||
|
|
||||||
|
- **Severity:** `is_full_closure == true` -> 3, else 1 (the upstream `severity`
|
||||||
|
string is "None" on most records; not mapped in v1).
|
||||||
|
- **Decipherable as-is:** mostly. Road + location + description + county/state are
|
||||||
|
user-ready; the geocoder fills `city` from the joined coordinates.
|
||||||
|
- **Removal semantics:** none in v1. Events age out of the upstream feed; the
|
||||||
|
14-day dedup sweep expires stale ids.
|
||||||
|
|
||||||
### wzdx — FHWA Work Zone Data Exchange (state-DOT work zones)
|
### wzdx — FHWA Work Zone Data Exchange (state-DOT work zones)
|
||||||
|
|
||||||
Active road work zones discovered from the federal WZDx Feed Registry and each
|
Active road work zones discovered from the federal WZDx Feed Registry and each
|
||||||
|
|
|
||||||
15
sql/migrations/026_add_state_511_atis_adapter.sql
Normal file
15
sql/migrations/026_add_state_511_atis_adapter.sql
Normal file
|
|
@ -0,0 +1,15 @@
|
||||||
|
-- Migration: 026_add_state_511_atis_adapter
|
||||||
|
-- Adds the state_511_atis adapter (Castle Rock ATIS 511) onto the EXISTING
|
||||||
|
-- CENTRAL_TRAFFIC stream. No new stream -> no central-archive restart needed
|
||||||
|
-- (see feedback_new_stream_needs_archive_restart). Ships disabled with the one
|
||||||
|
-- verified Idaho deployment; future verified Castle Rock states are settings rows.
|
||||||
|
-- Additive-only: idempotent via ON CONFLICT DO NOTHING.
|
||||||
|
|
||||||
|
INSERT INTO config.adapters (name, enabled, cadence_s, settings)
|
||||||
|
VALUES (
|
||||||
|
'state_511_atis',
|
||||||
|
false,
|
||||||
|
300,
|
||||||
|
'{"states": [{"code": "ID", "base_url": "https://511.idaho.gov"}]}'::jsonb
|
||||||
|
)
|
||||||
|
ON CONFLICT (name) DO NOTHING;
|
||||||
278
src/central/adapters/state_511_atis.py
Normal file
278
src/central/adapters/state_511_atis.py
Normal file
|
|
@ -0,0 +1,278 @@
|
||||||
|
"""State 511 (Castle Rock ATIS) adapter — Idaho first.
|
||||||
|
|
||||||
|
Castle Rock's ATIS platform exposes two endpoints per layer that must be joined:
|
||||||
|
- GET /map/mapIcons/<Layer> -> thin markers: {itemId, location:[lat,lon], ...}
|
||||||
|
- POST /List/GetData/<Layer> -> rich DataTables rows keyed by id==itemId
|
||||||
|
The marker feed has coordinates but no text; the List feed has road name /
|
||||||
|
description / county / severity but no coordinates. We join on id.
|
||||||
|
|
||||||
|
Layers map to traffic event_types (wzdx precedent — category drives the GUI
|
||||||
|
event_type via split_part, subject is central.traffic.<event_type>.<state>):
|
||||||
|
Incidents -> incident, Closures -> closure, Construction -> work_zone.
|
||||||
|
Cameras are telemetry (data_class) and ship as a separate adapter later.
|
||||||
|
|
||||||
|
Templatized per state via settings {"states":[{"code","base_url"}]}; only Idaho
|
||||||
|
is verified (Oregon/Wyoming are not Castle Rock). Add states as settings rows
|
||||||
|
once each host's URL shape is confirmed. Dedup is inherited from SourceAdapter.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
import sqlite3
|
||||||
|
from collections.abc import AsyncIterator
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import aiohttp
|
||||||
|
from pydantic import BaseModel
|
||||||
|
from tenacity import (
|
||||||
|
retry,
|
||||||
|
retry_if_exception_type,
|
||||||
|
stop_after_attempt,
|
||||||
|
wait_exponential_jitter,
|
||||||
|
)
|
||||||
|
|
||||||
|
from central.adapter import SourceAdapter
|
||||||
|
from central.config_models import AdapterConfig
|
||||||
|
from central.config_store import ConfigStore
|
||||||
|
from central.models import Event, Geo
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# Castle Rock layer -> Central event_type (category = "<event_type>.state_511_atis").
|
||||||
|
LAYER_EVENT_TYPE: dict[str, str] = {
|
||||||
|
"Incidents": "incident",
|
||||||
|
"Closures": "closure",
|
||||||
|
"Construction": "work_zone",
|
||||||
|
}
|
||||||
|
|
||||||
|
# DataTables server-side body. POST is required (GET returns an empty data array);
|
||||||
|
# length covers Idaho's largest layer today (~114) with headroom — warn if exceeded.
|
||||||
|
_LIST_PAGE_LENGTH = 1000
|
||||||
|
_LIST_BODY = {
|
||||||
|
"draw": "1", "start": "0", "length": str(_LIST_PAGE_LENGTH),
|
||||||
|
"columns[0][data]": "0", "order[0][column]": "0",
|
||||||
|
"order[0][dir]": "asc", "search[value]": "",
|
||||||
|
}
|
||||||
|
_XHR = {"X-Requested-With": "XMLHttpRequest"}
|
||||||
|
|
||||||
|
_FETCH_CONCURRENCY = 4
|
||||||
|
_FETCH_TIMEOUT_S = 30
|
||||||
|
|
||||||
|
_DEDUP_DDL = (
|
||||||
|
"CREATE TABLE IF NOT EXISTS published_ids ("
|
||||||
|
"adapter TEXT NOT NULL, event_id TEXT NOT NULL, "
|
||||||
|
"first_seen TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, "
|
||||||
|
"last_seen TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, "
|
||||||
|
"PRIMARY KEY (adapter, event_id))"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_us_dt(value: str | None) -> datetime | None:
|
||||||
|
"""Parse Castle Rock's US-format local timestamp (e.g. "5/25/26, 2:32 PM").
|
||||||
|
|
||||||
|
No timezone is supplied; treated as naive -> UTC (approximate — the freshness
|
||||||
|
signal is last_updated ordering, not absolute TZ). Returns None on failure.
|
||||||
|
"""
|
||||||
|
if not value:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return datetime.strptime(value.strip(), "%m/%d/%y, %I:%M %p").replace(tzinfo=timezone.utc)
|
||||||
|
except (ValueError, TypeError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class StateConfig(BaseModel):
|
||||||
|
"""One Castle Rock 511 deployment to poll."""
|
||||||
|
|
||||||
|
code: str # 2-letter state code, e.g. "ID"
|
||||||
|
base_url: str # e.g. "https://511.idaho.gov"
|
||||||
|
|
||||||
|
|
||||||
|
class State511ATISSettings(BaseModel):
|
||||||
|
"""states: verified Castle Rock deployments. Empty = nothing to poll."""
|
||||||
|
|
||||||
|
states: list[StateConfig] = []
|
||||||
|
|
||||||
|
|
||||||
|
class State511ATISAdapter(SourceAdapter):
|
||||||
|
"""Castle Rock ATIS 511 adapter (incidents / closures / construction)."""
|
||||||
|
|
||||||
|
name = "state_511_atis"
|
||||||
|
display_name = "State 511 (Castle Rock ATIS)"
|
||||||
|
description = (
|
||||||
|
"State DOT 511 incidents, closures, and road work from the Castle Rock "
|
||||||
|
"ATIS platform. Joins the map-marker and detail-list endpoints per layer. "
|
||||||
|
"Verified for Idaho; add states as settings rows once each is confirmed."
|
||||||
|
)
|
||||||
|
settings_schema = State511ATISSettings
|
||||||
|
requires_api_key = None
|
||||||
|
api_key_field = None
|
||||||
|
wizard_order = None # Ships disabled
|
||||||
|
default_cadence_s = 300
|
||||||
|
data_class = "event"
|
||||||
|
# Coords come from the marker join; geocoder fills city (county/state are upstream).
|
||||||
|
enrichment_locations = [("latitude", "longitude")]
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
config: AdapterConfig,
|
||||||
|
config_store: ConfigStore,
|
||||||
|
cursor_db_path: Path,
|
||||||
|
) -> None:
|
||||||
|
self._config_store = config_store
|
||||||
|
self._cursor_db_path = cursor_db_path
|
||||||
|
self._session: aiohttp.ClientSession | None = None
|
||||||
|
self._db: sqlite3.Connection | None = None
|
||||||
|
self._states: list[StateConfig] = self._read_states(config)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _read_states(config: AdapterConfig) -> list[StateConfig]:
|
||||||
|
raw = config.settings.get("states") or []
|
||||||
|
return [StateConfig(**s) for s in raw]
|
||||||
|
|
||||||
|
async def startup(self) -> None:
|
||||||
|
self._session = aiohttp.ClientSession(
|
||||||
|
timeout=aiohttp.ClientTimeout(total=_FETCH_TIMEOUT_S),
|
||||||
|
headers={"User-Agent": "Central/0.9 (+state_511_atis)"},
|
||||||
|
)
|
||||||
|
self._db = sqlite3.connect(self._cursor_db_path)
|
||||||
|
self._db.execute(_DEDUP_DDL)
|
||||||
|
self._db.execute("CREATE INDEX IF NOT EXISTS published_ids_last_seen ON published_ids (last_seen)")
|
||||||
|
self._db.commit()
|
||||||
|
logger.info("state_511_atis adapter started",
|
||||||
|
extra={"states": [s.code for s in self._states]})
|
||||||
|
|
||||||
|
async def shutdown(self) -> None:
|
||||||
|
if self._session:
|
||||||
|
await self._session.close()
|
||||||
|
self._session = None
|
||||||
|
if self._db:
|
||||||
|
self._db.close()
|
||||||
|
self._db = None
|
||||||
|
|
||||||
|
async def apply_config(self, new_config: AdapterConfig) -> None:
|
||||||
|
self._states = self._read_states(new_config)
|
||||||
|
logger.info("state_511_atis config updated",
|
||||||
|
extra={"states": [s.code for s in self._states]})
|
||||||
|
|
||||||
|
@retry(
|
||||||
|
stop=stop_after_attempt(3),
|
||||||
|
wait=wait_exponential_jitter(initial=1, max=30),
|
||||||
|
retry=retry_if_exception_type((aiohttp.ClientError, TimeoutError)),
|
||||||
|
)
|
||||||
|
async def _fetch_markers(self, base_url: str, layer: str) -> dict[str, tuple[float, float]]:
|
||||||
|
"""GET /map/mapIcons/<Layer> -> {itemId: (lat, lon)}."""
|
||||||
|
assert self._session is not None
|
||||||
|
async with self._session.get(f"{base_url}/map/mapIcons/{layer}") as resp:
|
||||||
|
resp.raise_for_status()
|
||||||
|
doc = await resp.json(content_type=None)
|
||||||
|
out: dict[str, tuple[float, float]] = {}
|
||||||
|
for m in (doc.get("item2") or []):
|
||||||
|
loc = m.get("location")
|
||||||
|
if isinstance(loc, list) and len(loc) == 2 and m.get("itemId") is not None:
|
||||||
|
out[str(m["itemId"])] = (float(loc[0]), float(loc[1]))
|
||||||
|
return out
|
||||||
|
|
||||||
|
async def _fetch_details(self, base_url: str, layer: str) -> list[dict[str, Any]]:
|
||||||
|
"""POST /List/GetData/<Layer> (DataTables) -> rich rows. [] on failure."""
|
||||||
|
assert self._session is not None
|
||||||
|
try:
|
||||||
|
async with self._session.post(
|
||||||
|
f"{base_url}/List/GetData/{layer}", data=_LIST_BODY, headers=_XHR
|
||||||
|
) as resp:
|
||||||
|
resp.raise_for_status()
|
||||||
|
doc = await resp.json(content_type=None)
|
||||||
|
except (aiohttp.ClientError, TimeoutError, asyncio.TimeoutError, ValueError) as exc:
|
||||||
|
logger.warning("state_511_atis detail fetch failed",
|
||||||
|
extra={"layer": layer, "base_url": base_url, "error": str(exc)})
|
||||||
|
return []
|
||||||
|
total = doc.get("recordsTotal") or 0
|
||||||
|
rows = doc.get("data") or []
|
||||||
|
if total > _LIST_PAGE_LENGTH:
|
||||||
|
logger.warning("state_511_atis layer exceeds page length; add pagination",
|
||||||
|
extra={"layer": layer, "recordsTotal": total, "length": _LIST_PAGE_LENGTH})
|
||||||
|
return rows
|
||||||
|
|
||||||
|
def _build_event(
|
||||||
|
self, detail: dict[str, Any], coords: tuple[float, float] | None,
|
||||||
|
state_code: str, layer: str,
|
||||||
|
) -> Event | None:
|
||||||
|
record_id = detail.get("id")
|
||||||
|
if record_id is None:
|
||||||
|
return None
|
||||||
|
event_type = LAYER_EVENT_TYPE[layer]
|
||||||
|
lat, lon = (coords if coords else (None, None))
|
||||||
|
return Event(
|
||||||
|
id=f"{state_code}:{layer}:{record_id}",
|
||||||
|
adapter=self.name,
|
||||||
|
category=f"{event_type}.state_511_atis",
|
||||||
|
time=(_parse_us_dt(detail.get("lastUpdated"))
|
||||||
|
or _parse_us_dt(detail.get("startDate"))
|
||||||
|
or datetime.now(timezone.utc)),
|
||||||
|
expires=_parse_us_dt(detail.get("endDate")),
|
||||||
|
severity=(3 if detail.get("isFullClosure") else 1),
|
||||||
|
geo=Geo(
|
||||||
|
centroid=(lon, lat) if lat is not None and lon is not None else None,
|
||||||
|
regions=[f"US-{state_code}"],
|
||||||
|
primary_region=f"US-{state_code}",
|
||||||
|
),
|
||||||
|
data={
|
||||||
|
"roadway_name": detail.get("roadwayName"),
|
||||||
|
"description": (detail.get("description") or "").strip() or None,
|
||||||
|
"event_sub_type": detail.get("eventSubType"),
|
||||||
|
"direction": detail.get("direction"),
|
||||||
|
"location_description": detail.get("locationDescription"),
|
||||||
|
"county": detail.get("county"),
|
||||||
|
"state": detail.get("state"),
|
||||||
|
"start_date": detail.get("startDate"),
|
||||||
|
"last_updated": detail.get("lastUpdated"),
|
||||||
|
"is_full_closure": detail.get("isFullClosure"),
|
||||||
|
"layer": layer,
|
||||||
|
"state_code": state_code,
|
||||||
|
"latitude": lat, # enrichment_locations pair (canonical)
|
||||||
|
"longitude": lon,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
async def poll(self) -> AsyncIterator[Event]:
|
||||||
|
if not self._session:
|
||||||
|
raise RuntimeError("Session not initialized")
|
||||||
|
sem = asyncio.Semaphore(_FETCH_CONCURRENCY)
|
||||||
|
|
||||||
|
async def _layer(state: StateConfig, layer: str):
|
||||||
|
async with sem:
|
||||||
|
try:
|
||||||
|
markers = await self._fetch_markers(state.base_url, layer)
|
||||||
|
except (aiohttp.ClientError, TimeoutError) as exc:
|
||||||
|
logger.warning("state_511_atis marker fetch failed",
|
||||||
|
extra={"layer": layer, "state": state.code, "error": str(exc)})
|
||||||
|
markers = {}
|
||||||
|
details = await self._fetch_details(state.base_url, layer)
|
||||||
|
return state.code, layer, markers, details
|
||||||
|
|
||||||
|
tasks = [_layer(s, layer) for s in self._states for layer in LAYER_EVENT_TYPE]
|
||||||
|
yielded = 0
|
||||||
|
for state_code, layer, markers, details in await asyncio.gather(*tasks):
|
||||||
|
for detail in details:
|
||||||
|
try:
|
||||||
|
coords = markers.get(str(detail.get("id")))
|
||||||
|
event = self._build_event(detail, coords, state_code, layer)
|
||||||
|
except Exception: # one bad record never sinks the poll
|
||||||
|
logger.exception("state_511_atis record parse failed",
|
||||||
|
extra={"layer": layer, "state": state_code})
|
||||||
|
continue
|
||||||
|
if event is None:
|
||||||
|
continue
|
||||||
|
yield event
|
||||||
|
yielded += 1
|
||||||
|
|
||||||
|
self.sweep_old_ids()
|
||||||
|
logger.info("state_511_atis poll completed", extra={"events_yielded": yielded})
|
||||||
|
|
||||||
|
def subject_for(self, event: Event) -> str:
|
||||||
|
d = event.data
|
||||||
|
event_type = LAYER_EVENT_TYPE.get(d.get("layer"), "incident")
|
||||||
|
code = (d.get("state_code") or "").lower() or "unknown"
|
||||||
|
return f"central.traffic.{event_type}.{code}"
|
||||||
|
|
@ -2651,7 +2651,7 @@ ADAPTER_GROUPS = {
|
||||||
"Space": ["swpc_alerts", "swpc_kindex", "swpc_protons"],
|
"Space": ["swpc_alerts", "swpc_kindex", "swpc_protons"],
|
||||||
"Geophysical": ["usgs_quake", "nwis"],
|
"Geophysical": ["usgs_quake", "nwis"],
|
||||||
"Earth Observation": ["eonet"],
|
"Earth Observation": ["eonet"],
|
||||||
"Transportation": ["wzdx"],
|
"Transportation": ["wzdx", "state_511_atis"],
|
||||||
}
|
}
|
||||||
# Same palette the map legend uses, indexed by sorted-adapter position.
|
# Same palette the map legend uses, indexed by sorted-adapter position.
|
||||||
EVENTS_PALETTE = [
|
EVENTS_PALETTE = [
|
||||||
|
|
|
||||||
10
src/central/gui/templates/_event_rows/state_511_atis.html
Normal file
10
src/central/gui/templates/_event_rows/state_511_atis.html
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
{# State 511 (Castle Rock ATIS) detail rows. Fields from payload->data->data;
|
||||||
|
every block guarded. direction "Unknown"/"None" suppressed (wzdx lesson). #}
|
||||||
|
{% set d = (event.data.get('data') or {}).get('data') or {} %}
|
||||||
|
{% if d.get('roadway_name') %}<dt>Road</dt><dd>{{ d.roadway_name }}{% if d.get('direction') and d.direction not in ['Unknown', 'None'] %} ({{ d.direction }}){% endif %}</dd>{% endif %}
|
||||||
|
{% if d.get('location_description') %}<dt>Location</dt><dd>{{ d.location_description }}</dd>{% endif %}
|
||||||
|
{% if d.get('event_sub_type') %}<dt>Type</dt><dd>{{ d.event_sub_type }}</dd>{% endif %}
|
||||||
|
{% if d.get('is_full_closure') %}<dt>Full closure</dt><dd>Yes</dd>{% endif %}
|
||||||
|
{% if d.get('start_date') %}<dt>Started</dt><dd>{{ d.start_date }}</dd>{% endif %}
|
||||||
|
{% if d.get('last_updated') %}<dt>Updated</dt><dd>{{ d.last_updated }}</dd>{% endif %}
|
||||||
|
{% if d.get('description') %}<dt>Description</dt><dd>{{ d.description | truncate(220) }}</dd>{% endif %}
|
||||||
|
|
@ -0,0 +1,7 @@
|
||||||
|
{# State 511 (Castle Rock ATIS) one-line subject. Lead label by layer + road +
|
||||||
|
cross-street location; county/state render in the generic Location column
|
||||||
|
(it falls back to d.county/d.state). Fields from payload->data->data. #}
|
||||||
|
{% set d = (event.data.get('data') or {}).get('data') or {} %}
|
||||||
|
{%- set labels = {'Incidents': 'Incident', 'Closures': 'Closure', 'Construction': 'Road work'} -%}
|
||||||
|
{%- set lead = labels.get(d.get('layer'), 'Traffic event') -%}
|
||||||
|
{{ lead }}{% if d.get('roadway_name') %} on {{ d.roadway_name }}{% endif %}{% if d.get('location_description') %} — {{ d.location_description }}{% endif %}
|
||||||
1
tests/fixtures/state_511_atis_closures.json
vendored
Normal file
1
tests/fixtures/state_511_atis_closures.json
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
{"data":[{"DT_RowId":"17","tooltipUrl":"/tooltip/Closures/17?lang=%7Blang%7D&noCss=true","id":17,"type":"Closures","layerName":"Closures","roadwayName":"N McDermott Rd","description":" Long term road construction on N McDermott Rd Both Directions from Five Mile Creek to US-20. All lanes closed. 1/30/2023 2:24 PM Mon, Tue, Wed, Thu, Fri, Sat, Sun: Active all day<div class='cellSpacer'><i><b>Comments:</b></i> Open to local traffic only.</div>","sourceId":"469","source":"ERS","comment":"Open to local traffic only.","eventSubType":"longTermRoadConstruction","startDate":"1/30/23, 2:24 PM","endDate":null,"lastUpdated":"6/18/24, 6:48 PM","isFullClosure":true,"severity":"None","direction":"Both","locationDescription":"Five Mile Creek | US-20","detourDescription":null,"laneDescription":"All lanes closed","recurrenceDescription":"<b>Mon, Tue, Wed, Thu, Fri, Sat, Sun:</b><br/>Active all day<br/><br/>","widthRestriction":null,"heightRestriction":null,"heightUnderRestriction":null,"lengthRestriction":null,"weightRestriction":null,"majorEvent":null,"county":"Ada","region":"Idaho Falls/Pocatello","state":"Idaho","country":"United States","showOnMap":true,"restrictions":"Width Restriction: <br>Height Restriction: <br>Length Restriction: <br>Weight Restriction: <br>Speed Restriction: <br>"}]}
|
||||||
1
tests/fixtures/state_511_atis_construction.json
vendored
Normal file
1
tests/fixtures/state_511_atis_construction.json
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
{"data":[{"DT_RowId":"23","tooltipUrl":"/tooltip/Construction/23?lang=%7Blang%7D&noCss=true","id":23,"type":"Roadwork","layerName":"Construction","roadwayName":"SH-81","description":" Work on the shoulder on SH-81 near Poverty Gulch. 7/8/2024 9:29 PM Mon: 12:00 PM - 5:00 PM, Tue, Wed, Thu, Fri, Sat, Sun: Active all day Activities: use caution, warning.","sourceId":"4277","source":"ERS","comment":null,"eventSubType":"workOnTheShoulder","startDate":"7/8/24, 9:29 PM","endDate":null,"lastUpdated":"6/11/25, 3:02 PM","isFullClosure":false,"severity":"None","direction":"Unknown","locationDescription":"Poverty Gulch","detourDescription":null,"laneDescription":"","recurrenceDescription":"<b>Mon:</b><br/>12:00 PM - 5:00 PM<br/><br/><b>Tue, Wed, Thu, Fri, Sat, Sun:</b><br/>Active all day<br/><br/>","widthRestriction":null,"heightRestriction":null,"heightUnderRestriction":null,"lengthRestriction":null,"weightRestriction":null,"majorEvent":null,"county":"Cassia","region":"Idaho Falls/Pocatello","state":"Idaho","country":"United States","showOnMap":true,"restrictions":"Width Restriction: <br>Height Restriction: <br>Length Restriction: <br>Weight Restriction: <br>Speed Restriction: <br>"}]}
|
||||||
1
tests/fixtures/state_511_atis_incidents.json
vendored
Normal file
1
tests/fixtures/state_511_atis_incidents.json
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
{"data":[{"DT_RowId":"33579","tooltipUrl":"/tooltip/Incidents/33579?lang=%7Blang%7D&noCss=true","id":33579,"type":"Incidents","layerName":"Incidents","roadwayName":"US-95","description":" Roadway Blocked on US-95 Both Directions near MM (469). All lanes blocked. Activities: Expect Delays, Reduced to Single Lane, Alternating Direction of Travel, Use Caution.<div class='cellSpacer'><i><b>Comments:</b></i> Milepost 469, roadway blocked. Expect delays. Use caution.</div>","sourceId":"10991","source":"ERS","comment":"Milepost 469, roadway blocked. Expect delays. Use caution.","eventSubType":"roadwayBlocked","startDate":"5/25/26, 2:32 PM","endDate":null,"lastUpdated":"5/25/26, 3:40 PM","isFullClosure":false,"severity":"None","direction":"Both","locationDescription":"Ponderosa Mobile Home Park","detourDescription":null,"laneDescription":"All lanes blocked","recurrenceDescription":null,"widthRestriction":null,"heightRestriction":null,"heightUnderRestriction":null,"lengthRestriction":null,"weightRestriction":null,"majorEvent":null,"county":"Bonner","region":"Idaho Falls/Pocatello","state":"Idaho","country":"United States","showOnMap":true,"restrictions":"Width Restriction: <br>Height Restriction: <br>Length Restriction: <br>Weight Restriction: <br>Speed Restriction: <br>"}]}
|
||||||
1
tests/fixtures/state_511_atis_markers_closures.json
vendored
Normal file
1
tests/fixtures/state_511_atis_markers_closures.json
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
{"item2":[{"polyline":{"path":"qbliGhv{eUsk@MeTEsBAsAGiAQw@Qi@MoAYmA]aA]}@a@[?[JSNKNENCTEzCGnAIX","color":"#CC0004"},"itemId":"17","location":[43.6485700000001,-116.47349],"icon":{"url":"/Generated/Content/Images/511/map_closure.svg"},"title":""}]}
|
||||||
1
tests/fixtures/state_511_atis_markers_construction.json
vendored
Normal file
1
tests/fixtures/state_511_atis_markers_construction.json
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
{"item2":[{"itemId":"23","location":[42.5168038430856,-113.711287649613],"icon":{},"title":""}]}
|
||||||
1
tests/fixtures/state_511_atis_markers_incidents.json
vendored
Normal file
1
tests/fixtures/state_511_atis_markers_incidents.json
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
{"item2":[{"itemId":"33579","location":[48.2055675659533,-116.563364000498],"icon":{},"title":""}]}
|
||||||
|
|
@ -59,7 +59,7 @@ def _per_adapter_subsections(doc: str) -> list[str]:
|
||||||
assert m, "doc missing '## 6. Per-adapter reference' section"
|
assert m, "doc missing '## 6. Per-adapter reference' section"
|
||||||
section = m.group(1)
|
section = m.group(1)
|
||||||
|
|
||||||
heading_re = re.compile(r"^### ([a-z_]+) — ", re.MULTILINE)
|
heading_re = re.compile(r"^### ([a-z0-9_]+) — ", re.MULTILINE)
|
||||||
return heading_re.findall(section)
|
return heading_re.findall(section)
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1143,6 +1143,7 @@ _SAMPLE_INNER = {
|
||||||
"wfigs_incidents": {"county": "Montezuma", "state": "CO"},
|
"wfigs_incidents": {"county": "Montezuma", "state": "CO"},
|
||||||
"wfigs_perimeters": {"county": "Carbon", "state": "MT"},
|
"wfigs_perimeters": {"county": "Carbon", "state": "MT"},
|
||||||
"wzdx": {"road_names": ["I-80"], "direction": "eastbound"},
|
"wzdx": {"road_names": ["I-80"], "direction": "eastbound"},
|
||||||
|
"state_511_atis": {"layer": "Incidents", "roadway_name": "US-95", "location_description": "Ponderosa Mobile Home Park"},
|
||||||
}
|
}
|
||||||
|
|
||||||
# Exact expected subjects for the deterministic adapters. swpc_alerts is omitted
|
# Exact expected subjects for the deterministic adapters. swpc_alerts is omitted
|
||||||
|
|
|
||||||
133
tests/test_state_511_atis.py
Normal file
133
tests/test_state_511_atis.py
Normal file
|
|
@ -0,0 +1,133 @@
|
||||||
|
"""Tests for the state_511_atis adapter (Castle Rock ATIS, Idaho).
|
||||||
|
|
||||||
|
Fixtures are real captures (one record + its matching marker per layer):
|
||||||
|
state_511_atis_<layer>.json -- POST /List/GetData/<Layer> .data[0:1]
|
||||||
|
state_511_atis_markers_<layer>.json -- GET /map/mapIcons/<Layer> matching item2
|
||||||
|
|
||||||
|
No tests/conftest isolation entry is added: dedup uses the supervisor-injected
|
||||||
|
cursors.db (inherited mixin) and discovery is stateless -- no adapter-owned cache.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import MagicMock
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from central.adapter import SourceAdapter
|
||||||
|
from central.adapters.state_511_atis import (
|
||||||
|
LAYER_EVENT_TYPE,
|
||||||
|
State511ATISAdapter,
|
||||||
|
_parse_us_dt,
|
||||||
|
)
|
||||||
|
from central.config_models import AdapterConfig
|
||||||
|
|
||||||
|
FIX = Path(__file__).parent / "fixtures"
|
||||||
|
DETAIL = {lyr: json.loads((FIX / f"state_511_atis_{lyr.lower()}.json").read_text())["data"][0]
|
||||||
|
for lyr in ("Incidents", "Closures", "Construction")}
|
||||||
|
MARK = {lyr: json.loads((FIX / f"state_511_atis_markers_{lyr.lower()}.json").read_text())["item2"][0]
|
||||||
|
for lyr in ("Incidents", "Closures", "Construction")}
|
||||||
|
|
||||||
|
|
||||||
|
def _coords(layer):
|
||||||
|
loc = MARK[layer]["location"]
|
||||||
|
return (loc[0], loc[1])
|
||||||
|
|
||||||
|
|
||||||
|
def _cfg():
|
||||||
|
return AdapterConfig(
|
||||||
|
name="state_511_atis", enabled=True, cadence_s=300,
|
||||||
|
settings={"states": [{"code": "ID", "base_url": "https://511.idaho.gov"}]},
|
||||||
|
updated_at=datetime.now(timezone.utc),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def adapter(tmp_path):
|
||||||
|
return State511ATISAdapter(_cfg(), MagicMock(), tmp_path / "cursors.db")
|
||||||
|
|
||||||
|
|
||||||
|
def test_layer_event_type_map():
|
||||||
|
assert LAYER_EVENT_TYPE == {"Incidents": "incident", "Closures": "closure", "Construction": "work_zone"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_us_dt():
|
||||||
|
assert _parse_us_dt("5/25/26, 2:32 PM") == datetime(2026, 5, 25, 14, 32, tzinfo=timezone.utc)
|
||||||
|
assert _parse_us_dt("") is None
|
||||||
|
assert _parse_us_dt("not a date") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_dedup_key(adapter):
|
||||||
|
e = adapter._build_event(DETAIL["Incidents"], _coords("Incidents"), "ID", "Incidents")
|
||||||
|
assert e.id == "ID:Incidents:33579"
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_incident(adapter):
|
||||||
|
e = adapter._build_event(DETAIL["Incidents"], _coords("Incidents"), "ID", "Incidents")
|
||||||
|
assert e.category == "incident.state_511_atis"
|
||||||
|
assert e.severity == 1
|
||||||
|
assert e.data["roadway_name"] == "US-95"
|
||||||
|
assert e.data["county"] == "Bonner"
|
||||||
|
assert e.data["latitude"] is not None and e.data["longitude"] is not None
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_closure_full_closure_severity(adapter):
|
||||||
|
e = adapter._build_event(DETAIL["Closures"], _coords("Closures"), "ID", "Closures")
|
||||||
|
assert e.category == "closure.state_511_atis"
|
||||||
|
assert e.data["is_full_closure"] is True
|
||||||
|
assert e.severity == 3 # isFullClosure -> 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_construction_maps_to_work_zone(adapter):
|
||||||
|
e = adapter._build_event(DETAIL["Construction"], _coords("Construction"), "ID", "Construction")
|
||||||
|
assert e.category == "work_zone.state_511_atis" # layer Construction, type "Roadwork"
|
||||||
|
assert e.severity == 1
|
||||||
|
assert e.data["roadway_name"] == "SH-81"
|
||||||
|
|
||||||
|
|
||||||
|
def test_join_missing_coords(adapter):
|
||||||
|
e = adapter._build_event(DETAIL["Incidents"], None, "ID", "Incidents")
|
||||||
|
assert e.data["latitude"] is None and e.data["longitude"] is None
|
||||||
|
assert e.geo.centroid is None # still built, just no map point
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("layer,et", [("Incidents", "incident"), ("Closures", "closure"), ("Construction", "work_zone")])
|
||||||
|
def test_subject_for(adapter, layer, et):
|
||||||
|
e = adapter._build_event(DETAIL[layer], _coords(layer), "ID", layer)
|
||||||
|
assert adapter.subject_for(e) == f"central.traffic.{et}.id"
|
||||||
|
|
||||||
|
|
||||||
|
def test_summary_partial_renders():
|
||||||
|
from central.gui.routes import _derive_subject
|
||||||
|
inner = {"layer": "Incidents", "roadway_name": "US-95", "location_description": "Ponderosa Mobile Home Park"}
|
||||||
|
row = {"adapter": "state_511_atis", "data": {"data": {"data": inner}}}
|
||||||
|
assert _derive_subject(row) == "Incident on US-95 — Ponderosa Mobile Home Park"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_poll_joins_and_yields(adapter):
|
||||||
|
await adapter.startup()
|
||||||
|
|
||||||
|
async def fake_markers(base_url, layer):
|
||||||
|
m = MARK[layer]
|
||||||
|
return {str(m["itemId"]): (m["location"][0], m["location"][1])}
|
||||||
|
|
||||||
|
async def fake_details(base_url, layer):
|
||||||
|
return [DETAIL[layer]]
|
||||||
|
|
||||||
|
adapter._fetch_markers = fake_markers
|
||||||
|
adapter._fetch_details = fake_details
|
||||||
|
events = [e async for e in adapter.poll()]
|
||||||
|
await adapter.shutdown()
|
||||||
|
assert len(events) == 3 # one ID state x three layers
|
||||||
|
assert {e.category for e in events} == {
|
||||||
|
"incident.state_511_atis", "closure.state_511_atis", "work_zone.state_511_atis",
|
||||||
|
}
|
||||||
|
assert all(e.adapter == "state_511_atis" for e in events)
|
||||||
|
|
||||||
|
|
||||||
|
def test_inherits_dedup_mixin():
|
||||||
|
for m in ("is_published", "mark_published", "sweep_old_ids"):
|
||||||
|
assert m not in State511ATISAdapter.__dict__, f"redefines {m}"
|
||||||
|
assert getattr(State511ATISAdapter, m) is getattr(SourceAdapter, m)
|
||||||
Loading…
Add table
Add a link
Reference in a new issue