feat(telemetry-separation): dedicated /telemetry tab split from /events by adapter data_class (v0.7.4)
PR #5 of the v0.7.x GUI rework arc. Production code; central-gui restart only
(supervisor untouched -- data_class is read only by central-gui per request).
- SourceAdapter gains a `data_class` class attr (Literal["event","telemetry"],
default "event"). NWIS opts in as "telemetry" (continuous high-volume water
gauges); every other adapter stays "event". The /events vs /telemetry split is
thus registry-derived from class attrs -- no hardcoded adapter-name lists.
- routes.py refactor: `_class_adapter_names(data_class)` and a `data_class` arg
on `_adapter_filter_options` scope the flat + domain-grouped chip/legend lists
to a class (colors stay keyed to the FULL registry, so an adapter keeps one
color across tabs). `_fetch_events` accepts `class_adapters` and adds an
`adapter = ANY(...)` condition. Shared `_events_query`, `_events_page(data_class,
base_path)` and `_events_rows_fragment(...)` back both tabs; `/events`,
`/events/rows`, `/telemetry`, `/telemetry/rows` are thin wrappers.
- Templates parameterized with a `base_path` context var (form action, hx-get,
hx-push-url header, clear-all redirect, JS BASE_PATH const); the `_events_rows`
paginator macro takes `base`. Same templates serve both tabs; nav gains a
Telemetry link.
- /events.json UNCHANGED -- the cursor path sets no `class_adapters`, so the
subject + pagination contract is intact (TestEventsJsonSubject still passes).
Adds TestTelemetrySeparation (data_class defaults, registry split 11 event / 1
telemetry, class-scoped filter options, color stability, and the `adapter =
ANY(...)` SQL shape incl. the no-class events.json path). Updates the events
frontend tests for the base_path-parameterized templates.
Full suite: 682 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 07:34:08 +00:00
|
|
|
"""Tests for v0.7.4 telemetry/event separation: SourceAdapter.data_class,
|
|
|
|
|
registry split, class-scoped filter options, and the data_class SQL filter.
|
|
|
|
|
Registry-derived (no hardcoded adapter lists beyond the nwis pin). No live DB.
|
|
|
|
|
"""
|
|
|
|
|
from unittest.mock import AsyncMock, MagicMock, patch
|
|
|
|
|
|
|
|
|
|
import pytest
|
|
|
|
|
|
|
|
|
|
from central.adapter import SourceAdapter
|
|
|
|
|
from central.adapter_discovery import discover_adapters
|
|
|
|
|
from central.gui import routes
|
|
|
|
|
|
feat(tomtom_flow): TomTom Orbis vector flow-tile telemetry adapter + CENTRAL_TRAFFIC_FLOW (v0.9.3)
Third CENTRAL_TRAFFIC-family member, first telemetry traffic source. Polls a
configured tile coverage set (Idaho metros, z=10), fetches Orbis vector flow
tiles, decodes per-segment relative_speed + road geometry, emits one telemetry
Event per road segment per poll to the new CENTRAL_TRAFFIC_FLOW stream. Renders
as colored polylines (green free-flow -> red jam) on the /telemetry map.
Production code; supervisor + gui + ARCHIVE restart (NEW event-bearing stream
central.traffic_flow.> -> archive must resubscribe). Ships disabled; needs a
"tomtom" api key in config.api_keys before enable.
- Subject central.traffic_flow.{z}.{x}.{y} (token traffic_flow, non-overlapping
with central.traffic.>). category="flow.tomtom_flow" -> GUI event_type "flow".
- Severity from relative_speed: >=0.75=1, 0.5-0.75=2, 0.25-0.5=3, <0.25=4.
- Cadence 300s; 7-day retention (high-volume telemetry). Dedup minute-bucketed,
inherited from the v0.9.1 SourceAdapter mixin.
- Shared tomtom_flow_parse module (decode + slippy-tile georeference) reused by
the v0.9.4 on-demand passthrough endpoint.
- Generic framework change (Option A, ~3 lines, inert for the other 14
adapters): Geo.geometry optional field + archive _build_geom_sql prefers it,
so segments persist their real LineString to the PostGIS geom column.
- Idaho-only (Orbis tier confirmed live). Cameras + Navi passthrough are follow-ups.
- deps: mapbox-vector-tile (vector PBF decode); itsdangerous promoted to an
explicit dependency (gui/csrf.py + gui/wizard.py imported it as an undeclared
transitive that uv re-lock would otherwise prune).
Full suite: 780 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 23:25:44 +00:00
|
|
|
# Adapters with data_class="telemetry" (the pinned split; grow as telemetry adapters land).
|
2026-06-09 00:54:19 -06:00
|
|
|
# v0.11.0 added celestrak_tle (orbital state -- continuous-ish refresh, telemetry-class).
|
v0.12.0: sat_positions adapter (live global satellite positions) + sat_common refactor
## Architectural framing
The v0.11.1 `satpass_predict` adapter is **observer-anchored**: "when does satellite X pass over fixed observer Y, and what's the elevation/azimuth at that observer's site?" It answers a fixed-QTH question and emits one event per (observer, satellite, AOS) tuple.
The new `sat_positions` adapter is the **global** counterpart: "where is satellite X right now?" No observer. One event per tracked NORAD ID per poll, on subject `central.sat.position.<norad_id>`. Consumers (meshAI, GUI map widgets, anything that wants a live world map) subscribe to `central.sat.position.>` and plot.
They complement each other; neither replaces the other.
Direct quote from Matt's use-case: *"location of the sats... map of where the sats are then we have meshai or whatever other service calling central's data grab it and do whatever work it needed."* This adapter is that.
## sat_common extraction rationale
The four pure SGP4 / coordinate helpers (`EARTH_RADIUS_KM`, `gmst_rad`, `eci_to_ecef`, `subsatellite_point`) were private symbols inside `satpass_predict.py`. `sat_positions` needs the same three helpers. Three options were considered:
1. **Cross-import** from `satpass_predict.py` — creates an adapter-to-adapter dependency, ugly.
2. **Extract to `sat_common.py`** — matches the existing `wfigs_common.py` / `swpc_common.py` precedent. Both adapters become siblings of a shared helper module. ✓ chosen.
3. **Duplicate** — math drift over time.
Symbol names dropped their leading underscore on extraction (public-API convention matching `swpc_common.parse_swpc_timestamp` / `wfigs_common.severity_from_acres`). Existing internal call sites in `satpass_predict.py` were updated via mechanical `replace_all`. Observer-specific helpers (`_observer_ecef`, `_topocentric_az_el`, `_visibility_footprint`, `_severity_from_elev`, `_build_pass_geometry`, `_next_passes`) stay in `satpass_predict.py` per YAGNI — they're not used by `sat_positions` today.
Existing `tests/test_satpass_predict.py` was updated mechanically to import the helpers from `sat_common` via aliases (preserves the underscore-prefixed local names in the tests so the rest of the test body needs no change). All 44 satpass_predict tests pass unchanged.
## CENTRAL_SAT stream cap bump
`config.streams.max_bytes` for `CENTRAL_SAT` goes from **1 GiB → 5 GiB** in migration 039. Sizing math:
- celestrak_tle: ~190 sats × 1 envelope/day = ~190 events/day = ~1.4k events/week. Fit in 1 GiB easily.
- sat_positions: ~190 sats × 1440 ticks/day (60s cadence) = **~273.6k events/day = ~1.9M events/week**. At ~1 KB per envelope including the CloudEvents wrapper, that's **~1.9 GiB/week**.
- Plus existing TLE + pass envelopes already on the stream → ~3 GiB headroom needed.
- 5 GiB = 5368709120 bytes = operator-tunable margin without over-provisioning.
`STREAM_CATEGORY_DOMAINS["CENTRAL_SAT"]` extends from `("tle", "pass")` to `("tle", "pass", "position")` so the supervisor's retention sweep covers position events too.
## Subject + dedup
| Field | Value |
|---|---|
| Subject | `central.sat.position.<norad_id>` — one subject per satellite, globally |
| Dedup id | `<norad_id>:<position_iso>` where `position_iso` is the propagation timestamp truncated to whole seconds (defensive collapse if cadence is ever tightened) |
| Severity | 1 (informational telemetry, no alerting) |
| data_class | `telemetry` — surfaces on `/telemetry`, not `/events` |
| Cadence | 60s default; operator-tunable via standard `cadence_s` field |
## Settings shape
```json
{"track_only_norad_ids": [], "max_tle_age_days": 14}
```
- Empty `track_only_norad_ids` = track every NORAD ID with a fresh TLE in the events table (derive-from-celestrak_tle, default behavior).
- Non-empty list pins to those NORAD IDs only (operator override — "I only care about the ISS and these 12 Starlink sats").
- `max_tle_age_days` bounds TLE freshness; LEO drag means TLEs go stale in days, GEO is good for months. Parameterized into the SQL query as a timedelta interval so operator-tightened windows (e.g. 3d) apply without code change.
## Event.data fields
`norad_id`, `satellite_name`, `lon_deg`, `lat_deg`, `alt_km`, `velocity_kmps`, `heading_deg`, `tle_epoch`.
- `lon_deg`/`lat_deg`/`alt_km`: sub-satellite point via SGP4 → ECI → ECEF rotation → spherical-earth lon/lat/alt.
- `velocity_kmps`: magnitude of the SGP4 ECI velocity vector. ECI vs ECEF difference is ~6% for LEO (earth rotation 0.46 km/s vs 7.7 km/s orbital speed); fine for consumer "the sat is moving at X km/s" text.
- `heading_deg`: great-circle initial bearing from the sub-sat point at `t` to the sub-sat point at `t+1s` (finite-difference; simpler than rotating velocity through GMST + the earth-rotation cross term).
## Diff size — flag for review
**+894 / -63 = +831 net** across 14 files. Spec budget was ≤700 lines. **Over by ~131 net** (or ~194 gross).
Breakdown:
- `sat_positions.py`: 286 lines (under the ≤350 adapter line cap ✓)
- `sat_common.py`: 65 lines (the extraction)
- Migration 039: 58 lines (heavy on inline comments documenting the size math; could trim ~25 lines if you want)
- satpass_predict.py: net -1 line (refactor; lost 4 helper defs and one constant comment, gained 5-line import block)
- Templates: 14 lines (event_rows + event_summaries partials)
- Wiring: 4 lines (supervisor + ADAPTER_GROUPS)
- Docs (CONSUMER-INTEGRATION.md): 40 lines (required by `tests/test_consumer_doc.py::test_every_adapter_has_a_subsection`)
- **Tests: 426 lines.** This is the bulk of the overage.
The tests are all spec-mandated (sub-sat math, velocity range, heading range, build_event, subject_for, empty-TLE, track_only gate, stale-TLE skip, sat_common helpers, regression-guard on the moved helpers via test_satpass_predict.py preservation). I could shrink `test_sat_positions.py` by consolidating the 11 spec-mandated tests into fewer parameterized cases, but each test pins one behavior the spec called out by name. Flagging for your call: keep as-is, or do you want a tighter parameterized version?
## Test plan
- [x] `pytest tests/test_sat_common.py tests/test_sat_positions.py` — **28 new tests, all pass**.
- [x] `pytest tests/test_satpass_predict.py` — **44/44 pass** (regression guard: existing tests work after the sat_common extraction).
- [x] `pytest tests/test_events_feed_frontend.py` — **119/119 pass** (JSON-feed coverage extended to include sat_positions sample event + expected subject string).
- [x] `pytest tests/test_telemetry_separation.py` — **9/9 pass** (`_TELEMETRY` pin extended to include `sat_positions`).
- [x] `pytest tests/test_consumer_doc.py` — **6/6 pass** (CONSUMER-INTEGRATION.md `### sat_positions` subsection added).
- [x] `pytest tests/test_producer_doc.py` — **10/10 pass** (no PRODUCER-INTEGRATION update needed; CENTRAL_SAT stream is pre-existing).
- [x] Full sweep `pytest tests/` (excluding postgres-dep files): **1209 passed, 1 skipped, 0 failures**.
- [x] Ruff: clean on all new code. 3 pre-existing F841 unused-variable warnings (supervisor.py:390 `poll_start`, test_events_feed_frontend.py:425 / :466 `result`) confirmed via `git blame` to be from commits May 2026 — not introduced.
## Deploy plan
1. Squash-merge → tag v0.12.0 at merge SHA → push tag.
2. `ssh central`, `git pull` on `/opt/central`. **No `uv sync`** (no new dep).
3. **`central-migrate`** to apply migration 039 (seeds `config.adapters` row + bumps `config.streams.max_bytes` for CENTRAL_SAT).
4. `sudo systemctl restart central-supervisor` (picks up STREAM_CATEGORY_DOMAINS extension + new adapter discovery).
5. `sudo systemctl restart central-gui` (picks up new partials + ADAPTER_GROUPS change).
6. **No** `central-archive` restart (CENTRAL_SAT stream already exists; no new stream).
7. Verify: `nats stream info CENTRAL_SAT` shows max_bytes=5368709120; supervisor journal shows sat_positions discovered.
8. Smoke-test: enable celestrak_tle first if not already, wait for one poll, then enable sat_positions via GUI. Within 60s expect one `central.sat.position.<norad_id>` event per tracked sat on the stream.
## Halt acknowledgment
Per spec acceptance bar #6: **squash-merge NOT authorized**. Branch + PR open. Halting for line-by-line review.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-06-09 15:23:32 -06:00
|
|
|
# v0.12.0 added sat_positions (60s sub-sat point per tracked satellite).
|
|
|
|
|
_TELEMETRY = ["celestrak_tle", "itd_511_cameras", "nwis", "sat_positions", "tomtom_flow"]
|
feat(tomtom_flow): TomTom Orbis vector flow-tile telemetry adapter + CENTRAL_TRAFFIC_FLOW (v0.9.3)
Third CENTRAL_TRAFFIC-family member, first telemetry traffic source. Polls a
configured tile coverage set (Idaho metros, z=10), fetches Orbis vector flow
tiles, decodes per-segment relative_speed + road geometry, emits one telemetry
Event per road segment per poll to the new CENTRAL_TRAFFIC_FLOW stream. Renders
as colored polylines (green free-flow -> red jam) on the /telemetry map.
Production code; supervisor + gui + ARCHIVE restart (NEW event-bearing stream
central.traffic_flow.> -> archive must resubscribe). Ships disabled; needs a
"tomtom" api key in config.api_keys before enable.
- Subject central.traffic_flow.{z}.{x}.{y} (token traffic_flow, non-overlapping
with central.traffic.>). category="flow.tomtom_flow" -> GUI event_type "flow".
- Severity from relative_speed: >=0.75=1, 0.5-0.75=2, 0.25-0.5=3, <0.25=4.
- Cadence 300s; 7-day retention (high-volume telemetry). Dedup minute-bucketed,
inherited from the v0.9.1 SourceAdapter mixin.
- Shared tomtom_flow_parse module (decode + slippy-tile georeference) reused by
the v0.9.4 on-demand passthrough endpoint.
- Generic framework change (Option A, ~3 lines, inert for the other 14
adapters): Geo.geometry optional field + archive _build_geom_sql prefers it,
so segments persist their real LineString to the PostGIS geom column.
- Idaho-only (Orbis tier confirmed live). Cameras + Navi passthrough are follow-ups.
- deps: mapbox-vector-tile (vector PBF decode); itsdangerous promoted to an
explicit dependency (gui/csrf.py + gui/wizard.py imported it as an undeclared
transitive that uv re-lock would otherwise prune).
Full suite: 780 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 23:25:44 +00:00
|
|
|
|
feat(telemetry-separation): dedicated /telemetry tab split from /events by adapter data_class (v0.7.4)
PR #5 of the v0.7.x GUI rework arc. Production code; central-gui restart only
(supervisor untouched -- data_class is read only by central-gui per request).
- SourceAdapter gains a `data_class` class attr (Literal["event","telemetry"],
default "event"). NWIS opts in as "telemetry" (continuous high-volume water
gauges); every other adapter stays "event". The /events vs /telemetry split is
thus registry-derived from class attrs -- no hardcoded adapter-name lists.
- routes.py refactor: `_class_adapter_names(data_class)` and a `data_class` arg
on `_adapter_filter_options` scope the flat + domain-grouped chip/legend lists
to a class (colors stay keyed to the FULL registry, so an adapter keeps one
color across tabs). `_fetch_events` accepts `class_adapters` and adds an
`adapter = ANY(...)` condition. Shared `_events_query`, `_events_page(data_class,
base_path)` and `_events_rows_fragment(...)` back both tabs; `/events`,
`/events/rows`, `/telemetry`, `/telemetry/rows` are thin wrappers.
- Templates parameterized with a `base_path` context var (form action, hx-get,
hx-push-url header, clear-all redirect, JS BASE_PATH const); the `_events_rows`
paginator macro takes `base`. Same templates serve both tabs; nav gains a
Telemetry link.
- /events.json UNCHANGED -- the cursor path sets no `class_adapters`, so the
subject + pagination contract is intact (TestEventsJsonSubject still passes).
Adds TestTelemetrySeparation (data_class defaults, registry split 11 event / 1
telemetry, class-scoped filter options, color stability, and the `adapter =
ANY(...)` SQL shape incl. the no-class events.json path). Updates the events
frontend tests for the base_path-parameterized templates.
Full suite: 682 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 07:34:08 +00:00
|
|
|
|
|
|
|
|
# --- data_class defaults / registry split -----------------------------------
|
|
|
|
|
|
|
|
|
|
def test_base_default_is_event():
|
|
|
|
|
assert SourceAdapter.data_class == "event"
|
|
|
|
|
|
|
|
|
|
|
2026-06-09 00:54:19 -06:00
|
|
|
def test_registry_split_event_vs_telemetry():
|
feat(telemetry-separation): dedicated /telemetry tab split from /events by adapter data_class (v0.7.4)
PR #5 of the v0.7.x GUI rework arc. Production code; central-gui restart only
(supervisor untouched -- data_class is read only by central-gui per request).
- SourceAdapter gains a `data_class` class attr (Literal["event","telemetry"],
default "event"). NWIS opts in as "telemetry" (continuous high-volume water
gauges); every other adapter stays "event". The /events vs /telemetry split is
thus registry-derived from class attrs -- no hardcoded adapter-name lists.
- routes.py refactor: `_class_adapter_names(data_class)` and a `data_class` arg
on `_adapter_filter_options` scope the flat + domain-grouped chip/legend lists
to a class (colors stay keyed to the FULL registry, so an adapter keeps one
color across tabs). `_fetch_events` accepts `class_adapters` and adds an
`adapter = ANY(...)` condition. Shared `_events_query`, `_events_page(data_class,
base_path)` and `_events_rows_fragment(...)` back both tabs; `/events`,
`/events/rows`, `/telemetry`, `/telemetry/rows` are thin wrappers.
- Templates parameterized with a `base_path` context var (form action, hx-get,
hx-push-url header, clear-all redirect, JS BASE_PATH const); the `_events_rows`
paginator macro takes `base`. Same templates serve both tabs; nav gains a
Telemetry link.
- /events.json UNCHANGED -- the cursor path sets no `class_adapters`, so the
subject + pagination contract is intact (TestEventsJsonSubject still passes).
Adds TestTelemetrySeparation (data_class defaults, registry split 11 event / 1
telemetry, class-scoped filter options, color stability, and the `adapter =
ANY(...)` SQL shape incl. the no-class events.json path). Updates the events
frontend tests for the base_path-parameterized templates.
Full suite: 682 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 07:34:08 +00:00
|
|
|
reg = discover_adapters()
|
|
|
|
|
by_class = {}
|
|
|
|
|
for name, cls in reg.items():
|
|
|
|
|
by_class.setdefault(getattr(cls, "data_class", "event"), []).append(name)
|
feat(tomtom_flow): TomTom Orbis vector flow-tile telemetry adapter + CENTRAL_TRAFFIC_FLOW (v0.9.3)
Third CENTRAL_TRAFFIC-family member, first telemetry traffic source. Polls a
configured tile coverage set (Idaho metros, z=10), fetches Orbis vector flow
tiles, decodes per-segment relative_speed + road geometry, emits one telemetry
Event per road segment per poll to the new CENTRAL_TRAFFIC_FLOW stream. Renders
as colored polylines (green free-flow -> red jam) on the /telemetry map.
Production code; supervisor + gui + ARCHIVE restart (NEW event-bearing stream
central.traffic_flow.> -> archive must resubscribe). Ships disabled; needs a
"tomtom" api key in config.api_keys before enable.
- Subject central.traffic_flow.{z}.{x}.{y} (token traffic_flow, non-overlapping
with central.traffic.>). category="flow.tomtom_flow" -> GUI event_type "flow".
- Severity from relative_speed: >=0.75=1, 0.5-0.75=2, 0.25-0.5=3, <0.25=4.
- Cadence 300s; 7-day retention (high-volume telemetry). Dedup minute-bucketed,
inherited from the v0.9.1 SourceAdapter mixin.
- Shared tomtom_flow_parse module (decode + slippy-tile georeference) reused by
the v0.9.4 on-demand passthrough endpoint.
- Generic framework change (Option A, ~3 lines, inert for the other 14
adapters): Geo.geometry optional field + archive _build_geom_sql prefers it,
so segments persist their real LineString to the PostGIS geom column.
- Idaho-only (Orbis tier confirmed live). Cameras + Navi passthrough are follow-ups.
- deps: mapbox-vector-tile (vector PBF decode); itsdangerous promoted to an
explicit dependency (gui/csrf.py + gui/wizard.py imported it as an undeclared
transitive that uv re-lock would otherwise prune).
Full suite: 780 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 23:25:44 +00:00
|
|
|
assert sorted(by_class.get("telemetry", [])) == _TELEMETRY
|
feat(telemetry-separation): dedicated /telemetry tab split from /events by adapter data_class (v0.7.4)
PR #5 of the v0.7.x GUI rework arc. Production code; central-gui restart only
(supervisor untouched -- data_class is read only by central-gui per request).
- SourceAdapter gains a `data_class` class attr (Literal["event","telemetry"],
default "event"). NWIS opts in as "telemetry" (continuous high-volume water
gauges); every other adapter stays "event". The /events vs /telemetry split is
thus registry-derived from class attrs -- no hardcoded adapter-name lists.
- routes.py refactor: `_class_adapter_names(data_class)` and a `data_class` arg
on `_adapter_filter_options` scope the flat + domain-grouped chip/legend lists
to a class (colors stay keyed to the FULL registry, so an adapter keeps one
color across tabs). `_fetch_events` accepts `class_adapters` and adds an
`adapter = ANY(...)` condition. Shared `_events_query`, `_events_page(data_class,
base_path)` and `_events_rows_fragment(...)` back both tabs; `/events`,
`/events/rows`, `/telemetry`, `/telemetry/rows` are thin wrappers.
- Templates parameterized with a `base_path` context var (form action, hx-get,
hx-push-url header, clear-all redirect, JS BASE_PATH const); the `_events_rows`
paginator macro takes `base`. Same templates serve both tabs; nav gains a
Telemetry link.
- /events.json UNCHANGED -- the cursor path sets no `class_adapters`, so the
subject + pagination contract is intact (TestEventsJsonSubject still passes).
Adds TestTelemetrySeparation (data_class defaults, registry split 11 event / 1
telemetry, class-scoped filter options, color stability, and the `adapter =
ANY(...)` SQL shape incl. the no-class events.json path). Updates the events
frontend tests for the base_path-parameterized templates.
Full suite: 682 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 07:34:08 +00:00
|
|
|
# Everything else is event-class; the split must cover the whole registry.
|
feat(tomtom_flow): TomTom Orbis vector flow-tile telemetry adapter + CENTRAL_TRAFFIC_FLOW (v0.9.3)
Third CENTRAL_TRAFFIC-family member, first telemetry traffic source. Polls a
configured tile coverage set (Idaho metros, z=10), fetches Orbis vector flow
tiles, decodes per-segment relative_speed + road geometry, emits one telemetry
Event per road segment per poll to the new CENTRAL_TRAFFIC_FLOW stream. Renders
as colored polylines (green free-flow -> red jam) on the /telemetry map.
Production code; supervisor + gui + ARCHIVE restart (NEW event-bearing stream
central.traffic_flow.> -> archive must resubscribe). Ships disabled; needs a
"tomtom" api key in config.api_keys before enable.
- Subject central.traffic_flow.{z}.{x}.{y} (token traffic_flow, non-overlapping
with central.traffic.>). category="flow.tomtom_flow" -> GUI event_type "flow".
- Severity from relative_speed: >=0.75=1, 0.5-0.75=2, 0.25-0.5=3, <0.25=4.
- Cadence 300s; 7-day retention (high-volume telemetry). Dedup minute-bucketed,
inherited from the v0.9.1 SourceAdapter mixin.
- Shared tomtom_flow_parse module (decode + slippy-tile georeference) reused by
the v0.9.4 on-demand passthrough endpoint.
- Generic framework change (Option A, ~3 lines, inert for the other 14
adapters): Geo.geometry optional field + archive _build_geom_sql prefers it,
so segments persist their real LineString to the PostGIS geom column.
- Idaho-only (Orbis tier confirmed live). Cameras + Navi passthrough are follow-ups.
- deps: mapbox-vector-tile (vector PBF decode); itsdangerous promoted to an
explicit dependency (gui/csrf.py + gui/wizard.py imported it as an undeclared
transitive that uv re-lock would otherwise prune).
Full suite: 780 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 23:25:44 +00:00
|
|
|
assert sorted(by_class.get("event", [])) == sorted(n for n in reg if n not in _TELEMETRY)
|
|
|
|
|
assert len(by_class.get("event", [])) == len(reg) - len(_TELEMETRY)
|
feat(telemetry-separation): dedicated /telemetry tab split from /events by adapter data_class (v0.7.4)
PR #5 of the v0.7.x GUI rework arc. Production code; central-gui restart only
(supervisor untouched -- data_class is read only by central-gui per request).
- SourceAdapter gains a `data_class` class attr (Literal["event","telemetry"],
default "event"). NWIS opts in as "telemetry" (continuous high-volume water
gauges); every other adapter stays "event". The /events vs /telemetry split is
thus registry-derived from class attrs -- no hardcoded adapter-name lists.
- routes.py refactor: `_class_adapter_names(data_class)` and a `data_class` arg
on `_adapter_filter_options` scope the flat + domain-grouped chip/legend lists
to a class (colors stay keyed to the FULL registry, so an adapter keeps one
color across tabs). `_fetch_events` accepts `class_adapters` and adds an
`adapter = ANY(...)` condition. Shared `_events_query`, `_events_page(data_class,
base_path)` and `_events_rows_fragment(...)` back both tabs; `/events`,
`/events/rows`, `/telemetry`, `/telemetry/rows` are thin wrappers.
- Templates parameterized with a `base_path` context var (form action, hx-get,
hx-push-url header, clear-all redirect, JS BASE_PATH const); the `_events_rows`
paginator macro takes `base`. Same templates serve both tabs; nav gains a
Telemetry link.
- /events.json UNCHANGED -- the cursor path sets no `class_adapters`, so the
subject + pagination contract is intact (TestEventsJsonSubject still passes).
Adds TestTelemetrySeparation (data_class defaults, registry split 11 event / 1
telemetry, class-scoped filter options, color stability, and the `adapter =
ANY(...)` SQL shape incl. the no-class events.json path). Updates the events
frontend tests for the base_path-parameterized templates.
Full suite: 682 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 07:34:08 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_class_adapter_names():
|
|
|
|
|
assert "nwis" not in routes._class_adapter_names("event")
|
feat(tomtom_flow): TomTom Orbis vector flow-tile telemetry adapter + CENTRAL_TRAFFIC_FLOW (v0.9.3)
Third CENTRAL_TRAFFIC-family member, first telemetry traffic source. Polls a
configured tile coverage set (Idaho metros, z=10), fetches Orbis vector flow
tiles, decodes per-segment relative_speed + road geometry, emits one telemetry
Event per road segment per poll to the new CENTRAL_TRAFFIC_FLOW stream. Renders
as colored polylines (green free-flow -> red jam) on the /telemetry map.
Production code; supervisor + gui + ARCHIVE restart (NEW event-bearing stream
central.traffic_flow.> -> archive must resubscribe). Ships disabled; needs a
"tomtom" api key in config.api_keys before enable.
- Subject central.traffic_flow.{z}.{x}.{y} (token traffic_flow, non-overlapping
with central.traffic.>). category="flow.tomtom_flow" -> GUI event_type "flow".
- Severity from relative_speed: >=0.75=1, 0.5-0.75=2, 0.25-0.5=3, <0.25=4.
- Cadence 300s; 7-day retention (high-volume telemetry). Dedup minute-bucketed,
inherited from the v0.9.1 SourceAdapter mixin.
- Shared tomtom_flow_parse module (decode + slippy-tile georeference) reused by
the v0.9.4 on-demand passthrough endpoint.
- Generic framework change (Option A, ~3 lines, inert for the other 14
adapters): Geo.geometry optional field + archive _build_geom_sql prefers it,
so segments persist their real LineString to the PostGIS geom column.
- Idaho-only (Orbis tier confirmed live). Cameras + Navi passthrough are follow-ups.
- deps: mapbox-vector-tile (vector PBF decode); itsdangerous promoted to an
explicit dependency (gui/csrf.py + gui/wizard.py imported it as an undeclared
transitive that uv re-lock would otherwise prune).
Full suite: 780 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 23:25:44 +00:00
|
|
|
assert sorted(routes._class_adapter_names("telemetry")) == _TELEMETRY
|
feat(telemetry-separation): dedicated /telemetry tab split from /events by adapter data_class (v0.7.4)
PR #5 of the v0.7.x GUI rework arc. Production code; central-gui restart only
(supervisor untouched -- data_class is read only by central-gui per request).
- SourceAdapter gains a `data_class` class attr (Literal["event","telemetry"],
default "event"). NWIS opts in as "telemetry" (continuous high-volume water
gauges); every other adapter stays "event". The /events vs /telemetry split is
thus registry-derived from class attrs -- no hardcoded adapter-name lists.
- routes.py refactor: `_class_adapter_names(data_class)` and a `data_class` arg
on `_adapter_filter_options` scope the flat + domain-grouped chip/legend lists
to a class (colors stay keyed to the FULL registry, so an adapter keeps one
color across tabs). `_fetch_events` accepts `class_adapters` and adds an
`adapter = ANY(...)` condition. Shared `_events_query`, `_events_page(data_class,
base_path)` and `_events_rows_fragment(...)` back both tabs; `/events`,
`/events/rows`, `/telemetry`, `/telemetry/rows` are thin wrappers.
- Templates parameterized with a `base_path` context var (form action, hx-get,
hx-push-url header, clear-all redirect, JS BASE_PATH const); the `_events_rows`
paginator macro takes `base`. Same templates serve both tabs; nav gains a
Telemetry link.
- /events.json UNCHANGED -- the cursor path sets no `class_adapters`, so the
subject + pagination contract is intact (TestEventsJsonSubject still passes).
Adds TestTelemetrySeparation (data_class defaults, registry split 11 event / 1
telemetry, class-scoped filter options, color stability, and the `adapter =
ANY(...)` SQL shape incl. the no-class events.json path). Updates the events
frontend tests for the base_path-parameterized templates.
Full suite: 682 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 07:34:08 +00:00
|
|
|
assert "usgs_quake" in routes._class_adapter_names("event")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- class-scoped chip-picker / legend options -------------------------------
|
|
|
|
|
|
|
|
|
|
def test_event_options_exclude_nwis():
|
|
|
|
|
flat, grouped = routes._adapter_filter_options("event")
|
|
|
|
|
names = {a["name"] for a in flat}
|
|
|
|
|
assert "nwis" not in names
|
feat(tomtom_flow): TomTom Orbis vector flow-tile telemetry adapter + CENTRAL_TRAFFIC_FLOW (v0.9.3)
Third CENTRAL_TRAFFIC-family member, first telemetry traffic source. Polls a
configured tile coverage set (Idaho metros, z=10), fetches Orbis vector flow
tiles, decodes per-segment relative_speed + road geometry, emits one telemetry
Event per road segment per poll to the new CENTRAL_TRAFFIC_FLOW stream. Renders
as colored polylines (green free-flow -> red jam) on the /telemetry map.
Production code; supervisor + gui + ARCHIVE restart (NEW event-bearing stream
central.traffic_flow.> -> archive must resubscribe). Ships disabled; needs a
"tomtom" api key in config.api_keys before enable.
- Subject central.traffic_flow.{z}.{x}.{y} (token traffic_flow, non-overlapping
with central.traffic.>). category="flow.tomtom_flow" -> GUI event_type "flow".
- Severity from relative_speed: >=0.75=1, 0.5-0.75=2, 0.25-0.5=3, <0.25=4.
- Cadence 300s; 7-day retention (high-volume telemetry). Dedup minute-bucketed,
inherited from the v0.9.1 SourceAdapter mixin.
- Shared tomtom_flow_parse module (decode + slippy-tile georeference) reused by
the v0.9.4 on-demand passthrough endpoint.
- Generic framework change (Option A, ~3 lines, inert for the other 14
adapters): Geo.geometry optional field + archive _build_geom_sql prefers it,
so segments persist their real LineString to the PostGIS geom column.
- Idaho-only (Orbis tier confirmed live). Cameras + Navi passthrough are follow-ups.
- deps: mapbox-vector-tile (vector PBF decode); itsdangerous promoted to an
explicit dependency (gui/csrf.py + gui/wizard.py imported it as an undeclared
transitive that uv re-lock would otherwise prune).
Full suite: 780 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 23:25:44 +00:00
|
|
|
assert len(flat) == len(discover_adapters()) - len(_TELEMETRY)
|
feat(telemetry-separation): dedicated /telemetry tab split from /events by adapter data_class (v0.7.4)
PR #5 of the v0.7.x GUI rework arc. Production code; central-gui restart only
(supervisor untouched -- data_class is read only by central-gui per request).
- SourceAdapter gains a `data_class` class attr (Literal["event","telemetry"],
default "event"). NWIS opts in as "telemetry" (continuous high-volume water
gauges); every other adapter stays "event". The /events vs /telemetry split is
thus registry-derived from class attrs -- no hardcoded adapter-name lists.
- routes.py refactor: `_class_adapter_names(data_class)` and a `data_class` arg
on `_adapter_filter_options` scope the flat + domain-grouped chip/legend lists
to a class (colors stay keyed to the FULL registry, so an adapter keeps one
color across tabs). `_fetch_events` accepts `class_adapters` and adds an
`adapter = ANY(...)` condition. Shared `_events_query`, `_events_page(data_class,
base_path)` and `_events_rows_fragment(...)` back both tabs; `/events`,
`/events/rows`, `/telemetry`, `/telemetry/rows` are thin wrappers.
- Templates parameterized with a `base_path` context var (form action, hx-get,
hx-push-url header, clear-all redirect, JS BASE_PATH const); the `_events_rows`
paginator macro takes `base`. Same templates serve both tabs; nav gains a
Telemetry link.
- /events.json UNCHANGED -- the cursor path sets no `class_adapters`, so the
subject + pagination contract is intact (TestEventsJsonSubject still passes).
Adds TestTelemetrySeparation (data_class defaults, registry split 11 event / 1
telemetry, class-scoped filter options, color stability, and the `adapter =
ANY(...)` SQL shape incl. the no-class events.json path). Updates the events
frontend tests for the base_path-parameterized templates.
Full suite: 682 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 07:34:08 +00:00
|
|
|
grouped_values = {opt["value"] for _, items in grouped for opt in items}
|
|
|
|
|
assert "nwis" not in grouped_values
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_telemetry_options_only_nwis():
|
|
|
|
|
flat, grouped = routes._adapter_filter_options("telemetry")
|
feat(tomtom_flow): TomTom Orbis vector flow-tile telemetry adapter + CENTRAL_TRAFFIC_FLOW (v0.9.3)
Third CENTRAL_TRAFFIC-family member, first telemetry traffic source. Polls a
configured tile coverage set (Idaho metros, z=10), fetches Orbis vector flow
tiles, decodes per-segment relative_speed + road geometry, emits one telemetry
Event per road segment per poll to the new CENTRAL_TRAFFIC_FLOW stream. Renders
as colored polylines (green free-flow -> red jam) on the /telemetry map.
Production code; supervisor + gui + ARCHIVE restart (NEW event-bearing stream
central.traffic_flow.> -> archive must resubscribe). Ships disabled; needs a
"tomtom" api key in config.api_keys before enable.
- Subject central.traffic_flow.{z}.{x}.{y} (token traffic_flow, non-overlapping
with central.traffic.>). category="flow.tomtom_flow" -> GUI event_type "flow".
- Severity from relative_speed: >=0.75=1, 0.5-0.75=2, 0.25-0.5=3, <0.25=4.
- Cadence 300s; 7-day retention (high-volume telemetry). Dedup minute-bucketed,
inherited from the v0.9.1 SourceAdapter mixin.
- Shared tomtom_flow_parse module (decode + slippy-tile georeference) reused by
the v0.9.4 on-demand passthrough endpoint.
- Generic framework change (Option A, ~3 lines, inert for the other 14
adapters): Geo.geometry optional field + archive _build_geom_sql prefers it,
so segments persist their real LineString to the PostGIS geom column.
- Idaho-only (Orbis tier confirmed live). Cameras + Navi passthrough are follow-ups.
- deps: mapbox-vector-tile (vector PBF decode); itsdangerous promoted to an
explicit dependency (gui/csrf.py + gui/wizard.py imported it as an undeclared
transitive that uv re-lock would otherwise prune).
Full suite: 780 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 23:25:44 +00:00
|
|
|
assert sorted(a["name"] for a in flat) == _TELEMETRY
|
feat(telemetry-separation): dedicated /telemetry tab split from /events by adapter data_class (v0.7.4)
PR #5 of the v0.7.x GUI rework arc. Production code; central-gui restart only
(supervisor untouched -- data_class is read only by central-gui per request).
- SourceAdapter gains a `data_class` class attr (Literal["event","telemetry"],
default "event"). NWIS opts in as "telemetry" (continuous high-volume water
gauges); every other adapter stays "event". The /events vs /telemetry split is
thus registry-derived from class attrs -- no hardcoded adapter-name lists.
- routes.py refactor: `_class_adapter_names(data_class)` and a `data_class` arg
on `_adapter_filter_options` scope the flat + domain-grouped chip/legend lists
to a class (colors stay keyed to the FULL registry, so an adapter keeps one
color across tabs). `_fetch_events` accepts `class_adapters` and adds an
`adapter = ANY(...)` condition. Shared `_events_query`, `_events_page(data_class,
base_path)` and `_events_rows_fragment(...)` back both tabs; `/events`,
`/events/rows`, `/telemetry`, `/telemetry/rows` are thin wrappers.
- Templates parameterized with a `base_path` context var (form action, hx-get,
hx-push-url header, clear-all redirect, JS BASE_PATH const); the `_events_rows`
paginator macro takes `base`. Same templates serve both tabs; nav gains a
Telemetry link.
- /events.json UNCHANGED -- the cursor path sets no `class_adapters`, so the
subject + pagination contract is intact (TestEventsJsonSubject still passes).
Adds TestTelemetrySeparation (data_class defaults, registry split 11 event / 1
telemetry, class-scoped filter options, color stability, and the `adapter =
ANY(...)` SQL shape incl. the no-class events.json path). Updates the events
frontend tests for the base_path-parameterized templates.
Full suite: 682 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 07:34:08 +00:00
|
|
|
grouped_values = [opt["value"] for _, items in grouped for opt in items]
|
feat(tomtom_flow): TomTom Orbis vector flow-tile telemetry adapter + CENTRAL_TRAFFIC_FLOW (v0.9.3)
Third CENTRAL_TRAFFIC-family member, first telemetry traffic source. Polls a
configured tile coverage set (Idaho metros, z=10), fetches Orbis vector flow
tiles, decodes per-segment relative_speed + road geometry, emits one telemetry
Event per road segment per poll to the new CENTRAL_TRAFFIC_FLOW stream. Renders
as colored polylines (green free-flow -> red jam) on the /telemetry map.
Production code; supervisor + gui + ARCHIVE restart (NEW event-bearing stream
central.traffic_flow.> -> archive must resubscribe). Ships disabled; needs a
"tomtom" api key in config.api_keys before enable.
- Subject central.traffic_flow.{z}.{x}.{y} (token traffic_flow, non-overlapping
with central.traffic.>). category="flow.tomtom_flow" -> GUI event_type "flow".
- Severity from relative_speed: >=0.75=1, 0.5-0.75=2, 0.25-0.5=3, <0.25=4.
- Cadence 300s; 7-day retention (high-volume telemetry). Dedup minute-bucketed,
inherited from the v0.9.1 SourceAdapter mixin.
- Shared tomtom_flow_parse module (decode + slippy-tile georeference) reused by
the v0.9.4 on-demand passthrough endpoint.
- Generic framework change (Option A, ~3 lines, inert for the other 14
adapters): Geo.geometry optional field + archive _build_geom_sql prefers it,
so segments persist their real LineString to the PostGIS geom column.
- Idaho-only (Orbis tier confirmed live). Cameras + Navi passthrough are follow-ups.
- deps: mapbox-vector-tile (vector PBF decode); itsdangerous promoted to an
explicit dependency (gui/csrf.py + gui/wizard.py imported it as an undeclared
transitive that uv re-lock would otherwise prune).
Full suite: 780 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 23:25:44 +00:00
|
|
|
assert sorted(grouped_values) == _TELEMETRY
|
feat(telemetry-separation): dedicated /telemetry tab split from /events by adapter data_class (v0.7.4)
PR #5 of the v0.7.x GUI rework arc. Production code; central-gui restart only
(supervisor untouched -- data_class is read only by central-gui per request).
- SourceAdapter gains a `data_class` class attr (Literal["event","telemetry"],
default "event"). NWIS opts in as "telemetry" (continuous high-volume water
gauges); every other adapter stays "event". The /events vs /telemetry split is
thus registry-derived from class attrs -- no hardcoded adapter-name lists.
- routes.py refactor: `_class_adapter_names(data_class)` and a `data_class` arg
on `_adapter_filter_options` scope the flat + domain-grouped chip/legend lists
to a class (colors stay keyed to the FULL registry, so an adapter keeps one
color across tabs). `_fetch_events` accepts `class_adapters` and adds an
`adapter = ANY(...)` condition. Shared `_events_query`, `_events_page(data_class,
base_path)` and `_events_rows_fragment(...)` back both tabs; `/events`,
`/events/rows`, `/telemetry`, `/telemetry/rows` are thin wrappers.
- Templates parameterized with a `base_path` context var (form action, hx-get,
hx-push-url header, clear-all redirect, JS BASE_PATH const); the `_events_rows`
paginator macro takes `base`. Same templates serve both tabs; nav gains a
Telemetry link.
- /events.json UNCHANGED -- the cursor path sets no `class_adapters`, so the
subject + pagination contract is intact (TestEventsJsonSubject still passes).
Adds TestTelemetrySeparation (data_class defaults, registry split 11 event / 1
telemetry, class-scoped filter options, color stability, and the `adapter =
ANY(...)` SQL shape incl. the no-class events.json path). Updates the events
frontend tests for the base_path-parameterized templates.
Full suite: 682 passed, 1 skipped (central and unprivileged zvx, 3x each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 07:34:08 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_colors_stable_across_classes():
|
|
|
|
|
"""A given adapter keeps the same color on /events and /telemetry (colors
|
|
|
|
|
are keyed to the full registry, not the per-tab subset)."""
|
|
|
|
|
full, _ = routes._adapter_filter_options()
|
|
|
|
|
full_color = {a["name"]: a["color"] for a in full}
|
|
|
|
|
ev, _ = routes._adapter_filter_options("event")
|
|
|
|
|
for a in ev:
|
|
|
|
|
assert a["color"] == full_color[a["name"]]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- data_class SQL filter (captured SQL) ------------------------------------
|
|
|
|
|
|
|
|
|
|
async def _capture(parsed):
|
|
|
|
|
captured = {}
|
|
|
|
|
|
|
|
|
|
async def fake_fetch(query, *args):
|
|
|
|
|
captured["query"] = query
|
|
|
|
|
captured["params"] = list(args)
|
|
|
|
|
return []
|
|
|
|
|
|
|
|
|
|
conn = MagicMock()
|
|
|
|
|
conn.fetch = fake_fetch
|
|
|
|
|
pool = MagicMock()
|
|
|
|
|
pool.acquire.return_value.__aenter__ = AsyncMock(return_value=conn)
|
|
|
|
|
pool.acquire.return_value.__aexit__ = AsyncMock(return_value=None)
|
|
|
|
|
with patch("central.gui.routes.get_pool", return_value=pool):
|
|
|
|
|
await routes._fetch_events(parsed)
|
|
|
|
|
return captured
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_class_adapters_adds_adapter_any_condition():
|
|
|
|
|
parsed, _ = routes._parse_events_params({"time": "all"}, default_offset=0)
|
|
|
|
|
parsed["class_adapters"] = routes._class_adapter_names("event")
|
|
|
|
|
cap = await _capture(parsed)
|
|
|
|
|
assert "adapter = ANY($" in cap["query"]
|
|
|
|
|
assert routes._class_adapter_names("event") in cap["params"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_no_class_adapters_no_class_condition():
|
|
|
|
|
"""events.json path: no class_adapters -> no extra adapter filter (all classes)."""
|
|
|
|
|
parsed, _ = routes._parse_events_params({"time": "all"}) # cursor-mode, no class
|
|
|
|
|
assert parsed.get("class_adapters") is None
|
|
|
|
|
cap = await _capture(parsed)
|
|
|
|
|
# The only adapter=ANY would come from a user filter, which we didn't set.
|
|
|
|
|
assert "adapter = ANY($" not in cap["query"]
|