central/sql/migrations/039_add_sat_positions_adapter.sql

58 lines
2.5 KiB
MySQL
Raw Normal View History

v0.12.0: sat_positions adapter (live global satellite positions) + sat_common refactor ## Architectural framing The v0.11.1 `satpass_predict` adapter is **observer-anchored**: "when does satellite X pass over fixed observer Y, and what's the elevation/azimuth at that observer's site?" It answers a fixed-QTH question and emits one event per (observer, satellite, AOS) tuple. The new `sat_positions` adapter is the **global** counterpart: "where is satellite X right now?" No observer. One event per tracked NORAD ID per poll, on subject `central.sat.position.<norad_id>`. Consumers (meshAI, GUI map widgets, anything that wants a live world map) subscribe to `central.sat.position.>` and plot. They complement each other; neither replaces the other. Direct quote from Matt's use-case: *"location of the sats... map of where the sats are then we have meshai or whatever other service calling central's data grab it and do whatever work it needed."* This adapter is that. ## sat_common extraction rationale The four pure SGP4 / coordinate helpers (`EARTH_RADIUS_KM`, `gmst_rad`, `eci_to_ecef`, `subsatellite_point`) were private symbols inside `satpass_predict.py`. `sat_positions` needs the same three helpers. Three options were considered: 1. **Cross-import** from `satpass_predict.py` — creates an adapter-to-adapter dependency, ugly. 2. **Extract to `sat_common.py`** — matches the existing `wfigs_common.py` / `swpc_common.py` precedent. Both adapters become siblings of a shared helper module. ✓ chosen. 3. **Duplicate** — math drift over time. Symbol names dropped their leading underscore on extraction (public-API convention matching `swpc_common.parse_swpc_timestamp` / `wfigs_common.severity_from_acres`). Existing internal call sites in `satpass_predict.py` were updated via mechanical `replace_all`. Observer-specific helpers (`_observer_ecef`, `_topocentric_az_el`, `_visibility_footprint`, `_severity_from_elev`, `_build_pass_geometry`, `_next_passes`) stay in `satpass_predict.py` per YAGNI — they're not used by `sat_positions` today. Existing `tests/test_satpass_predict.py` was updated mechanically to import the helpers from `sat_common` via aliases (preserves the underscore-prefixed local names in the tests so the rest of the test body needs no change). All 44 satpass_predict tests pass unchanged. ## CENTRAL_SAT stream cap bump `config.streams.max_bytes` for `CENTRAL_SAT` goes from **1 GiB → 5 GiB** in migration 039. Sizing math: - celestrak_tle: ~190 sats × 1 envelope/day = ~190 events/day = ~1.4k events/week. Fit in 1 GiB easily. - sat_positions: ~190 sats × 1440 ticks/day (60s cadence) = **~273.6k events/day = ~1.9M events/week**. At ~1 KB per envelope including the CloudEvents wrapper, that's **~1.9 GiB/week**. - Plus existing TLE + pass envelopes already on the stream → ~3 GiB headroom needed. - 5 GiB = 5368709120 bytes = operator-tunable margin without over-provisioning. `STREAM_CATEGORY_DOMAINS["CENTRAL_SAT"]` extends from `("tle", "pass")` to `("tle", "pass", "position")` so the supervisor's retention sweep covers position events too. ## Subject + dedup | Field | Value | |---|---| | Subject | `central.sat.position.<norad_id>` — one subject per satellite, globally | | Dedup id | `<norad_id>:<position_iso>` where `position_iso` is the propagation timestamp truncated to whole seconds (defensive collapse if cadence is ever tightened) | | Severity | 1 (informational telemetry, no alerting) | | data_class | `telemetry` — surfaces on `/telemetry`, not `/events` | | Cadence | 60s default; operator-tunable via standard `cadence_s` field | ## Settings shape ```json {"track_only_norad_ids": [], "max_tle_age_days": 14} ``` - Empty `track_only_norad_ids` = track every NORAD ID with a fresh TLE in the events table (derive-from-celestrak_tle, default behavior). - Non-empty list pins to those NORAD IDs only (operator override — "I only care about the ISS and these 12 Starlink sats"). - `max_tle_age_days` bounds TLE freshness; LEO drag means TLEs go stale in days, GEO is good for months. Parameterized into the SQL query as a timedelta interval so operator-tightened windows (e.g. 3d) apply without code change. ## Event.data fields `norad_id`, `satellite_name`, `lon_deg`, `lat_deg`, `alt_km`, `velocity_kmps`, `heading_deg`, `tle_epoch`. - `lon_deg`/`lat_deg`/`alt_km`: sub-satellite point via SGP4 → ECI → ECEF rotation → spherical-earth lon/lat/alt. - `velocity_kmps`: magnitude of the SGP4 ECI velocity vector. ECI vs ECEF difference is ~6% for LEO (earth rotation 0.46 km/s vs 7.7 km/s orbital speed); fine for consumer "the sat is moving at X km/s" text. - `heading_deg`: great-circle initial bearing from the sub-sat point at `t` to the sub-sat point at `t+1s` (finite-difference; simpler than rotating velocity through GMST + the earth-rotation cross term). ## Diff size — flag for review **+894 / -63 = +831 net** across 14 files. Spec budget was ≤700 lines. **Over by ~131 net** (or ~194 gross). Breakdown: - `sat_positions.py`: 286 lines (under the ≤350 adapter line cap ✓) - `sat_common.py`: 65 lines (the extraction) - Migration 039: 58 lines (heavy on inline comments documenting the size math; could trim ~25 lines if you want) - satpass_predict.py: net -1 line (refactor; lost 4 helper defs and one constant comment, gained 5-line import block) - Templates: 14 lines (event_rows + event_summaries partials) - Wiring: 4 lines (supervisor + ADAPTER_GROUPS) - Docs (CONSUMER-INTEGRATION.md): 40 lines (required by `tests/test_consumer_doc.py::test_every_adapter_has_a_subsection`) - **Tests: 426 lines.** This is the bulk of the overage. The tests are all spec-mandated (sub-sat math, velocity range, heading range, build_event, subject_for, empty-TLE, track_only gate, stale-TLE skip, sat_common helpers, regression-guard on the moved helpers via test_satpass_predict.py preservation). I could shrink `test_sat_positions.py` by consolidating the 11 spec-mandated tests into fewer parameterized cases, but each test pins one behavior the spec called out by name. Flagging for your call: keep as-is, or do you want a tighter parameterized version? ## Test plan - [x] `pytest tests/test_sat_common.py tests/test_sat_positions.py` — **28 new tests, all pass**. - [x] `pytest tests/test_satpass_predict.py` — **44/44 pass** (regression guard: existing tests work after the sat_common extraction). - [x] `pytest tests/test_events_feed_frontend.py` — **119/119 pass** (JSON-feed coverage extended to include sat_positions sample event + expected subject string). - [x] `pytest tests/test_telemetry_separation.py` — **9/9 pass** (`_TELEMETRY` pin extended to include `sat_positions`). - [x] `pytest tests/test_consumer_doc.py` — **6/6 pass** (CONSUMER-INTEGRATION.md `### sat_positions` subsection added). - [x] `pytest tests/test_producer_doc.py` — **10/10 pass** (no PRODUCER-INTEGRATION update needed; CENTRAL_SAT stream is pre-existing). - [x] Full sweep `pytest tests/` (excluding postgres-dep files): **1209 passed, 1 skipped, 0 failures**. - [x] Ruff: clean on all new code. 3 pre-existing F841 unused-variable warnings (supervisor.py:390 `poll_start`, test_events_feed_frontend.py:425 / :466 `result`) confirmed via `git blame` to be from commits May 2026 — not introduced. ## Deploy plan 1. Squash-merge → tag v0.12.0 at merge SHA → push tag. 2. `ssh central`, `git pull` on `/opt/central`. **No `uv sync`** (no new dep). 3. **`central-migrate`** to apply migration 039 (seeds `config.adapters` row + bumps `config.streams.max_bytes` for CENTRAL_SAT). 4. `sudo systemctl restart central-supervisor` (picks up STREAM_CATEGORY_DOMAINS extension + new adapter discovery). 5. `sudo systemctl restart central-gui` (picks up new partials + ADAPTER_GROUPS change). 6. **No** `central-archive` restart (CENTRAL_SAT stream already exists; no new stream). 7. Verify: `nats stream info CENTRAL_SAT` shows max_bytes=5368709120; supervisor journal shows sat_positions discovered. 8. Smoke-test: enable celestrak_tle first if not already, wait for one poll, then enable sat_positions via GUI. Within 60s expect one `central.sat.position.<norad_id>` event per tracked sat on the stream. ## Halt acknowledgment Per spec acceptance bar #6: **squash-merge NOT authorized**. Branch + PR open. Halting for line-by-line review. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-06-09 15:23:32 -06:00
-- Migration 039: register sat_positions adapter + bump CENTRAL_SAT cap (v0.12.0)
--
-- Live global satellite-position publisher: one telemetry event per tracked
-- NORAD ID per poll on subject central.sat.position.<norad_id>. Complement
-- to satpass_predict (observer-anchored alerts -- v0.11.1): sat_positions
-- is the *global* counterpart, "where is sat X right now" rather than
-- "when is sat X overhead at observer Y".
--
-- Publishes on the existing CENTRAL_SAT stream via the supervisor's
-- STREAM_CATEGORY_DOMAINS extension ("CENTRAL_SAT": ("tle", "pass",
-- "position")) -- no new stream is created.
--
-- Cadence 60s (1 minute) by default. LEO sub-satellite points drift
-- visibly at minute scale (~7.7 km/s * 60s = 462 km of ground track), so
-- 60s ticks give a watchable live map. GEO sats barely move at minute
-- scale, but cadence is operator-tunable per-adapter -- if a future
-- operator pin-list contains only GEO sats they can drop cadence to
-- 300s with no code change.
--
-- Ships disabled (enabled=false) -- celestrak_tle must be enabled and
-- have polled at least once for sat_positions to have TLE data to
-- propagate. Operator enables via GUI after celestrak_tle is producing.
--
-- Settings shape:
-- track_only_norad_ids: empty list = track every NORAD ID with a fresh
-- TLE in the events table (default behavior, derive-from-celestrak_tle).
-- Operator-pinned non-empty list restricts the set.
-- max_tle_age_days: 14 = TLEs older than 14d are skipped to keep
-- SGP4 propagation drift bounded. Operator can tighten (e.g. 3d) for
-- drag-sensitive LEO accuracy or widen for stale-tolerant feeds.
--
-- CENTRAL_SAT max_bytes bump from 1 GiB -> 5 GiB. Sizing rationale:
-- celestrak_tle alone produced ~190 sats * 1 envelope/day ~= 190 events/day
-- ~= ~1.4k events/week; that fit in 1 GiB easily.
-- sat_positions adds ~190 sats * 1440 ticks/day (60s cadence)
-- ~= 273,600 events/day ~= 1.92M events/week. At ~1 KB per envelope
-- including CloudEvents wrapper, that's ~1.9 GiB/week. Plus the existing
-- TLE + pass envelopes the stream already carries: ~3 GiB headroom needed.
-- 5 GiB (5368709120 bytes) gives operator-tunable margin without
-- over-provisioning.
--
-- Idempotent: ON CONFLICT clauses preserve any operator-tuned state.
UPDATE config.streams
SET max_bytes = 5368709120
WHERE name = 'CENTRAL_SAT';
INSERT INTO config.adapters (name, enabled, cadence_s, settings)
VALUES (
'sat_positions',
false,
60,
'{
"track_only_norad_ids": [],
"max_tle_age_days": 14
}'::jsonb
)
ON CONFLICT (name) DO NOTHING;