mirror of
https://github.com/zvx-echo6/central.git
synced 2026-05-21 18:14:44 +02:00
5d64a8f70d
NASA WaterData OGC API v0 (latest-continuous collection) — polls configured parameter codes within an operator-set bbox and publishes on the new CENTRAL_HYDRO stream. - Subject: central.hydro.<parameter_code>.<agency>.<bare_site_no> (e.g. central.hydro.00060.usgs.05420500). The agency/site decomposition lives in a single _subject_tokens_for_id helper. - Default parameter codes: 00060 (discharge), 00065 (gage height), 00010 (water temperature). Operator-tunable; single SoT in _DEFAULT_PARAMETER_CODES — no parallel literals. - Composite dedup: nwis:<monitoring_location_id>:<param>:<time_iso>. Prefix kept in dedup key for cross-agency uniqueness. - Pagination: follows OGC 'rel=next' link until absent (cursor-based). - Region bbox is REQUIRED in practice; adapter logs WARN at startup if region is None (does not refuse to start). - New stream CENTRAL_HYDRO added to streams.py registry (one line). Retention mirrors CENTRAL_DISASTER (7 days, 1 GiB). - No removal pattern in v1 — sites are static; missing data is the signal. Upstream divergences from the original spec brief, caught by pre-build curl: - Collection is 'latest-continuous', not 'instantaneous-values'. - Site filter param is 'monitoring_location_id' (singular), not 'monitoring_locations_id' (plural). - Site identifier requires agency prefix in queries (USGS-NNNNN). - feature.id is a per-record UUID, not stable; dedup uses joint key. Ships disabled; operator enables via GUI after setting a bbox.
29 lines
1.2 KiB
SQL
29 lines
1.2 KiB
SQL
-- Migration: 023_add_nwis_adapter_and_hydro_stream
|
|
-- Adds the CENTRAL_HYDRO JetStream stream row AND the NWIS adapter row.
|
|
-- Folded into a single migration because the adapter publishes onto
|
|
-- central.hydro.> — both rows ship together.
|
|
--
|
|
-- Stream retention mirrors CENTRAL_DISASTER (7 days, 1 GiB).
|
|
-- Adapter ships disabled; operator enables via GUI after setting a bbox.
|
|
--
|
|
-- The settings JSON below is the literal output of NWISSettings().model_dump_json()
|
|
-- at migration-author time. Regenerate via:
|
|
-- sudo -u central .venv/bin/python -c \
|
|
-- "from central.adapters.nwis import NWISSettings; print(NWISSettings().model_dump_json())"
|
|
-- Do NOT hand-edit the parameter_codes here — _DEFAULT_PARAMETER_CODES in
|
|
-- src/central/adapters/nwis.py is the single source of truth.
|
|
--
|
|
-- Idempotent: both inserts use ON CONFLICT DO NOTHING.
|
|
|
|
INSERT INTO config.streams (name, max_age_s, max_bytes)
|
|
VALUES ('CENTRAL_HYDRO', 604800, 1073741824)
|
|
ON CONFLICT (name) DO NOTHING;
|
|
|
|
INSERT INTO config.adapters (name, enabled, cadence_s, settings)
|
|
VALUES (
|
|
'nwis',
|
|
false,
|
|
900,
|
|
'{"parameter_codes":["00060","00065","00010"],"region":null}'::jsonb
|
|
)
|
|
ON CONFLICT (name) DO NOTHING;
|