matt/central
1
0
Fork 0
mirror of https://github.com/zvx-echo6/central.git synced 2026-08-26 09:21:36 +00:00
Code Issues Projects Releases Packages Wiki Activity

chore: normalize line endings to LF

Browse source
This commit is contained in:
Matt Johnson 2026-05-16 21:27:30 +00:00
commit 374a8c067f
26 changed files with 5357 additions and 5346 deletions
Download patch file Download diff file Expand all files Collapse all files

19
.gitattributes vendored Normal file
View file

@ -0,0 +1,19 @@
# Normalize line endings to LF across the repo.
# Prevents CRLF/LF churn in PR diffs.
* text=auto eol=lf
# Explicit text types
*.py text eol=lf
*.sql text eol=lf
*.md text eol=lf
*.toml text eol=lf
*.yaml text eol=lf
*.yml text eol=lf
*.json text eol=lf
*.sh text eol=lf
*.service text eol=lf
# Binary types
*.pyc binary
*.db binary
*.key binary

422
docs/BUG-CADENCE-DECREASE.md
View file

@ -1,211 +1,211 @@
# Bug Investigation: Cadence Decrease Hot-Reload
**Date:** 2026-05-16
**Component:** central-supervisor
**File:** `supervisor.py`
---
## 1. Reproduction
### Test Case: Decrease 60s → 30s
```
Tlast (poll completed): 04:18:24Z
Config change applied: 04:18:30Z (approx)
Expected next poll: 04:18:54Z (Tlast + 30s)
Actual next poll: 04:19:24Z (Tlast + 60s - OLD cadence)
Subsequent polls: Also at 60s intervals
```
### Log Evidence
```json
{"ts": "...", "msg": "Rescheduled adapter", "adapter": "nws", "old_cadence_s": 60, "new_cadence_s": 30, "next_poll": "2026-05-16T04:18:54+00:00"}
```
- "Rescheduled adapter" log fires with **correct** calculated next_poll
- Actual poll occurs at OLD cadence time
- Subsequent polls continue at OLD cadence
### Contrast: Increase 60s → 90s (WORKS)
```
Tlast: 03:16:34Z
Config change: 03:16:36Z
Expected next poll: 03:18:04Z (Tlast + 90s)
Actual next poll: 03:18:04Z ✅
```
---
## 2. Root Cause
### Location
`supervisor.py` lines 395-450 (`_reschedule_adapter`) and lines 144-181 (`_run_adapter_loop`)
### The Bug
The `cancel_event.set()` call in `_reschedule_adapter` does not reliably wake the `asyncio.wait_for()` in the adapter loop when the cadence is **decreased**.
### Why It Happens
1. **Event handler holds lock during signal:**
```python
# _on_config_change (line 466)
async with self._lock:
new_config = await self._config_source.get_adapter(adapter_name)
# ...
await self._reschedule_adapter(adapter_name, new_config) # sets cancel_event here
```
2. **Reschedule updates config then signals:**
```python
# _reschedule_adapter
state.config = new_config # Line 420
state.adapter.cadence_s = new_cadence # Line 423
# ... logging ...
state.cancel_event.set() # Line 450 - inside lock context
```
3. **Asyncio event delivery delay:**
The `asyncio.Event.set()` queues a wakeup for waiting tasks, but the signal delivery is subject to asyncio's task scheduler. When called from within an `async with` block, the event may not be processed until the current task yields or the lock context exits.
4. **Timing difference between increase and decrease:**
- **Increase (60→90):** Loop has ~30-50s remaining sleep. Event signal arrives well before timeout.
- **Decrease (90→60):** Loop may be ~10s from timeout. By the time event signal is processed, timeout has already fired.
5. **Why subsequent polls use old cadence:**
When the loop times out naturally (rather than being woken by event), it proceeds to poll. After poll completes, `state.last_completed_poll` is updated. The loop then reads `state.config.cadence_s` for the NEXT iteration — but if `state.config` was somehow not durably updated (or there's a stale reference), it uses the old value.
**Alternative theory:** The `state.config = new_config` assignment creates a new config object, but the loop may be reading from a captured reference to the old object if there's any closure behavior we're not seeing.
---
## 3. Proposed Fix
### Option A: Force immediate reschedule (Recommended)
Move the cancel logic OUTSIDE the lock, and use a more aggressive wake pattern:
```python
async def _reschedule_adapter(self, name: str, new_config: AdapterConfig) -> None:
state = self._adapter_states.get(name)
if state is None or not state.is_running:
await self._start_adapter(new_config)
return
old_cadence = state.config.cadence_s
new_cadence = new_config.cadence_s
# Update config atomically
state.config = new_config
state.adapter.cadence_s = new_cadence
# ... (NWS-specific updates, logging) ...
# Cancel and wait for acknowledgment
state.cancel_event.set()
await asyncio.sleep(0) # Force task switch to process event
```
### Option B: Stop and restart the loop task
For cadence changes, stop the current loop task and create a new one:
```python
async def _reschedule_adapter(self, name: str, new_config: AdapterConfig) -> None:
state = self._adapter_states.get(name)
if state is None:
await self._start_adapter(new_config)
return
# Preserve last_completed_poll
preserved_poll = state.last_completed_poll
# Stop current loop
await self._stop_adapter(name)
# Update config
state.config = new_config
state.last_completed_poll = preserved_poll
# Restart loop
await self._start_adapter(new_config)
```
### Option C: Double-signal pattern
Set the event, yield, then set again to ensure delivery:
```python
state.cancel_event.set()
await asyncio.sleep(0)
state.cancel_event.set() # Redundant but ensures visibility
```
---
## 4. Test Gap
### Missing Tests
The test file `test_config_source_new.py` only tests ConfigSource behavior (list, get, protocol compliance). There are **no tests** for:
1. `_reschedule_adapter` interrupting a sleeping loop
2. Cadence decrease being applied mid-sleep
3. Cadence increase being applied mid-sleep
4. Rate-limit guarantee after reschedule
5. `cancel_event` mechanism in isolation
### Recommended Tests
```python
@pytest.mark.asyncio
async def test_cadence_decrease_applies_immediately():
"""Cadence decrease should wake sleeping loop and reschedule."""
# Setup: Adapter polling at 60s cadence
# Action: Change cadence to 30s while sleeping
# Assert: Next poll at last_poll + 30s, not last_poll + 60s
@pytest.mark.asyncio
async def test_cadence_increase_applies_on_next_cycle():
"""Cadence increase should wake sleeping loop and extend wait."""
# Setup: Adapter polling at 60s cadence
# Action: Change cadence to 90s while sleeping
# Assert: Next poll at last_poll + 90s
@pytest.mark.asyncio
async def test_cancel_event_wakes_sleeping_loop():
"""cancel_event.set() should interrupt asyncio.wait_for()."""
# Unit test for the event mechanism in isolation
```
---
## 5. State at End
### LXC State (Reverted)
- **Cadence in DB:** 60s ✅
- **Actual poll interval:** 60s ✅
- **Supervisor restarted:** 2026-05-16T04:43:40Z
- **Verified polls:**
```
04:43:40.964 - First poll after restart
04:44:41.171 - Second poll (61s later) ✅
```
### Mitigation Until Fix
After any cadence change (especially decrease), verify actual poll intervals. If incorrect, restart supervisor:
```bash
systemctl restart central-supervisor
```
---
## Summary
| Item | Details |
|------|---------|
| **Bug** | Cadence decrease hot-reload doesn't apply without restart |
| **Root cause** | `cancel_event.set()` inside lock context has delayed delivery |
| **Affects** | Cadence decreases only; increases work correctly |
| **Workaround** | Restart supervisor after cadence decrease |
| **Fix effort** | Low - add `await asyncio.sleep(0)` after event.set() |
| **Test coverage** | None for hot-reload mechanism |
# Bug Investigation: Cadence Decrease Hot-Reload
**Date:** 2026-05-16
**Component:** central-supervisor
**File:** `supervisor.py`
---
## 1. Reproduction
### Test Case: Decrease 60s → 30s
```
Tlast (poll completed): 04:18:24Z
Config change applied: 04:18:30Z (approx)
Expected next poll: 04:18:54Z (Tlast + 30s)
Actual next poll: 04:19:24Z (Tlast + 60s - OLD cadence)
Subsequent polls: Also at 60s intervals
```
### Log Evidence
```json
{"ts": "...", "msg": "Rescheduled adapter", "adapter": "nws", "old_cadence_s": 60, "new_cadence_s": 30, "next_poll": "2026-05-16T04:18:54+00:00"}
```
- "Rescheduled adapter" log fires with **correct** calculated next_poll
- Actual poll occurs at OLD cadence time
- Subsequent polls continue at OLD cadence
### Contrast: Increase 60s → 90s (WORKS)
```
Tlast: 03:16:34Z
Config change: 03:16:36Z
Expected next poll: 03:18:04Z (Tlast + 90s)
Actual next poll: 03:18:04Z ✅
```
---
## 2. Root Cause
### Location
`supervisor.py` lines 395-450 (`_reschedule_adapter`) and lines 144-181 (`_run_adapter_loop`)
### The Bug
The `cancel_event.set()` call in `_reschedule_adapter` does not reliably wake the `asyncio.wait_for()` in the adapter loop when the cadence is **decreased**.
### Why It Happens
1. **Event handler holds lock during signal:**
```python
# _on_config_change (line 466)
async with self._lock:
new_config = await self._config_source.get_adapter(adapter_name)
# ...
await self._reschedule_adapter(adapter_name, new_config) # sets cancel_event here
```
2. **Reschedule updates config then signals:**
```python
# _reschedule_adapter
state.config = new_config # Line 420
state.adapter.cadence_s = new_cadence # Line 423
# ... logging ...
state.cancel_event.set() # Line 450 - inside lock context
```
3. **Asyncio event delivery delay:**
The `asyncio.Event.set()` queues a wakeup for waiting tasks, but the signal delivery is subject to asyncio's task scheduler. When called from within an `async with` block, the event may not be processed until the current task yields or the lock context exits.
4. **Timing difference between increase and decrease:**
- **Increase (60→90):** Loop has ~30-50s remaining sleep. Event signal arrives well before timeout.
- **Decrease (90→60):** Loop may be ~10s from timeout. By the time event signal is processed, timeout has already fired.
5. **Why subsequent polls use old cadence:**
When the loop times out naturally (rather than being woken by event), it proceeds to poll. After poll completes, `state.last_completed_poll` is updated. The loop then reads `state.config.cadence_s` for the NEXT iteration — but if `state.config` was somehow not durably updated (or there's a stale reference), it uses the old value.
**Alternative theory:** The `state.config = new_config` assignment creates a new config object, but the loop may be reading from a captured reference to the old object if there's any closure behavior we're not seeing.
---
## 3. Proposed Fix
### Option A: Force immediate reschedule (Recommended)
Move the cancel logic OUTSIDE the lock, and use a more aggressive wake pattern:
```python
async def _reschedule_adapter(self, name: str, new_config: AdapterConfig) -> None:
state = self._adapter_states.get(name)
if state is None or not state.is_running:
await self._start_adapter(new_config)
return
old_cadence = state.config.cadence_s
new_cadence = new_config.cadence_s
# Update config atomically
state.config = new_config
state.adapter.cadence_s = new_cadence
# ... (NWS-specific updates, logging) ...
# Cancel and wait for acknowledgment
state.cancel_event.set()
await asyncio.sleep(0) # Force task switch to process event
```
### Option B: Stop and restart the loop task
For cadence changes, stop the current loop task and create a new one:
```python
async def _reschedule_adapter(self, name: str, new_config: AdapterConfig) -> None:
state = self._adapter_states.get(name)
if state is None:
await self._start_adapter(new_config)
return
# Preserve last_completed_poll
preserved_poll = state.last_completed_poll
# Stop current loop
await self._stop_adapter(name)
# Update config
state.config = new_config
state.last_completed_poll = preserved_poll
# Restart loop
await self._start_adapter(new_config)
```
### Option C: Double-signal pattern
Set the event, yield, then set again to ensure delivery:
```python
state.cancel_event.set()
await asyncio.sleep(0)
state.cancel_event.set() # Redundant but ensures visibility
```
---
## 4. Test Gap
### Missing Tests
The test file `test_config_source_new.py` only tests ConfigSource behavior (list, get, protocol compliance). There are **no tests** for:
1. `_reschedule_adapter` interrupting a sleeping loop
2. Cadence decrease being applied mid-sleep
3. Cadence increase being applied mid-sleep
4. Rate-limit guarantee after reschedule
5. `cancel_event` mechanism in isolation
### Recommended Tests
```python
@pytest.mark.asyncio
async def test_cadence_decrease_applies_immediately():
"""Cadence decrease should wake sleeping loop and reschedule."""
# Setup: Adapter polling at 60s cadence
# Action: Change cadence to 30s while sleeping
# Assert: Next poll at last_poll + 30s, not last_poll + 60s
@pytest.mark.asyncio
async def test_cadence_increase_applies_on_next_cycle():
"""Cadence increase should wake sleeping loop and extend wait."""
# Setup: Adapter polling at 60s cadence
# Action: Change cadence to 90s while sleeping
# Assert: Next poll at last_poll + 90s
@pytest.mark.asyncio
async def test_cancel_event_wakes_sleeping_loop():
"""cancel_event.set() should interrupt asyncio.wait_for()."""
# Unit test for the event mechanism in isolation
```
---
## 5. State at End
### LXC State (Reverted)
- **Cadence in DB:** 60s ✅
- **Actual poll interval:** 60s ✅
- **Supervisor restarted:** 2026-05-16T04:43:40Z
- **Verified polls:**
```
04:43:40.964 - First poll after restart
04:44:41.171 - Second poll (61s later) ✅
```
### Mitigation Until Fix
After any cadence change (especially decrease), verify actual poll intervals. If incorrect, restart supervisor:
```bash
systemctl restart central-supervisor
```
---
## Summary
| Item | Details |
|------|---------|
| **Bug** | Cadence decrease hot-reload doesn't apply without restart |
| **Root cause** | `cancel_event.set()` inside lock context has delayed delivery |
| **Affects** | Cadence decreases only; increases work correctly |
| **Workaround** | Restart supervisor after cadence decrease |
| **Fix effort** | Low - add `await asyncio.sleep(0)` after event.set() |
| **Test coverage** | None for hot-reload mechanism |

110
docs/PHASE-1B-NOTES.md
View file

@ -1,58 +1,58 @@
# Phase 1B Planning Notes
Design notes for Phase 1B GUI features. These are planning items, not
implementation specifications.
## Stream Retention GUI
### Per-Stream Configuration
- Show each stream from `config.streams` table
- Editable max_age_s with preset chips: 1d, 7d, 14d, 30d, 365d
- Custom numeric input allowed (operator can enter 90d, etc.)
- Changes trigger NATS stream update via supervisor hot-reload
### Storage Monitor
Per stream, display:
- **Current bytes**: Live from `nats stream info`
- **Projected bytes**: Calculated from current rate × max_age
- **Days remaining**: Current_bytes / rate_per_day estimate
- Refresh: Real-time polling, not cached
### Global Server Cap
- Show `max_file_store` value as read-only reference
- Editing requires NATS server restart (out of scope for GUI)
- Display per-stream ceiling (30% of server cap) as context
## Region Picker
### Interactive Map
- Bbox selection via click-drag rectangle
- Same UI component for all adapters (NWS, FIRMS, USGS)
- Stores `{north, south, east, west}` floats
- Preview of coverage area with state/country boundaries
### Preset Regions
- Common presets: CONUS, Pacific Northwest, Mountain West
- Quick-select buttons alongside custom draw
## API Key Management
### Key Storage
- View configured API keys (alias only, not values)
- Add new keys with alias and value
- Values encrypted at rest in `config.api_keys`
- Rotation: update value, track `rotated_at`
### Required Keys by Adapter
- **FIRMS** (Phase 1a-6): `MAP_KEY` for NASA FIRMS API
- Future adapters may require additional keys
## Technical Notes
- All GUI changes write to `config.*` tables
- Supervisor receives NOTIFY and hot-reloads
- No service restarts required for config changes
- Stream retention changes apply within 5 seconds
# Phase 1B Planning Notes
Design notes for Phase 1B GUI features. These are planning items, not
implementation specifications.
## Stream Retention GUI
### Per-Stream Configuration
- Show each stream from `config.streams` table
- Editable max_age_s with preset chips: 1d, 7d, 14d, 30d, 365d
- Custom numeric input allowed (operator can enter 90d, etc.)
- Changes trigger NATS stream update via supervisor hot-reload
### Storage Monitor
Per stream, display:
- **Current bytes**: Live from `nats stream info`
- **Projected bytes**: Calculated from current rate × max_age
- **Days remaining**: Current_bytes / rate_per_day estimate
- Refresh: Real-time polling, not cached
### Global Server Cap
- Show `max_file_store` value as read-only reference
- Editing requires NATS server restart (out of scope for GUI)
- Display per-stream ceiling (30% of server cap) as context
## Region Picker
### Interactive Map
- Bbox selection via click-drag rectangle
- Same UI component for all adapters (NWS, FIRMS, USGS)
- Stores `{north, south, east, west}` floats
- Preview of coverage area with state/country boundaries
### Preset Regions
- Common presets: CONUS, Pacific Northwest, Mountain West
- Quick-select buttons alongside custom draw
## API Key Management
### Key Storage
- View configured API keys (alias only, not values)
- Add new keys with alias and value
- Values encrypted at rest in `config.api_keys`
- Rotation: update value, track `rotated_at`
### Required Keys by Adapter
- **FIRMS** (Phase 1a-6): `MAP_KEY` for NASA FIRMS API
- Future adapters may require additional keys
## Technical Notes
- All GUI changes write to `config.*` tables
- Supervisor receives NOTIFY and hot-reloads
- No service restarts required for config changes
- Stream retention changes apply within 5 seconds
## FIRMS Adapter Configuration

868
docs/PHASE-1a-3-VERIFICATION.md
View file

@ -1,434 +1,434 @@
# Phase 1a-3 Verification Evidence
## T0 Baseline (TOML config mode, post-merge deploy)
**Timestamp:** 2026-05-16T03:10:51Z
### Upstream Alert IDs
```json
[
"urn:oid:2.49.0.1.840.0.e22a439ed29ed11e4b3686d9fac419ce7ad40059.001.1",
"urn:oid:2.49.0.1.840.0.b7acbf4f0381fb83c1b3f732a4ac9ca16a6204d1.002.1",
"urn:oid:2.49.0.1.840.0.e420a03d4bb13559e9bd61c714d8753fa6a4f66d.001.1",
"urn:oid:2.49.0.1.840.0.82fc471559645fcc3fefe49b4855bde43a7dde2b.001.1",
"urn:oid:2.49.0.1.840.0.add970d087c8d383436ee5958fc56100408aaf2e.001.1",
"urn:oid:2.49.0.1.840.0.f620e3599001fc9937324d55df89b55e475c5568.001.1",
"urn:oid:2.49.0.1.840.0.f620e3599001fc9937324d55df89b55e475c5568.002.1",
"urn:oid:2.49.0.1.840.0.dbde432f293a71618bf9908e5adcf9e5dd27e27c.006.1",
"urn:oid:2.49.0.1.840.0.dbde432f293a71618bf9908e5adcf9e5dd27e27c.001.1",
"urn:oid:2.49.0.1.840.0.dbde432f293a71618bf9908e5adcf9e5dd27e27c.003.1",
"urn:oid:2.49.0.1.840.0.dbde432f293a71618bf9908e5adcf9e5dd27e27c.001.2",
"urn:oid:2.49.0.1.840.0.dbde432f293a71618bf9908e5adcf9e5dd27e27c.005.1",
"urn:oid:2.49.0.1.840.0.b5173bc4f407f3889ea8e9284af261796d04972b.002.1",
"urn:oid:2.49.0.1.840.0.18277c28967847fb1b9e61f5afc236e42659e27b.001.1",
"urn:oid:2.49.0.1.840.0.b5173bc4f407f3889ea8e9284af261796d04972b.001.1",
"urn:oid:2.49.0.1.840.0.86299b43bf001e6c38df077a9b2d8d8e1e7b9116.002.2"
]
```
### Database State
```
count | max
-------+------------------------
30 | 2026-05-16 02:45:00+00
```
### Fresh Envelope Sample (post-restart)
```json
{
"id": "https://api.weather.gov/alerts/urn:oid:2.49.0.1.840.0.35f852d42f3149d3e1722c14e6ffc2e977e48d1b.001.1",
"source": "central/adapters/nws",
"type": "central.wx.alert.lake_wind_advisory.v1",
"time": "2026-05-16T02:45:00+00:00",
"datacontenttype": "application/json",
"centralschemaversion": "1.0.0",
"centralcategory": "wx.alert.lake_wind_advisory",
"centralseverity": 2,
"specversion": "1.0",
"data": { ... }
}
```
**CloudEvents verification:**
- `specversion: "1.0"` ✅
- `type` starts with `central.` (NOT `hub.`) ✅
- Extension attributes use `central*` prefix ✅
- `centralschemaversion` (NOT `hubschemaversion`)
- `centralcategory` (NOT `hubcategory`)
- `centralseverity` (NOT `hubseverity`)
---
## Phase B Step 2: Config Source Cutover (TOML → DB)
**Timestamp:** 2026-05-16T03:13:33Z
### Environment Change
```
# /etc/central/central.env - added:
CENTRAL_CONFIG_SOURCE=db
```
### Supervisor Journal Evidence
```json
{"ts": "2026-05-16T03:13:33.430635+00:00", "level": "INFO", "logger": "central.supervisor", "msg": "Config source: db", "config_source": "db"}
{"ts": "2026-05-16T03:13:33.460162+00:00", "level": "INFO", "logger": "central.config_store", "msg": "Config listener connected to database"}
```
### Archive Journal Evidence
```json
{"ts": "2026-05-16T03:14:03.413008+00:00", "level": "INFO", "logger": "central.archive", "msg": "Archive starting", "nats_url": "nats://localhost:4222", "config_source": "db"}
```
**Result:** Both services running with DB-backed config ✅
---
## Phase B Step 3: Hot-Reload Cadence Test
**Test:** Change cadence from 60s → 90s while adapter is running.
**Goal:** Verify next poll is at Tlast + new_cadence (not old cadence, not immediate).
### Timeline
```
Tlast (last poll): 03:16:34.317219Z
Config change: 03:16:36Z
Expected next poll: 03:18:04.317Z (Tlast + 90s)
Actual next poll: 03:18:04.502Z ✅
```
### Journal Evidence
```json
{"ts": "2026-05-16T03:16:34.317219+00:00", "level": "INFO", "logger": "central.adapters.nws", "msg": "NWS yielded events", "count": 16}
{"ts": "2026-05-16T03:16:37.488781+00:00", "level": "INFO", "logger": "central.supervisor", "msg": "Config change received", "table": "adapters", "key": "nws"}
{"ts": "2026-05-16T03:16:37.511029+00:00", "level": "INFO", "logger": "central.supervisor", "msg": "Rescheduled adapter", "adapter": "nws", "old_cadence_s": 60, "new_cadence_s": 90, "next_poll": "2026-05-16T03:18:04.317651+00:00"}
{"ts": "2026-05-16T03:18:04.502991+00:00", "level": "INFO", "logger": "central.adapters.nws", "msg": "NWS poll completed", "status": 200, "feature_count": 355}
```
**Result:** Rate-limit guarantee upheld. Poll occurred at Tlast + 90s (NOT Tlast + 60s). ✅
---
## Phase B Step 4: Hot-Reload Enable/Disable Test
**Test:** Disable adapter, wait, re-enable.
**Goal:** Verify next poll is at Tlast + cadence (not immediate on re-enable).
### Timeline
```
Tlast (last poll): 03:19:34.758524Z
Disabled at: 03:20:37Z
Re-enabled at: 03:20:48Z
Expected next poll: 03:21:04.758Z (Tlast + 90s)
Actual next poll: 03:21:04.940Z ✅
```
### Journal Evidence
```json
{"ts": "2026-05-16T03:19:34.757999+00:00", "level": "INFO", "logger": "central.adapters.nws", "msg": "NWS yielded events", "count": 16}
{"ts": "2026-05-16T03:20:37.616723+00:00", "level": "INFO", "logger": "central.supervisor", "msg": "Adapter stopped", "adapter": "nws", "preserved_last_poll": "2026-05-16T03:19:34.758524+00:00"}
{"ts": "2026-05-16T03:20:48.947358+00:00", "level": "INFO", "logger": "central.supervisor", "msg": "Adapter restarted", "adapter": "nws", "cadence_s": 90, "preserved_last_poll": "2026-05-16T03:19:34.758524+00:00", "next_poll": "2026-05-16T03:21:04.758524+00:00"}
{"ts": "2026-05-16T03:21:04.940891+00:00", "level": "INFO", "logger": "central.adapters.nws", "msg": "NWS poll completed", "status": 200, "feature_count": 354}
```
**Key observations:**
- `preserved_last_poll` appears in BOTH stop and restart logs (proves state retained)
- `next_poll` calculated from `preserved_last_poll + cadence_s` (not from current time)
- Poll did NOT happen immediately on re-enable
**Result:** Rate-limit guarantee upheld through enable/disable cycle. ✅
---
## Phase B Step 5: T1 Capture and Soak
**T1 Timestamp:** 2026-05-16T03:23:19Z
**T2 Timestamp:** 2026-05-16T03:33:48Z
### T1 State
- Upstream alerts: 16
- DB events: 30
### T2 State (after 10-minute soak)
- Upstream alerts: 16
- DB events: 30
### Poll Activity During Soak
```
03:24:05 - NWS poll completed, status: 200, feature_count: 355
03:25:35 - NWS poll completed, status: 200, feature_count: 357
03:27:05 - NWS poll completed, status: 200, feature_count: 358
03:28:35 - NWS poll completed, status: 200, feature_count: 360
03:30:05 - NWS poll completed, status: 200, feature_count: 357
03:31:35 - NWS poll completed, status: 200, feature_count: 356
03:33:05 - NWS poll completed, status: 200, feature_count: 355
```
**Errors during soak:** None ✅
---
## Phase B Step 6: Data Integrity Check
### Verification
```
Upstream alerts: 16
DB events (total): 30
Missing from DB: 0
All upstream alerts found in DB ✓
```
**Result:** Zero missed alerts. Data integrity confirmed. ✅
---
## Phase B Verification Summary
| Step | Test | Result |
|------|------|--------|
| 2 | Config source cutover | ✅ "Config source: db" in logs |
| 3 | Cadence hot-reload | ✅ Poll at Tlast + new_cadence |
| 4 | Enable/disable cycle | ✅ Rate-limit preserved |
| 5 | 10-minute soak | ✅ No errors |
| 6 | Data integrity | ✅ All alerts in DB |
**Phase B Complete.** System running stable on DB-backed config.
---
## Cadence Revert (Close-out)
**Timestamp:** 2026-05-16T03:54:14Z
### Issue Discovered
During close-out verification, polls were observed at 90s intervals despite
DB showing `cadence_s = 60`. Investigation revealed the live reschedule
from 90→60 (done at 03:23:08 during Phase B) didn't properly update the
in-flight scheduling.
### Resolution
Supervisor restart was required to clear stale state:
```bash
systemctl restart central-supervisor
```
### Post-Restart Verification
**DB State:**
```sql
SELECT name, cadence_s, updated_at FROM config.adapters WHERE name='nws';
```
```
name | cadence_s | updated_at
------+-----------+-------------------------------
nws | 60 | 2026-05-16 03:50:53.210963+00
```
**Poll Intervals After Restart:**
```
03:54:14.621376 - NWS poll completed (first poll after restart)
03:55:15.028963 - NWS poll completed (61s later) ✅
03:56:15.429013 - NWS poll completed (60s later) ✅
```
**Startup Log:**
```json
{"ts": "2026-05-16T03:54:14.318479+00:00", "msg": "Adapter started", "adapter": "nws", "cadence_s": 60}
```
### Bug Note
The cadence DECREASE (90→60) rate-limit test from Phase B showed correct
log output ("Rescheduled adapter" with new_cadence_s=60) but the actual
scheduling didn't update properly. The increase test (60→90) worked
correctly.
**Root cause:** Unknown - requires investigation. The `_reschedule_adapter`
method updates `state.config` and `state.adapter.cadence_s`, and signals
via `cancel_event`, but the scheduling loop may not be re-evaluating
correctly for decreases.
**Mitigation:** After any cadence change, verify actual poll intervals match
expected cadence. If not, restart supervisor.
**Result:** Cadence confirmed at 60s after restart. ✅
---
## Phase 1a-3 Close-out
**Timestamp:** 2026-05-16T04:03:17Z
### PR #3 Merge
- **Merge commit:** 0b23cc4
- **Strategy:** Merge commit (fast-forward)
- **Branch deleted:** feature/1a-3-phase-c-toml-retirement
### LXC Cleanup
**Remove obsolete env var:**
```bash
sed -i '/CENTRAL_CONFIG_SOURCE/d' /etc/central/central.env
```
**Resulting central.env:**
```
CENTRAL_DB_DSN=postgresql://central:***@localhost/central
CENTRAL_NATS_URL=nats://localhost:4222
CENTRAL_MASTER_KEY_PATH=/etc/central/master.key
CENTRAL_LOG_LEVEL=INFO
```
**Retire TOML file:**
```bash
mv /etc/central/central.toml /etc/central/central.toml.retired
```
**Directory listing:**
```
-rw-r----- central central 193 central.env
-rw-r----- central central 1074 central.toml.retired
-rw------- central central 45 master.key
```
### Post-Restart Verification
**Supervisor startup:**
```json
{"ts": "2026-05-16T04:01:18.430800+00:00", "msg": "Config source: db", "config_source": "db"}
{"ts": "2026-05-16T04:01:18.459241+00:00", "msg": "Adapter started", "adapter": "nws", "cadence_s": 60}
{"ts": "2026-05-16T04:01:18.459641+00:00", "msg": "Config listener connected to database"}
{"ts": "2026-05-16T04:01:18.595928+00:00", "msg": "NWS poll completed", "status": 200}
```
**Archive startup:**
```json
{"ts": "2026-05-16T04:01:48.442842+00:00", "msg": "Archive starting", "nats_url": "nats://localhost:4222"}
{"ts": "2026-05-16T04:01:48.468110+00:00", "msg": "Archive consumer ready"}
```
### CloudEvents Envelope Verification (seq 32)
```json
{
"type": "central.wx.alert.winter_weather_advisory.v1",
"source": "central.echo6.co",
"specversion": "1.0",
"centralschemaversion": "1.0",
"centralcategory": "wx.alert.winter_weather_advisory",
"centralseverity": 2
}
```
- Extension attributes use `central*` prefix ✅
### T3 Data Integrity Check
| Metric | T0 | T3 |
|--------|----|----|
| Upstream alerts | 16 | 17 |
| DB events | 30 | 32 |
| Missing | 0 | 0 |
**Result:** Zero alerts missed across T0 → T3. ✅
---
## Phase 1a-3 Final Summary
| Gate | Status |
|------|--------|
| Part 1: Cadence reverted to 60s | ✅ (required restart) |
| Part 2: PR #3 review - no blockers | ✅ |
| Part 3: PR #3 merged | ✅ (0b23cc4) |
| CENTRAL_CONFIG_SOURCE removed | ✅ |
| central.toml retired | ✅ |
| Services healthy | ✅ |
| CloudEvents central* prefix | ✅ |
| Data integrity T0→T3 | ✅ |
**Phase 1a-3 Complete.**
## Final Cadence-Decrease Fix Verification
**Date:** 2026-05-16T17:19-17:25 UTC
**Branch:** feature/remove-adapter-limiter
**Fix:** Removed internal AsyncLimiter from NWSAdapter
### Root Cause
The NWSAdapter had an internal AsyncLimiter(1, cadence_s) that duplicated
the supervisor rate-limit guarantee. When cadence changed via hot-reload,
state.adapter.cadence_s was updated but the internal _limiter retained
the old rate, causing the async with self._limiter context to block for
the remaining time of the old cadence window.
### Fix Applied
1. Removed self._limiter from NWSAdapter
2. Removed self.cadence_s attribute (no longer needed)
3. Removed state.adapter.cadence_s = new_cadence from supervisor
4. Removed aiolimiter dependency
### Verification Results
#### Test 1: Decrease 60 to 30s
```
Tlast: 17:20:38.282
Change: 17:20:39.649 (60 to 30)
Expected: 17:21:08.323 (Tlast + 30s)
Actual: 17:21:08.531 PASS
Subsequent: 17:21:38.751 (30s later) PASS
```
#### Test 2: Increase 30 to 60s
```
Tlast: 17:22:09.242
Change: 17:22:18.515 (30 to 60)
Expected: 17:23:09.284 (Tlast + 60s)
Actual: 17:23:09.634 PASS
```
#### Test 3: Decrease 60 to 15s
```
Tlast: 17:23:09.634
Change: 17:23:28.343 (60 to 15)
Expected: 17:23:24.677 (Tlast + 15s, already passed)
Actual: 17:23:28.736 (immediate, deadline passed) PASS
Subsequent: 17:23:44.129 (15s later) PASS
17:23:59.579 (15s later) PASS
```
#### Test 4: Restore 15 to 60s
```
Change: 17:24:21.355 (15 to 60)
Expected: 17:25:15.072 (Tlast + 60s)
```
### Journal Evidence
```
17:20:38 poll completed (baseline)
17:20:39 Rescheduled 60 to 30, next_poll=17:21:08
17:21:08 poll completed PASS (30s, not 60s)
17:21:38 poll completed PASS (30s interval)
17:22:09 poll completed
17:22:18 Rescheduled 30 to 60, next_poll=17:23:09
17:23:09 poll completed PASS (60s)
17:23:28 Rescheduled 60 to 15, next_poll=17:23:24 (past)
17:23:28 poll completed PASS (immediate)
17:23:44 poll completed PASS (15s)
17:23:59 poll completed PASS (15s)
17:24:21 Rescheduled 15 to 60, next_poll=17:25:15
```
### Conclusion
All cadence transitions work correctly:
- Decrease (60 to 30, 60 to 15): Next poll at Tlast + new_cadence PASS
- Increase (30 to 60, 15 to 60): Next poll at Tlast + new_cadence PASS
- Immediate poll when deadline already passed PASS
- Subsequent intervals use new cadence PASS
The internal AsyncLimiter was the root cause. Removing it allows the
supervisor rate-limit scheduling to work correctly without interference.
# Phase 1a-3 Verification Evidence
## T0 Baseline (TOML config mode, post-merge deploy)
**Timestamp:** 2026-05-16T03:10:51Z
### Upstream Alert IDs
```json
[
"urn:oid:2.49.0.1.840.0.e22a439ed29ed11e4b3686d9fac419ce7ad40059.001.1",
"urn:oid:2.49.0.1.840.0.b7acbf4f0381fb83c1b3f732a4ac9ca16a6204d1.002.1",
"urn:oid:2.49.0.1.840.0.e420a03d4bb13559e9bd61c714d8753fa6a4f66d.001.1",
"urn:oid:2.49.0.1.840.0.82fc471559645fcc3fefe49b4855bde43a7dde2b.001.1",
"urn:oid:2.49.0.1.840.0.add970d087c8d383436ee5958fc56100408aaf2e.001.1",
"urn:oid:2.49.0.1.840.0.f620e3599001fc9937324d55df89b55e475c5568.001.1",
"urn:oid:2.49.0.1.840.0.f620e3599001fc9937324d55df89b55e475c5568.002.1",
"urn:oid:2.49.0.1.840.0.dbde432f293a71618bf9908e5adcf9e5dd27e27c.006.1",
"urn:oid:2.49.0.1.840.0.dbde432f293a71618bf9908e5adcf9e5dd27e27c.001.1",
"urn:oid:2.49.0.1.840.0.dbde432f293a71618bf9908e5adcf9e5dd27e27c.003.1",
"urn:oid:2.49.0.1.840.0.dbde432f293a71618bf9908e5adcf9e5dd27e27c.001.2",
"urn:oid:2.49.0.1.840.0.dbde432f293a71618bf9908e5adcf9e5dd27e27c.005.1",
"urn:oid:2.49.0.1.840.0.b5173bc4f407f3889ea8e9284af261796d04972b.002.1",
"urn:oid:2.49.0.1.840.0.18277c28967847fb1b9e61f5afc236e42659e27b.001.1",
"urn:oid:2.49.0.1.840.0.b5173bc4f407f3889ea8e9284af261796d04972b.001.1",
"urn:oid:2.49.0.1.840.0.86299b43bf001e6c38df077a9b2d8d8e1e7b9116.002.2"
]
```
### Database State
```
count | max
-------+------------------------
30 | 2026-05-16 02:45:00+00
```
### Fresh Envelope Sample (post-restart)
```json
{
"id": "https://api.weather.gov/alerts/urn:oid:2.49.0.1.840.0.35f852d42f3149d3e1722c14e6ffc2e977e48d1b.001.1",
"source": "central/adapters/nws",
"type": "central.wx.alert.lake_wind_advisory.v1",
"time": "2026-05-16T02:45:00+00:00",
"datacontenttype": "application/json",
"centralschemaversion": "1.0.0",
"centralcategory": "wx.alert.lake_wind_advisory",
"centralseverity": 2,
"specversion": "1.0",
"data": { ... }
}
```
**CloudEvents verification:**
- `specversion: "1.0"` ✅
- `type` starts with `central.` (NOT `hub.`) ✅
- Extension attributes use `central*` prefix ✅
- `centralschemaversion` (NOT `hubschemaversion`)
- `centralcategory` (NOT `hubcategory`)
- `centralseverity` (NOT `hubseverity`)
---
## Phase B Step 2: Config Source Cutover (TOML → DB)
**Timestamp:** 2026-05-16T03:13:33Z
### Environment Change
```
# /etc/central/central.env - added:
CENTRAL_CONFIG_SOURCE=db
```
### Supervisor Journal Evidence
```json
{"ts": "2026-05-16T03:13:33.430635+00:00", "level": "INFO", "logger": "central.supervisor", "msg": "Config source: db", "config_source": "db"}
{"ts": "2026-05-16T03:13:33.460162+00:00", "level": "INFO", "logger": "central.config_store", "msg": "Config listener connected to database"}
```
### Archive Journal Evidence
```json
{"ts": "2026-05-16T03:14:03.413008+00:00", "level": "INFO", "logger": "central.archive", "msg": "Archive starting", "nats_url": "nats://localhost:4222", "config_source": "db"}
```
**Result:** Both services running with DB-backed config ✅
---
## Phase B Step 3: Hot-Reload Cadence Test
**Test:** Change cadence from 60s → 90s while adapter is running.
**Goal:** Verify next poll is at Tlast + new_cadence (not old cadence, not immediate).
### Timeline
```
Tlast (last poll): 03:16:34.317219Z
Config change: 03:16:36Z
Expected next poll: 03:18:04.317Z (Tlast + 90s)
Actual next poll: 03:18:04.502Z ✅
```
### Journal Evidence
```json
{"ts": "2026-05-16T03:16:34.317219+00:00", "level": "INFO", "logger": "central.adapters.nws", "msg": "NWS yielded events", "count": 16}
{"ts": "2026-05-16T03:16:37.488781+00:00", "level": "INFO", "logger": "central.supervisor", "msg": "Config change received", "table": "adapters", "key": "nws"}
{"ts": "2026-05-16T03:16:37.511029+00:00", "level": "INFO", "logger": "central.supervisor", "msg": "Rescheduled adapter", "adapter": "nws", "old_cadence_s": 60, "new_cadence_s": 90, "next_poll": "2026-05-16T03:18:04.317651+00:00"}
{"ts": "2026-05-16T03:18:04.502991+00:00", "level": "INFO", "logger": "central.adapters.nws", "msg": "NWS poll completed", "status": 200, "feature_count": 355}
```
**Result:** Rate-limit guarantee upheld. Poll occurred at Tlast + 90s (NOT Tlast + 60s). ✅
---
## Phase B Step 4: Hot-Reload Enable/Disable Test
**Test:** Disable adapter, wait, re-enable.
**Goal:** Verify next poll is at Tlast + cadence (not immediate on re-enable).
### Timeline
```
Tlast (last poll): 03:19:34.758524Z
Disabled at: 03:20:37Z
Re-enabled at: 03:20:48Z
Expected next poll: 03:21:04.758Z (Tlast + 90s)
Actual next poll: 03:21:04.940Z ✅
```
### Journal Evidence
```json
{"ts": "2026-05-16T03:19:34.757999+00:00", "level": "INFO", "logger": "central.adapters.nws", "msg": "NWS yielded events", "count": 16}
{"ts": "2026-05-16T03:20:37.616723+00:00", "level": "INFO", "logger": "central.supervisor", "msg": "Adapter stopped", "adapter": "nws", "preserved_last_poll": "2026-05-16T03:19:34.758524+00:00"}
{"ts": "2026-05-16T03:20:48.947358+00:00", "level": "INFO", "logger": "central.supervisor", "msg": "Adapter restarted", "adapter": "nws", "cadence_s": 90, "preserved_last_poll": "2026-05-16T03:19:34.758524+00:00", "next_poll": "2026-05-16T03:21:04.758524+00:00"}
{"ts": "2026-05-16T03:21:04.940891+00:00", "level": "INFO", "logger": "central.adapters.nws", "msg": "NWS poll completed", "status": 200, "feature_count": 354}
```
**Key observations:**
- `preserved_last_poll` appears in BOTH stop and restart logs (proves state retained)
- `next_poll` calculated from `preserved_last_poll + cadence_s` (not from current time)
- Poll did NOT happen immediately on re-enable
**Result:** Rate-limit guarantee upheld through enable/disable cycle. ✅
---
## Phase B Step 5: T1 Capture and Soak
**T1 Timestamp:** 2026-05-16T03:23:19Z
**T2 Timestamp:** 2026-05-16T03:33:48Z
### T1 State
- Upstream alerts: 16
- DB events: 30
### T2 State (after 10-minute soak)
- Upstream alerts: 16
- DB events: 30
### Poll Activity During Soak
```
03:24:05 - NWS poll completed, status: 200, feature_count: 355
03:25:35 - NWS poll completed, status: 200, feature_count: 357
03:27:05 - NWS poll completed, status: 200, feature_count: 358
03:28:35 - NWS poll completed, status: 200, feature_count: 360
03:30:05 - NWS poll completed, status: 200, feature_count: 357
03:31:35 - NWS poll completed, status: 200, feature_count: 356
03:33:05 - NWS poll completed, status: 200, feature_count: 355
```
**Errors during soak:** None ✅
---
## Phase B Step 6: Data Integrity Check
### Verification
```
Upstream alerts: 16
DB events (total): 30
Missing from DB: 0
All upstream alerts found in DB ✓
```
**Result:** Zero missed alerts. Data integrity confirmed. ✅
---
## Phase B Verification Summary
| Step | Test | Result |
|------|------|--------|
| 2 | Config source cutover | ✅ "Config source: db" in logs |
| 3 | Cadence hot-reload | ✅ Poll at Tlast + new_cadence |
| 4 | Enable/disable cycle | ✅ Rate-limit preserved |
| 5 | 10-minute soak | ✅ No errors |
| 6 | Data integrity | ✅ All alerts in DB |
**Phase B Complete.** System running stable on DB-backed config.
---
## Cadence Revert (Close-out)
**Timestamp:** 2026-05-16T03:54:14Z
### Issue Discovered
During close-out verification, polls were observed at 90s intervals despite
DB showing `cadence_s = 60`. Investigation revealed the live reschedule
from 90→60 (done at 03:23:08 during Phase B) didn't properly update the
in-flight scheduling.
### Resolution
Supervisor restart was required to clear stale state:
```bash
systemctl restart central-supervisor
```
### Post-Restart Verification
**DB State:**
```sql
SELECT name, cadence_s, updated_at FROM config.adapters WHERE name='nws';
```
```
name | cadence_s | updated_at
------+-----------+-------------------------------
nws | 60 | 2026-05-16 03:50:53.210963+00
```
**Poll Intervals After Restart:**
```
03:54:14.621376 - NWS poll completed (first poll after restart)
03:55:15.028963 - NWS poll completed (61s later) ✅
03:56:15.429013 - NWS poll completed (60s later) ✅
```
**Startup Log:**
```json
{"ts": "2026-05-16T03:54:14.318479+00:00", "msg": "Adapter started", "adapter": "nws", "cadence_s": 60}
```
### Bug Note
The cadence DECREASE (90→60) rate-limit test from Phase B showed correct
log output ("Rescheduled adapter" with new_cadence_s=60) but the actual
scheduling didn't update properly. The increase test (60→90) worked
correctly.
**Root cause:** Unknown - requires investigation. The `_reschedule_adapter`
method updates `state.config` and `state.adapter.cadence_s`, and signals
via `cancel_event`, but the scheduling loop may not be re-evaluating
correctly for decreases.
**Mitigation:** After any cadence change, verify actual poll intervals match
expected cadence. If not, restart supervisor.
**Result:** Cadence confirmed at 60s after restart. ✅
---
## Phase 1a-3 Close-out
**Timestamp:** 2026-05-16T04:03:17Z
### PR #3 Merge
- **Merge commit:** 0b23cc4
- **Strategy:** Merge commit (fast-forward)
- **Branch deleted:** feature/1a-3-phase-c-toml-retirement
### LXC Cleanup
**Remove obsolete env var:**
```bash
sed -i '/CENTRAL_CONFIG_SOURCE/d' /etc/central/central.env
```
**Resulting central.env:**
```
CENTRAL_DB_DSN=postgresql://central:***@localhost/central
CENTRAL_NATS_URL=nats://localhost:4222
CENTRAL_MASTER_KEY_PATH=/etc/central/master.key
CENTRAL_LOG_LEVEL=INFO
```
**Retire TOML file:**
```bash
mv /etc/central/central.toml /etc/central/central.toml.retired
```
**Directory listing:**
```
-rw-r----- central central 193 central.env
-rw-r----- central central 1074 central.toml.retired
-rw------- central central 45 master.key
```
### Post-Restart Verification
**Supervisor startup:**
```json
{"ts": "2026-05-16T04:01:18.430800+00:00", "msg": "Config source: db", "config_source": "db"}
{"ts": "2026-05-16T04:01:18.459241+00:00", "msg": "Adapter started", "adapter": "nws", "cadence_s": 60}
{"ts": "2026-05-16T04:01:18.459641+00:00", "msg": "Config listener connected to database"}
{"ts": "2026-05-16T04:01:18.595928+00:00", "msg": "NWS poll completed", "status": 200}
```
**Archive startup:**
```json
{"ts": "2026-05-16T04:01:48.442842+00:00", "msg": "Archive starting", "nats_url": "nats://localhost:4222"}
{"ts": "2026-05-16T04:01:48.468110+00:00", "msg": "Archive consumer ready"}
```
### CloudEvents Envelope Verification (seq 32)
```json
{
"type": "central.wx.alert.winter_weather_advisory.v1",
"source": "central.echo6.co",
"specversion": "1.0",
"centralschemaversion": "1.0",
"centralcategory": "wx.alert.winter_weather_advisory",
"centralseverity": 2
}
```
- Extension attributes use `central*` prefix ✅
### T3 Data Integrity Check
| Metric | T0 | T3 |
|--------|----|----|
| Upstream alerts | 16 | 17 |
| DB events | 30 | 32 |
| Missing | 0 | 0 |
**Result:** Zero alerts missed across T0 → T3. ✅
---
## Phase 1a-3 Final Summary
| Gate | Status |
|------|--------|
| Part 1: Cadence reverted to 60s | ✅ (required restart) |
| Part 2: PR #3 review - no blockers | ✅ |
| Part 3: PR #3 merged | ✅ (0b23cc4) |
| CENTRAL_CONFIG_SOURCE removed | ✅ |
| central.toml retired | ✅ |
| Services healthy | ✅ |
| CloudEvents central* prefix | ✅ |
| Data integrity T0→T3 | ✅ |
**Phase 1a-3 Complete.**
## Final Cadence-Decrease Fix Verification
**Date:** 2026-05-16T17:19-17:25 UTC
**Branch:** feature/remove-adapter-limiter
**Fix:** Removed internal AsyncLimiter from NWSAdapter
### Root Cause
The NWSAdapter had an internal AsyncLimiter(1, cadence_s) that duplicated
the supervisor rate-limit guarantee. When cadence changed via hot-reload,
state.adapter.cadence_s was updated but the internal _limiter retained
the old rate, causing the async with self._limiter context to block for
the remaining time of the old cadence window.
### Fix Applied
1. Removed self._limiter from NWSAdapter
2. Removed self.cadence_s attribute (no longer needed)
3. Removed state.adapter.cadence_s = new_cadence from supervisor
4. Removed aiolimiter dependency
### Verification Results
#### Test 1: Decrease 60 to 30s
```
Tlast: 17:20:38.282
Change: 17:20:39.649 (60 to 30)
Expected: 17:21:08.323 (Tlast + 30s)
Actual: 17:21:08.531 PASS
Subsequent: 17:21:38.751 (30s later) PASS
```
#### Test 2: Increase 30 to 60s
```
Tlast: 17:22:09.242
Change: 17:22:18.515 (30 to 60)
Expected: 17:23:09.284 (Tlast + 60s)
Actual: 17:23:09.634 PASS
```
#### Test 3: Decrease 60 to 15s
```
Tlast: 17:23:09.634
Change: 17:23:28.343 (60 to 15)
Expected: 17:23:24.677 (Tlast + 15s, already passed)
Actual: 17:23:28.736 (immediate, deadline passed) PASS
Subsequent: 17:23:44.129 (15s later) PASS
17:23:59.579 (15s later) PASS
```
#### Test 4: Restore 15 to 60s
```
Change: 17:24:21.355 (15 to 60)
Expected: 17:25:15.072 (Tlast + 60s)
```
### Journal Evidence
```
17:20:38 poll completed (baseline)
17:20:39 Rescheduled 60 to 30, next_poll=17:21:08
17:21:08 poll completed PASS (30s, not 60s)
17:21:38 poll completed PASS (30s interval)
17:22:09 poll completed
17:22:18 Rescheduled 30 to 60, next_poll=17:23:09
17:23:09 poll completed PASS (60s)
17:23:28 Rescheduled 60 to 15, next_poll=17:23:24 (past)
17:23:28 poll completed PASS (immediate)
17:23:44 poll completed PASS (15s)
17:23:59 poll completed PASS (15s)
17:24:21 Rescheduled 15 to 60, next_poll=17:25:15
```
### Conclusion
All cadence transitions work correctly:
- Decrease (60 to 30, 60 to 15): Next poll at Tlast + new_cadence PASS
- Increase (30 to 60, 15 to 60): Next poll at Tlast + new_cadence PASS
- Immediate poll when deadline already passed PASS
- Subsequent intervals use new cadence PASS
The internal AsyncLimiter was the root cause. Removing it allows the
supervisor rate-limit scheduling to work correctly without interference.

192
docs/environment.md
View file

@ -1,96 +1,96 @@
# Central Data Hub - Environment Reference
## Development Locations
### Active Development: CT104 (Central LXC)
All development work happens on the Central LXC container:
| Property | Value |
|----------|-------|
| **Hostname** | `central` |
| **Tailscale IP** | `100.64.0.12` |
| **LAN IP** | `192.168.1.104` |
| **SSH access** | `zvx@central` or `zvx@100.64.0.12` |
| **Repository path** | `/opt/central` |
| **Python venv** | `/opt/central/.venv` |
| **Services** | `central-supervisor`, `central-archive` |
### Parked Clone: Cortex
The cortex VM at `/home/zvx/projects/central` contains a clone that is
**not actively used for development**. It may be retired in the future.
Do not make changes there.
### Local Workstation: matt-desktop
The Windows workstation (matt-desktop) has no Central repository clones.
The directory `C:\Users\mtthw\central_work\` is scratch space only and
should not be used for commits.
## Repository
| Property | Value |
|----------|-------|
| **Origin** | `git@github.com:zvx-echo6/central.git` |
| **Main branch** | `main` |
| **Default user** | `central` (on CT104) |
## Services
### central-supervisor
The main adapter scheduler and event publisher. Polls upstream APIs,
normalizes events, and publishes to NATS JetStream.
```bash
# Status
systemctl status central-supervisor
# Logs
journalctl -u central-supervisor -f
# Restart (requires sudo)
sudo systemctl restart central-supervisor
```
### central-archive
Consumes events from NATS JetStream and archives to PostgreSQL/TimescaleDB.
```bash
# Status
systemctl status central-archive
# Logs
journalctl -u central-archive -f
```
## Database
PostgreSQL 16 with TimescaleDB runs on CT104:
```bash
# Connect as central user
psql -h localhost -U central -d central
# Check adapter config
SELECT name, cadence_s, enabled FROM config.adapters;
# Check recent events
SELECT id, time, category FROM events ORDER BY time DESC LIMIT 10;
```
## SSH Access from Windows
From matt-desktop, connect via Tailscale:
```bash
# Direct connection
ssh zvx@100.64.0.12
# Using hostname (if Tailscale DNS configured)
ssh zvx@central
```
Note: The `zvx` user requires password for sudo operations.
# Central Data Hub - Environment Reference
## Development Locations
### Active Development: CT104 (Central LXC)
All development work happens on the Central LXC container:
| Property | Value |
|----------|-------|
| **Hostname** | `central` |
| **Tailscale IP** | `100.64.0.12` |
| **LAN IP** | `192.168.1.104` |
| **SSH access** | `zvx@central` or `zvx@100.64.0.12` |
| **Repository path** | `/opt/central` |
| **Python venv** | `/opt/central/.venv` |
| **Services** | `central-supervisor`, `central-archive` |
### Parked Clone: Cortex
The cortex VM at `/home/zvx/projects/central` contains a clone that is
**not actively used for development**. It may be retired in the future.
Do not make changes there.
### Local Workstation: matt-desktop
The Windows workstation (matt-desktop) has no Central repository clones.
The directory `C:\Users\mtthw\central_work\` is scratch space only and
should not be used for commits.
## Repository
| Property | Value |
|----------|-------|
| **Origin** | `git@github.com:zvx-echo6/central.git` |
| **Main branch** | `main` |
| **Default user** | `central` (on CT104) |
## Services
### central-supervisor
The main adapter scheduler and event publisher. Polls upstream APIs,
normalizes events, and publishes to NATS JetStream.
```bash
# Status
systemctl status central-supervisor
# Logs
journalctl -u central-supervisor -f
# Restart (requires sudo)
sudo systemctl restart central-supervisor
```
### central-archive
Consumes events from NATS JetStream and archives to PostgreSQL/TimescaleDB.
```bash
# Status
systemctl status central-archive
# Logs
journalctl -u central-archive -f
```
## Database
PostgreSQL 16 with TimescaleDB runs on CT104:
```bash
# Connect as central user
psql -h localhost -U central -d central
# Check adapter config
SELECT name, cadence_s, enabled FROM config.adapters;
# Check recent events
SELECT id, time, category FROM events ORDER BY time DESC LIMIT 10;
```
## SSH Access from Windows
From matt-desktop, connect via Tailscale:
```bash
# Direct connection
ssh zvx@100.64.0.12
# Using hostname (if Tailscale DNS configured)
ssh zvx@central
```
Note: The `zvx` user requires password for sudo operations.

128
sql/migrations/001_create_config_schema.sql
View file

@ -1,64 +1,64 @@
-- Migration: 001_create_config_schema
-- Creates the config schema with adapters and api_keys tables.
-- Also seeds the NWS adapter row from current TOML config.
-- Create config schema
CREATE SCHEMA config;
-- Adapters configuration table
CREATE TABLE config.adapters (
name TEXT PRIMARY KEY,
enabled BOOLEAN NOT NULL DEFAULT true,
cadence_s INTEGER NOT NULL,
settings JSONB NOT NULL DEFAULT '{}'::jsonb,
paused_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- API keys table (encrypted values)
CREATE TABLE config.api_keys (
alias TEXT PRIMARY KEY,
encrypted_value BYTEA NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
rotated_at TIMESTAMPTZ,
last_used_at TIMESTAMPTZ
);
-- Notify function for config changes
CREATE OR REPLACE FUNCTION config.notify_config_change()
RETURNS trigger AS $$
DECLARE
key_value TEXT;
BEGIN
-- Handle different table structures
IF TG_TABLE_NAME = 'adapters' THEN
key_value := COALESCE(NEW.name, OLD.name, '');
ELSIF TG_TABLE_NAME = 'api_keys' THEN
key_value := COALESCE(NEW.alias, OLD.alias, '');
ELSE
key_value := '';
END IF;
PERFORM pg_notify('config_changed', TG_TABLE_NAME || ':' || key_value);
RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql;
-- Trigger for adapters table
CREATE TRIGGER adapters_notify
AFTER INSERT OR UPDATE OR DELETE ON config.adapters
FOR EACH ROW EXECUTE FUNCTION config.notify_config_change();
-- Trigger for api_keys table
CREATE TRIGGER api_keys_notify
AFTER INSERT OR UPDATE OR DELETE ON config.api_keys
FOR EACH ROW EXECUTE FUNCTION config.notify_config_change();
-- Seed NWS adapter from current TOML config values
INSERT INTO config.adapters (name, enabled, cadence_s, settings)
VALUES (
'nws',
true,
60,
'{"states": ["ID", "OR", "WA", "MT", "WY", "UT", "NV"], "contact_email": "mj@k7zvx.com"}'::jsonb
);
-- Migration: 001_create_config_schema
-- Creates the config schema with adapters and api_keys tables.
-- Also seeds the NWS adapter row from current TOML config.
-- Create config schema
CREATE SCHEMA config;
-- Adapters configuration table
CREATE TABLE config.adapters (
name TEXT PRIMARY KEY,
enabled BOOLEAN NOT NULL DEFAULT true,
cadence_s INTEGER NOT NULL,
settings JSONB NOT NULL DEFAULT '{}'::jsonb,
paused_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- API keys table (encrypted values)
CREATE TABLE config.api_keys (
alias TEXT PRIMARY KEY,
encrypted_value BYTEA NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
rotated_at TIMESTAMPTZ,
last_used_at TIMESTAMPTZ
);
-- Notify function for config changes
CREATE OR REPLACE FUNCTION config.notify_config_change()
RETURNS trigger AS $$
DECLARE
key_value TEXT;
BEGIN
-- Handle different table structures
IF TG_TABLE_NAME = 'adapters' THEN
key_value := COALESCE(NEW.name, OLD.name, '');
ELSIF TG_TABLE_NAME = 'api_keys' THEN
key_value := COALESCE(NEW.alias, OLD.alias, '');
ELSE
key_value := '';
END IF;
PERFORM pg_notify('config_changed', TG_TABLE_NAME || ':' || key_value);
RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql;
-- Trigger for adapters table
CREATE TRIGGER adapters_notify
AFTER INSERT OR UPDATE OR DELETE ON config.adapters
FOR EACH ROW EXECUTE FUNCTION config.notify_config_change();
-- Trigger for api_keys table
CREATE TRIGGER api_keys_notify
AFTER INSERT OR UPDATE OR DELETE ON config.api_keys
FOR EACH ROW EXECUTE FUNCTION config.notify_config_change();
-- Seed NWS adapter from current TOML config values
INSERT INTO config.adapters (name, enabled, cadence_s, settings)
VALUES (
'nws',
true,
60,
'{"states": ["ID", "OR", "WA", "MT", "WY", "UT", "NV"], "contact_email": "mj@k7zvx.com"}'::jsonb
);

92
sql/migrations/003_add_streams_table.sql
View file

@ -1,46 +1,46 @@
-- Migration: 003_add_streams_table
-- Creates the config.streams table for JetStream stream retention configuration.
-- Uses column-filtered NOTIFY to prevent self-loop when supervisor updates max_bytes.
-- Streams configuration table
CREATE TABLE config.streams (
name TEXT PRIMARY KEY,
max_age_s BIGINT NOT NULL,
max_bytes BIGINT NOT NULL DEFAULT 1073741824, -- 1GB default
managed_max_bytes BOOLEAN NOT NULL DEFAULT true,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- Auto-update trigger for updated_at
CREATE TRIGGER streams_set_updated_at
BEFORE UPDATE ON config.streams
FOR EACH ROW EXECUTE FUNCTION config.set_updated_at();
-- Column-filtered NOTIFY trigger for streams.
-- Fires on INSERT/DELETE always.
-- On UPDATE, only fires when max_age_s changes (operator-touchable field),
-- NOT when max_bytes changes (supervisor-managed), to prevent recompute loop.
CREATE OR REPLACE FUNCTION config.notify_streams_change()
RETURNS trigger AS $$
BEGIN
IF TG_OP = 'INSERT' OR TG_OP = 'DELETE' THEN
PERFORM pg_notify('config_changed', 'streams:' ||
COALESCE(NEW.name, OLD.name));
ELSIF TG_OP = 'UPDATE' AND
OLD.max_age_s IS DISTINCT FROM NEW.max_age_s THEN
PERFORM pg_notify('config_changed', 'streams:' || NEW.name);
END IF;
RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER streams_notify
AFTER INSERT OR UPDATE OR DELETE ON config.streams
FOR EACH ROW EXECUTE FUNCTION config.notify_streams_change();
-- Seed with current stream values from investigation
-- CENTRAL_WX: 7d max_age (604800s), 10GB max_bytes (will be clamped to 6GB on first recompute)
-- CENTRAL_META: 1d max_age (86400s), 100MB max_bytes (will be raised to 1GB floor)
INSERT INTO config.streams (name, max_age_s, max_bytes) VALUES
('CENTRAL_WX', 604800, 10737418240),
('CENTRAL_META', 86400, 104857600);
-- Migration: 003_add_streams_table
-- Creates the config.streams table for JetStream stream retention configuration.
-- Uses column-filtered NOTIFY to prevent self-loop when supervisor updates max_bytes.
-- Streams configuration table
CREATE TABLE config.streams (
name TEXT PRIMARY KEY,
max_age_s BIGINT NOT NULL,
max_bytes BIGINT NOT NULL DEFAULT 1073741824, -- 1GB default
managed_max_bytes BOOLEAN NOT NULL DEFAULT true,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- Auto-update trigger for updated_at
CREATE TRIGGER streams_set_updated_at
BEFORE UPDATE ON config.streams
FOR EACH ROW EXECUTE FUNCTION config.set_updated_at();
-- Column-filtered NOTIFY trigger for streams.
-- Fires on INSERT/DELETE always.
-- On UPDATE, only fires when max_age_s changes (operator-touchable field),
-- NOT when max_bytes changes (supervisor-managed), to prevent recompute loop.
CREATE OR REPLACE FUNCTION config.notify_streams_change()
RETURNS trigger AS $$
BEGIN
IF TG_OP = 'INSERT' OR TG_OP = 'DELETE' THEN
PERFORM pg_notify('config_changed', 'streams:' ||
COALESCE(NEW.name, OLD.name));
ELSIF TG_OP = 'UPDATE' AND
OLD.max_age_s IS DISTINCT FROM NEW.max_age_s THEN
PERFORM pg_notify('config_changed', 'streams:' || NEW.name);
END IF;
RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER streams_notify
AFTER INSERT OR UPDATE OR DELETE ON config.streams
FOR EACH ROW EXECUTE FUNCTION config.notify_streams_change();
-- Seed with current stream values from investigation
-- CENTRAL_WX: 7d max_age (604800s), 10GB max_bytes (will be clamped to 6GB on first recompute)
-- CENTRAL_META: 1d max_age (86400s), 100MB max_bytes (will be raised to 1GB floor)
INSERT INTO config.streams (name, max_age_s, max_bytes) VALUES
('CENTRAL_WX', 604800, 10737418240),
('CENTRAL_META', 86400, 104857600);

22
sql/migrations/004_nws_states_to_bbox.sql
View file

@ -1,11 +1,11 @@
-- Migration: 004_nws_states_to_bbox
-- Converts NWS adapter settings from states list to region bbox.
-- Bbox covers ID/OR/WA/MT/WY/UT/NV with buffer.
UPDATE config.adapters
SET settings = jsonb_set(
settings - 'states', -- Remove states key
'{region}',
'{"north": 49.5, "south": 31.0, "east": -102.0, "west": -124.5}'::jsonb
)
WHERE name = 'nws';
-- Migration: 004_nws_states_to_bbox
-- Converts NWS adapter settings from states list to region bbox.
-- Bbox covers ID/OR/WA/MT/WY/UT/NV with buffer.
UPDATE config.adapters
SET settings = jsonb_set(
settings - 'states', -- Remove states key
'{region}',
'{"north": 49.5, "south": 31.0, "east": -102.0, "west": -124.5}'::jsonb
)
WHERE name = 'nws';

860
src/central/adapters/firms.py
View file

@ -1,430 +1,430 @@
"""FIRMS (Fire Information for Resource Management System) adapter."""
import csv
import logging
import sqlite3
from collections.abc import AsyncIterator
from datetime import datetime, timezone
from io import StringIO
from pathlib import Path
from typing import Any
import aiohttp
from tenacity import (
retry,
stop_after_attempt,
wait_exponential_jitter,
retry_if_exception_type,
)
from central.adapter import SourceAdapter
from central.config_models import AdapterConfig, RegionConfig
from central.config_store import ConfigStore
from central.models import Event, Geo
logger = logging.getLogger(__name__)
# FIRMS API base URL
FIRMS_API_BASE = "https://firms.modaps.eosdis.nasa.gov/api/area/csv"
# Satellite name mapping
SATELLITE_SHORT = {
"VIIRS_SNPP_NRT": "viirs_snpp",
"VIIRS_NOAA20_NRT": "viirs_noaa20",
"VIIRS_NOAA21_NRT": "viirs_noaa21",
}
# Confidence mapping
CONFIDENCE_MAP = {
"l": "low",
"n": "nominal",
"h": "high",
}
# Severity mapping (confidence -> severity level)
SEVERITY_MAP = {
"high": 3,
"nominal": 2,
"low": 1,
}
class FIRMSAdapter(SourceAdapter):
"""NASA FIRMS fire hotspot adapter."""
name = "firms"
def __init__(
self,
config: AdapterConfig,
config_store: ConfigStore,
cursor_db_path: Path,
) -> None:
self._config_store = config_store
self._cursor_db_path = cursor_db_path
self._session: aiohttp.ClientSession | None = None
self._db: sqlite3.Connection | None = None
self._api_key: str | None = None
# Extract settings from config
self._api_key_alias: str = config.settings.get("api_key_alias", "firms")
self._satellites: list[str] = config.settings.get(
"satellites", ["VIIRS_SNPP_NRT", "VIIRS_NOAA20_NRT"]
)
# Parse region from settings
region_dict = config.settings.get("region")
if region_dict:
self.region: RegionConfig | None = RegionConfig(**region_dict)
else:
self.region = None
async def apply_config(self, new_config: AdapterConfig) -> None:
"""Apply new configuration from hot-reload."""
old_alias = self._api_key_alias
# Update settings
self._api_key_alias = new_config.settings.get("api_key_alias", "firms")
self._satellites = new_config.settings.get(
"satellites", ["VIIRS_SNPP_NRT", "VIIRS_NOAA20_NRT"]
)
# Update region
region_dict = new_config.settings.get("region")
if region_dict:
self.region = RegionConfig(**region_dict)
else:
self.region = None
# If API key alias changed, re-fetch the key
if self._api_key_alias != old_alias:
self._api_key = await self._config_store.get_api_key(self._api_key_alias)
if self._api_key:
logger.info("FIRMS API key reloaded", extra={"alias": self._api_key_alias})
else:
logger.warning(
"FIRMS API key not found after alias change",
extra={"alias": self._api_key_alias},
)
logger.info(
"FIRMS config applied",
extra={
"region": region_dict,
"satellites": self._satellites,
"api_key_alias": self._api_key_alias,
},
)
async def startup(self) -> None:
"""Initialize HTTP session, dedup tracker, and fetch API key."""
# Fetch API key
self._api_key = await self._config_store.get_api_key(self._api_key_alias)
if not self._api_key:
logger.error(
"FIRMS API key not found - polling will be skipped until key is set",
extra={"alias": self._api_key_alias},
)
# Initialize HTTP session
self._session = aiohttp.ClientSession(
timeout=aiohttp.ClientTimeout(total=60),
)
# Initialize dedup tracker (shared sqlite DB with NWS)
self._db = sqlite3.connect(str(self._cursor_db_path))
self._db.execute("""
CREATE TABLE IF NOT EXISTS published_ids (
adapter TEXT NOT NULL,
event_id TEXT NOT NULL,
first_seen TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_seen TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (adapter, event_id)
)
""")
self._db.execute("""
CREATE INDEX IF NOT EXISTS published_ids_last_seen
ON published_ids (last_seen)
""")
self._db.commit()
# Sweep old entries on startup (48h for FIRMS)
self.sweep_old_ids()
logger.info(
"FIRMS adapter started",
extra={
"region": {
"north": self.region.north,
"south": self.region.south,
"east": self.region.east,
"west": self.region.west,
} if self.region else None,
"satellites": self._satellites,
"api_key_present": self._api_key is not None,
},
)
async def shutdown(self) -> None:
"""Close HTTP session and database."""
if self._session:
await self._session.close()
self._session = None
if self._db:
self._db.close()
self._db = None
logger.info("FIRMS adapter shut down")
def is_published(self, stable_id: str) -> bool:
"""Check if an event has already been published."""
if not self._db:
return False
cur = self._db.execute(
"SELECT 1 FROM published_ids WHERE adapter = ? AND event_id = ?",
(self.name, stable_id),
)
return cur.fetchone() is not None
def mark_published(self, stable_id: str) -> None:
"""Mark an event as published."""
if not self._db:
return
self._db.execute(
"""
INSERT INTO published_ids (adapter, event_id, first_seen, last_seen)
VALUES (?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
ON CONFLICT (adapter, event_id) DO UPDATE SET
last_seen = CURRENT_TIMESTAMP
""",
(self.name, stable_id),
)
self._db.commit()
def sweep_old_ids(self) -> int:
"""Remove published_ids older than 48 hours. Returns count deleted."""
if not self._db:
return 0
cur = self._db.execute(
"DELETE FROM published_ids WHERE adapter = ? AND last_seen < datetime('now', '-48 hours')",
(self.name,),
)
self._db.commit()
count = cur.rowcount
if count > 0:
logger.info("FIRMS swept old dedup entries", extra={"count": count})
return count
def _build_stable_id(
self, satellite: str, acq_date: str, acq_time: str, lat: float, lon: float
) -> str:
"""Build stable ID for deduplication."""
# Round lat/lon to 0.001 degrees to handle floating-point comparison
lat_rounded = round(lat, 3)
lon_rounded = round(lon, 3)
return f"{satellite}:{acq_date}:{acq_time}:{lat_rounded}:{lon_rounded}"
def _build_url(self, satellite: str) -> str | None:
"""Build FIRMS API URL for a satellite."""
if not self._api_key or not self.region:
return None
# Area format: west,south,east,north
area = f"{self.region.west},{self.region.south},{self.region.east},{self.region.north}"
return f"{FIRMS_API_BASE}/{self._api_key}/{satellite}/{area}/1"
@retry(
stop=stop_after_attempt(3),
wait=wait_exponential_jitter(initial=2, max=30),
retry=retry_if_exception_type((aiohttp.ClientError,)),
reraise=True,
)
async def _fetch_csv(self, url: str) -> str:
"""Fetch CSV data from FIRMS API."""
if not self._session:
raise RuntimeError("Session not initialized")
async with self._session.get(url) as resp:
# Check for error responses
content_type = resp.headers.get("Content-Type", "")
if "text/html" in content_type:
text = await resp.text()
logger.error(
"FIRMS returned HTML (likely auth error)",
extra={"status": resp.status, "preview": text[:200]},
)
raise ValueError("FIRMS returned HTML instead of CSV")
resp.raise_for_status()
return await resp.text()
def _parse_csv(self, csv_text: str, satellite: str) -> list[dict[str, Any]]:
"""Parse FIRMS CSV response into list of dicts."""
rows = []
reader = csv.DictReader(StringIO(csv_text))
for row in reader:
try:
# Parse required fields
lat = float(row["latitude"])
lon = float(row["longitude"])
acq_date = row["acq_date"]
acq_time = row["acq_time"]
confidence_raw = row.get("confidence", "n").lower()
confidence = CONFIDENCE_MAP.get(confidence_raw, "nominal")
rows.append({
"latitude": lat,
"longitude": lon,
"bright_ti4": float(row.get("bright_ti4", 0)) if row.get("bright_ti4") else None,
"bright_ti5": float(row.get("bright_ti5", 0)) if row.get("bright_ti5") else None,
"scan": float(row.get("scan", 0)) if row.get("scan") else None,
"track": float(row.get("track", 0)) if row.get("track") else None,
"acq_date": acq_date,
"acq_time": acq_time,
"satellite": row.get("satellite", satellite),
"instrument": row.get("instrument", "VIIRS"),
"confidence": confidence,
"confidence_raw": confidence_raw,
"version": row.get("version", ""),
"frp": float(row.get("frp", 0)) if row.get("frp") else None,
"daynight": row.get("daynight", ""),
})
except (KeyError, ValueError) as e:
logger.warning(
"Failed to parse FIRMS row",
extra={"error": str(e), "row": dict(row)},
)
continue
return rows
def _row_to_event(self, row: dict[str, Any], satellite: str) -> Event:
"""Convert a parsed CSV row to an Event."""
satellite_short = SATELLITE_SHORT.get(satellite, satellite.lower().replace("_nrt", ""))
confidence = row["confidence"]
severity = SEVERITY_MAP.get(confidence, 1)
# Parse acquisition time
acq_date = row["acq_date"]
acq_time = row["acq_time"]
# acq_time is HHMM format
try:
time = datetime.strptime(
f"{acq_date} {acq_time}", "%Y-%m-%d %H%M"
).replace(tzinfo=timezone.utc)
except ValueError:
time = datetime.now(timezone.utc)
lat = row["latitude"]
lon = row["longitude"]
# Build stable ID
stable_id = self._build_stable_id(satellite, acq_date, acq_time, lat, lon)
geo = Geo(
centroid=(lon, lat), # GeoJSON order: lon, lat
bbox=(lon, lat, lon, lat), # Point bbox
regions=[],
primary_region=None,
)
return Event(
id=stable_id,
source="central/adapters/firms",
category=f"fire.hotspot.{satellite_short}.{confidence}",
time=time,
expires=None,
severity=severity,
geo=geo,
data=row,
)
async def poll(self) -> AsyncIterator[Event]:
"""Poll FIRMS API for fire hotspots."""
# Check API key
if not self._api_key:
# Try to fetch again in case it was added
self._api_key = await self._config_store.get_api_key(self._api_key_alias)
if not self._api_key:
logger.warning(
"FIRMS API key still not available, skipping poll",
extra={"alias": self._api_key_alias},
)
return
if not self.region:
logger.warning("FIRMS region not configured, skipping poll")
return
# Sweep old dedup entries periodically
self.sweep_old_ids()
total_features = 0
total_new = 0
for satellite in self._satellites:
url = self._build_url(satellite)
if not url:
continue
try:
csv_text = await self._fetch_csv(url)
rows = self._parse_csv(csv_text, satellite)
feature_count = len(rows)
total_features += feature_count
new_count = 0
for row in rows:
stable_id = self._build_stable_id(
satellite,
row["acq_date"],
row["acq_time"],
row["latitude"],
row["longitude"],
)
if self.is_published(stable_id):
continue
event = self._row_to_event(row, satellite)
yield event
self.mark_published(stable_id)
new_count += 1
total_new += new_count
logger.info(
"FIRMS satellite poll completed",
extra={
"satellite": satellite,
"feature_count": feature_count,
"new_count": new_count,
},
)
except Exception as e:
logger.error(
"FIRMS poll failed for satellite",
extra={"satellite": satellite, "error": str(e)},
)
continue
logger.info(
"FIRMS poll completed",
extra={
"total_features": total_features,
"total_new": total_new,
"satellites": self._satellites,
},
)
def subject_for_fire_hotspot(ev: Event) -> str:
"""Compute the NATS subject for a fire hotspot event.
Subject format: central.fire.hotspot.<satellite>.<confidence>
The category already contains the satellite and confidence info,
so we just prefix with 'central.'.
"""
# category is "fire.hotspot.<satellite>.<confidence>"
return f"central.{ev.category}"
"""FIRMS (Fire Information for Resource Management System) adapter."""
import csv
import logging
import sqlite3
from collections.abc import AsyncIterator
from datetime import datetime, timezone
from io import StringIO
from pathlib import Path
from typing import Any
import aiohttp
from tenacity import (
retry,
stop_after_attempt,
wait_exponential_jitter,
retry_if_exception_type,
)
from central.adapter import SourceAdapter
from central.config_models import AdapterConfig, RegionConfig
from central.config_store import ConfigStore
from central.models import Event, Geo
logger = logging.getLogger(__name__)
# FIRMS API base URL
FIRMS_API_BASE = "https://firms.modaps.eosdis.nasa.gov/api/area/csv"
# Satellite name mapping
SATELLITE_SHORT = {
"VIIRS_SNPP_NRT": "viirs_snpp",
"VIIRS_NOAA20_NRT": "viirs_noaa20",
"VIIRS_NOAA21_NRT": "viirs_noaa21",
}
# Confidence mapping
CONFIDENCE_MAP = {
"l": "low",
"n": "nominal",
"h": "high",
}
# Severity mapping (confidence -> severity level)
SEVERITY_MAP = {
"high": 3,
"nominal": 2,
"low": 1,
}
class FIRMSAdapter(SourceAdapter):
"""NASA FIRMS fire hotspot adapter."""
name = "firms"
def __init__(
self,
config: AdapterConfig,
config_store: ConfigStore,
cursor_db_path: Path,
) -> None:
self._config_store = config_store
self._cursor_db_path = cursor_db_path
self._session: aiohttp.ClientSession | None = None
self._db: sqlite3.Connection | None = None
self._api_key: str | None = None
# Extract settings from config
self._api_key_alias: str = config.settings.get("api_key_alias", "firms")
self._satellites: list[str] = config.settings.get(
"satellites", ["VIIRS_SNPP_NRT", "VIIRS_NOAA20_NRT"]
)
# Parse region from settings
region_dict = config.settings.get("region")
if region_dict:
self.region: RegionConfig | None = RegionConfig(**region_dict)
else:
self.region = None
async def apply_config(self, new_config: AdapterConfig) -> None:
"""Apply new configuration from hot-reload."""
old_alias = self._api_key_alias
# Update settings
self._api_key_alias = new_config.settings.get("api_key_alias", "firms")
self._satellites = new_config.settings.get(
"satellites", ["VIIRS_SNPP_NRT", "VIIRS_NOAA20_NRT"]
)
# Update region
region_dict = new_config.settings.get("region")
if region_dict:
self.region = RegionConfig(**region_dict)
else:
self.region = None
# If API key alias changed, re-fetch the key
if self._api_key_alias != old_alias:
self._api_key = await self._config_store.get_api_key(self._api_key_alias)
if self._api_key:
logger.info("FIRMS API key reloaded", extra={"alias": self._api_key_alias})
else:
logger.warning(
"FIRMS API key not found after alias change",
extra={"alias": self._api_key_alias},
)
logger.info(
"FIRMS config applied",
extra={
"region": region_dict,
"satellites": self._satellites,
"api_key_alias": self._api_key_alias,
},
)
async def startup(self) -> None:
"""Initialize HTTP session, dedup tracker, and fetch API key."""
# Fetch API key
self._api_key = await self._config_store.get_api_key(self._api_key_alias)
if not self._api_key:
logger.error(
"FIRMS API key not found - polling will be skipped until key is set",
extra={"alias": self._api_key_alias},
)
# Initialize HTTP session
self._session = aiohttp.ClientSession(
timeout=aiohttp.ClientTimeout(total=60),
)
# Initialize dedup tracker (shared sqlite DB with NWS)
self._db = sqlite3.connect(str(self._cursor_db_path))
self._db.execute("""
CREATE TABLE IF NOT EXISTS published_ids (
adapter TEXT NOT NULL,
event_id TEXT NOT NULL,
first_seen TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_seen TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (adapter, event_id)
)
""")
self._db.execute("""
CREATE INDEX IF NOT EXISTS published_ids_last_seen
ON published_ids (last_seen)
""")
self._db.commit()
# Sweep old entries on startup (48h for FIRMS)
self.sweep_old_ids()
logger.info(
"FIRMS adapter started",
extra={
"region": {
"north": self.region.north,
"south": self.region.south,
"east": self.region.east,
"west": self.region.west,
} if self.region else None,
"satellites": self._satellites,
"api_key_present": self._api_key is not None,
},
)
async def shutdown(self) -> None:
"""Close HTTP session and database."""
if self._session:
await self._session.close()
self._session = None
if self._db:
self._db.close()
self._db = None
logger.info("FIRMS adapter shut down")
def is_published(self, stable_id: str) -> bool:
"""Check if an event has already been published."""
if not self._db:
return False
cur = self._db.execute(
"SELECT 1 FROM published_ids WHERE adapter = ? AND event_id = ?",
(self.name, stable_id),
)
return cur.fetchone() is not None
def mark_published(self, stable_id: str) -> None:
"""Mark an event as published."""
if not self._db:
return
self._db.execute(
"""
INSERT INTO published_ids (adapter, event_id, first_seen, last_seen)
VALUES (?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
ON CONFLICT (adapter, event_id) DO UPDATE SET
last_seen = CURRENT_TIMESTAMP
""",
(self.name, stable_id),
)
self._db.commit()
def sweep_old_ids(self) -> int:
"""Remove published_ids older than 48 hours. Returns count deleted."""
if not self._db:
return 0
cur = self._db.execute(
"DELETE FROM published_ids WHERE adapter = ? AND last_seen < datetime('now', '-48 hours')",
(self.name,),
)
self._db.commit()
count = cur.rowcount
if count > 0:
logger.info("FIRMS swept old dedup entries", extra={"count": count})
return count
def _build_stable_id(
self, satellite: str, acq_date: str, acq_time: str, lat: float, lon: float
) -> str:
"""Build stable ID for deduplication."""
# Round lat/lon to 0.001 degrees to handle floating-point comparison
lat_rounded = round(lat, 3)
lon_rounded = round(lon, 3)
return f"{satellite}:{acq_date}:{acq_time}:{lat_rounded}:{lon_rounded}"
def _build_url(self, satellite: str) -> str | None:
"""Build FIRMS API URL for a satellite."""
if not self._api_key or not self.region:
return None
# Area format: west,south,east,north
area = f"{self.region.west},{self.region.south},{self.region.east},{self.region.north}"
return f"{FIRMS_API_BASE}/{self._api_key}/{satellite}/{area}/1"
@retry(
stop=stop_after_attempt(3),
wait=wait_exponential_jitter(initial=2, max=30),
retry=retry_if_exception_type((aiohttp.ClientError,)),
reraise=True,
)
async def _fetch_csv(self, url: str) -> str:
"""Fetch CSV data from FIRMS API."""
if not self._session:
raise RuntimeError("Session not initialized")
async with self._session.get(url) as resp:
# Check for error responses
content_type = resp.headers.get("Content-Type", "")
if "text/html" in content_type:
text = await resp.text()
logger.error(
"FIRMS returned HTML (likely auth error)",
extra={"status": resp.status, "preview": text[:200]},
)
raise ValueError("FIRMS returned HTML instead of CSV")
resp.raise_for_status()
return await resp.text()
def _parse_csv(self, csv_text: str, satellite: str) -> list[dict[str, Any]]:
"""Parse FIRMS CSV response into list of dicts."""
rows = []
reader = csv.DictReader(StringIO(csv_text))
for row in reader:
try:
# Parse required fields
lat = float(row["latitude"])
lon = float(row["longitude"])
acq_date = row["acq_date"]
acq_time = row["acq_time"]
confidence_raw = row.get("confidence", "n").lower()
confidence = CONFIDENCE_MAP.get(confidence_raw, "nominal")
rows.append({
"latitude": lat,
"longitude": lon,
"bright_ti4": float(row.get("bright_ti4", 0)) if row.get("bright_ti4") else None,
"bright_ti5": float(row.get("bright_ti5", 0)) if row.get("bright_ti5") else None,
"scan": float(row.get("scan", 0)) if row.get("scan") else None,
"track": float(row.get("track", 0)) if row.get("track") else None,
"acq_date": acq_date,
"acq_time": acq_time,
"satellite": row.get("satellite", satellite),
"instrument": row.get("instrument", "VIIRS"),
"confidence": confidence,
"confidence_raw": confidence_raw,
"version": row.get("version", ""),
"frp": float(row.get("frp", 0)) if row.get("frp") else None,
"daynight": row.get("daynight", ""),
})
except (KeyError, ValueError) as e:
logger.warning(
"Failed to parse FIRMS row",
extra={"error": str(e), "row": dict(row)},
)
continue
return rows
def _row_to_event(self, row: dict[str, Any], satellite: str) -> Event:
"""Convert a parsed CSV row to an Event."""
satellite_short = SATELLITE_SHORT.get(satellite, satellite.lower().replace("_nrt", ""))
confidence = row["confidence"]
severity = SEVERITY_MAP.get(confidence, 1)
# Parse acquisition time
acq_date = row["acq_date"]
acq_time = row["acq_time"]
# acq_time is HHMM format
try:
time = datetime.strptime(
f"{acq_date} {acq_time}", "%Y-%m-%d %H%M"
).replace(tzinfo=timezone.utc)
except ValueError:
time = datetime.now(timezone.utc)
lat = row["latitude"]
lon = row["longitude"]
# Build stable ID
stable_id = self._build_stable_id(satellite, acq_date, acq_time, lat, lon)
geo = Geo(
centroid=(lon, lat), # GeoJSON order: lon, lat
bbox=(lon, lat, lon, lat), # Point bbox
regions=[],
primary_region=None,
)
return Event(
id=stable_id,
source="central/adapters/firms",
category=f"fire.hotspot.{satellite_short}.{confidence}",
time=time,
expires=None,
severity=severity,
geo=geo,
data=row,
)
async def poll(self) -> AsyncIterator[Event]:
"""Poll FIRMS API for fire hotspots."""
# Check API key
if not self._api_key:
# Try to fetch again in case it was added
self._api_key = await self._config_store.get_api_key(self._api_key_alias)
if not self._api_key:
logger.warning(
"FIRMS API key still not available, skipping poll",
extra={"alias": self._api_key_alias},
)
return
if not self.region:
logger.warning("FIRMS region not configured, skipping poll")
return
# Sweep old dedup entries periodically
self.sweep_old_ids()
total_features = 0
total_new = 0
for satellite in self._satellites:
url = self._build_url(satellite)
if not url:
continue
try:
csv_text = await self._fetch_csv(url)
rows = self._parse_csv(csv_text, satellite)
feature_count = len(rows)
total_features += feature_count
new_count = 0
for row in rows:
stable_id = self._build_stable_id(
satellite,
row["acq_date"],
row["acq_time"],
row["latitude"],
row["longitude"],
)
if self.is_published(stable_id):
continue
event = self._row_to_event(row, satellite)
yield event
self.mark_published(stable_id)
new_count += 1
total_new += new_count
logger.info(
"FIRMS satellite poll completed",
extra={
"satellite": satellite,
"feature_count": feature_count,
"new_count": new_count,
},
)
except Exception as e:
logger.error(
"FIRMS poll failed for satellite",
extra={"satellite": satellite, "error": str(e)},
)
continue
logger.info(
"FIRMS poll completed",
extra={
"total_features": total_features,
"total_new": total_new,
"satellites": self._satellites,
},
)
def subject_for_fire_hotspot(ev: Event) -> str:
"""Compute the NATS subject for a fire hotspot event.
Subject format: central.fire.hotspot.<satellite>.<confidence>
The category already contains the satellite and confidence info,
so we just prefix with 'central.'.
"""
# category is "fire.hotspot.<satellite>.<confidence>"
return f"central.{ev.category}"

800
src/central/adapters/usgs_quake.py
View file

@ -1,400 +1,400 @@
"""USGS Earthquake Hazards Program adapter."""
import logging
import sqlite3
from collections.abc import AsyncIterator
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
import aiohttp
from shapely.geometry import Point, box as shapely_box
from tenacity import (
retry,
stop_after_attempt,
wait_exponential_jitter,
retry_if_exception_type,
)
from central.adapter import SourceAdapter
from central.config_models import AdapterConfig, RegionConfig
from central.config_store import ConfigStore
from central.models import Event, Geo
logger = logging.getLogger(__name__)
# USGS GeoJSON feed base URL
USGS_FEED_BASE = "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary"
# Valid feed options
VALID_FEEDS = {"all_hour", "all_day", "all_week", "all_month"}
def magnitude_tier(mag: float) -> str:
"""Classify magnitude into USGS-style tier."""
if mag < 3.0:
return "minor"
if mag < 4.0:
return "light"
if mag < 5.0:
return "moderate"
if mag < 6.0:
return "strong"
if mag < 7.0:
return "major"
return "great"
def magnitude_to_severity(mag: float) -> int:
"""Map magnitude to severity level (0-5)."""
if mag < 3.0:
return 0
if mag < 4.0:
return 1
if mag < 5.0:
return 2
if mag < 6.0:
return 3
if mag < 7.0:
return 4
return 5
class USGSQuakeAdapter(SourceAdapter):
"""USGS Earthquake Hazards Program adapter."""
name = "usgs_quake"
def __init__(
self,
config: AdapterConfig,
config_store: ConfigStore, # Unused, accepted for signature uniformity
cursor_db_path: Path,
) -> None:
self._cursor_db_path = cursor_db_path
self._session: aiohttp.ClientSession | None = None
self._db: sqlite3.Connection | None = None
# Extract settings from config
self._feed: str = config.settings.get("feed", "all_hour")
if self._feed not in VALID_FEEDS:
logger.warning(
"Invalid feed setting, using all_hour",
extra={"feed": self._feed, "valid": list(VALID_FEEDS)},
)
self._feed = "all_hour"
# Parse region from settings
region_dict = config.settings.get("region")
if region_dict:
self.region: RegionConfig | None = RegionConfig(**region_dict)
self._region_box = shapely_box(
self.region.west,
self.region.south,
self.region.east,
self.region.north,
)
else:
self.region = None
self._region_box = None
async def apply_config(self, new_config: AdapterConfig) -> None:
"""Apply new configuration from hot-reload."""
# Update feed
new_feed = new_config.settings.get("feed", "all_hour")
if new_feed in VALID_FEEDS:
self._feed = new_feed
else:
logger.warning(
"Invalid feed in new config, keeping current",
extra={"new_feed": new_feed, "current": self._feed},
)
# Update region
region_dict = new_config.settings.get("region")
if region_dict:
self.region = RegionConfig(**region_dict)
self._region_box = shapely_box(
self.region.west,
self.region.south,
self.region.east,
self.region.north,
)
else:
self.region = None
self._region_box = None
logger.info(
"USGS quake config applied",
extra={
"region": region_dict,
"feed": self._feed,
},
)
async def startup(self) -> None:
"""Initialize HTTP session and dedup tracker."""
# Initialize HTTP session
self._session = aiohttp.ClientSession(
headers={"User-Agent": "Central/1.0 (earthquake monitoring)"},
timeout=aiohttp.ClientTimeout(total=30),
)
# Initialize dedup tracker (shared sqlite DB)
self._db = sqlite3.connect(str(self._cursor_db_path))
self._db.execute("""
CREATE TABLE IF NOT EXISTS published_ids (
adapter TEXT NOT NULL,
event_id TEXT NOT NULL,
first_seen TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_seen TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (adapter, event_id)
)
""")
self._db.execute("""
CREATE INDEX IF NOT EXISTS published_ids_last_seen
ON published_ids (last_seen)
""")
self._db.commit()
# Sweep old entries on startup (7 days for quakes)
self.sweep_old_ids()
logger.info(
"USGS quake adapter started",
extra={
"region": {
"north": self.region.north,
"south": self.region.south,
"east": self.region.east,
"west": self.region.west,
} if self.region else None,
"feed": self._feed,
},
)
async def shutdown(self) -> None:
"""Close HTTP session and database."""
if self._session:
await self._session.close()
self._session = None
if self._db:
self._db.close()
self._db = None
logger.info("USGS quake adapter shut down")
def is_published(self, event_id: str) -> bool:
"""Check if an event has already been published."""
if not self._db:
return False
cur = self._db.execute(
"SELECT 1 FROM published_ids WHERE adapter = ? AND event_id = ?",
(self.name, event_id),
)
return cur.fetchone() is not None
def mark_published(self, event_id: str) -> None:
"""Mark an event as published."""
if not self._db:
return
self._db.execute(
"""
INSERT INTO published_ids (adapter, event_id, first_seen, last_seen)
VALUES (?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
ON CONFLICT (adapter, event_id) DO UPDATE SET
last_seen = CURRENT_TIMESTAMP
""",
(self.name, event_id),
)
self._db.commit()
def sweep_old_ids(self) -> int:
"""Remove published_ids older than 7 days. Returns count deleted."""
if not self._db:
return 0
cur = self._db.execute(
"DELETE FROM published_ids WHERE adapter = ? AND last_seen < datetime('now', '-7 days')",
(self.name,),
)
self._db.commit()
count = cur.rowcount
if count > 0:
logger.info("USGS quake swept old dedup entries", extra={"count": count})
return count
def _build_url(self) -> str:
"""Build USGS GeoJSON feed URL."""
return f"{USGS_FEED_BASE}/{self._feed}.geojson"
@retry(
stop=stop_after_attempt(3),
wait=wait_exponential_jitter(initial=1, max=15),
retry=retry_if_exception_type((aiohttp.ClientError,)),
reraise=True,
)
async def _fetch_geojson(self) -> dict[str, Any]:
"""Fetch GeoJSON data from USGS."""
if not self._session:
raise RuntimeError("Session not initialized")
url = self._build_url()
async with self._session.get(url) as resp:
resp.raise_for_status()
return await resp.json()
def _point_in_region(self, lon: float, lat: float) -> bool:
"""Check if point intersects region bbox using shapely."""
if self._region_box is None:
return True
point = Point(lon, lat)
return self._region_box.intersects(point)
def _feature_to_event(self, feature: dict[str, Any]) -> Event | None:
"""Convert a GeoJSON feature to an Event."""
props = feature.get("properties", {})
geometry = feature.get("geometry", {})
coords = geometry.get("coordinates", [])
# Validate required fields
event_id = feature.get("id")
if not event_id:
logger.warning("Feature missing id", extra={"properties": props})
return None
# Get magnitude - skip if null/missing (PM decision)
mag = props.get("mag")
if mag is None:
logger.debug(
"Skipping event with null magnitude",
extra={"id": event_id, "place": props.get("place")},
)
return None
try:
mag = float(mag)
except (TypeError, ValueError):
logger.warning(
"Invalid magnitude value",
extra={"id": event_id, "mag": mag},
)
return None
# Get coordinates [lon, lat, depth]
if len(coords) < 2:
logger.warning("Feature missing coordinates", extra={"id": event_id})
return None
lon, lat = coords[0], coords[1]
depth = coords[2] if len(coords) > 2 else None
# Region filter
if not self._point_in_region(lon, lat):
return None
# Parse event time (milliseconds since epoch)
time_ms = props.get("time")
if time_ms is not None:
try:
event_time = datetime.fromtimestamp(time_ms / 1000, tz=timezone.utc)
except (TypeError, ValueError, OSError):
event_time = datetime.now(timezone.utc)
else:
event_time = datetime.now(timezone.utc)
# Build tier and severity
tier = magnitude_tier(mag)
severity = magnitude_to_severity(mag)
# Build geo
geo = Geo(
centroid=(lon, lat),
bbox=(lon, lat, lon, lat),
regions=[],
primary_region=None,
)
# Build data payload
data = {
"magnitude": mag,
"place": props.get("place"),
"time_ms": time_ms,
"updated_ms": props.get("updated"),
"tz": props.get("tz"),
"url": props.get("url"),
"detail": props.get("detail"),
"felt": props.get("felt"),
"cdi": props.get("cdi"),
"mmi": props.get("mmi"),
"alert": props.get("alert"),
"status": props.get("status"),
"tsunami": props.get("tsunami"),
"sig": props.get("sig"),
"net": props.get("net"),
"code": props.get("code"),
"ids": props.get("ids"),
"sources": props.get("sources"),
"types": props.get("types"),
"nst": props.get("nst"),
"dmin": props.get("dmin"),
"rms": props.get("rms"),
"gap": props.get("gap"),
"magType": props.get("magType"),
"type": props.get("type"),
"title": props.get("title"),
"longitude": lon,
"latitude": lat,
"depth": depth,
}
return Event(
id=event_id,
source="central/adapters/usgs_quake",
category=f"quake.event.{tier}",
time=event_time,
expires=None,
severity=severity,
geo=geo,
data=data,
)
async def poll(self) -> AsyncIterator[Event]:
"""Poll USGS for earthquake data."""
if not self.region:
logger.warning("USGS quake region not configured, skipping poll")
return
# Sweep old dedup entries periodically
self.sweep_old_ids()
try:
data = await self._fetch_geojson()
except Exception as e:
logger.error("Failed to fetch USGS data", extra={"error": str(e)})
raise
features = data.get("features", [])
metadata = data.get("metadata", {})
logger.info(
"USGS quake poll completed",
extra={
"feature_count": len(features),
"title": metadata.get("title"),
"generated": metadata.get("generated"),
},
)
new_count = 0
for feature in features:
event = self._feature_to_event(feature)
if event is None:
continue
if self.is_published(event.id):
continue
yield event
self.mark_published(event.id)
new_count += 1
logger.info("USGS quake yielded events", extra={"count": new_count})
"""USGS Earthquake Hazards Program adapter."""
import logging
import sqlite3
from collections.abc import AsyncIterator
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
import aiohttp
from shapely.geometry import Point, box as shapely_box
from tenacity import (
retry,
stop_after_attempt,
wait_exponential_jitter,
retry_if_exception_type,
)
from central.adapter import SourceAdapter
from central.config_models import AdapterConfig, RegionConfig
from central.config_store import ConfigStore
from central.models import Event, Geo
logger = logging.getLogger(__name__)
# USGS GeoJSON feed base URL
USGS_FEED_BASE = "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary"
# Valid feed options
VALID_FEEDS = {"all_hour", "all_day", "all_week", "all_month"}
def magnitude_tier(mag: float) -> str:
"""Classify magnitude into USGS-style tier."""
if mag < 3.0:
return "minor"
if mag < 4.0:
return "light"
if mag < 5.0:
return "moderate"
if mag < 6.0:
return "strong"
if mag < 7.0:
return "major"
return "great"
def magnitude_to_severity(mag: float) -> int:
"""Map magnitude to severity level (0-5)."""
if mag < 3.0:
return 0
if mag < 4.0:
return 1
if mag < 5.0:
return 2
if mag < 6.0:
return 3
if mag < 7.0:
return 4
return 5
class USGSQuakeAdapter(SourceAdapter):
"""USGS Earthquake Hazards Program adapter."""
name = "usgs_quake"
def __init__(
self,
config: AdapterConfig,
config_store: ConfigStore, # Unused, accepted for signature uniformity
cursor_db_path: Path,
) -> None:
self._cursor_db_path = cursor_db_path
self._session: aiohttp.ClientSession | None = None
self._db: sqlite3.Connection | None = None
# Extract settings from config
self._feed: str = config.settings.get("feed", "all_hour")
if self._feed not in VALID_FEEDS:
logger.warning(
"Invalid feed setting, using all_hour",
extra={"feed": self._feed, "valid": list(VALID_FEEDS)},
)
self._feed = "all_hour"
# Parse region from settings
region_dict = config.settings.get("region")
if region_dict:
self.region: RegionConfig | None = RegionConfig(**region_dict)
self._region_box = shapely_box(
self.region.west,
self.region.south,
self.region.east,
self.region.north,
)
else:
self.region = None
self._region_box = None
async def apply_config(self, new_config: AdapterConfig) -> None:
"""Apply new configuration from hot-reload."""
# Update feed
new_feed = new_config.settings.get("feed", "all_hour")
if new_feed in VALID_FEEDS:
self._feed = new_feed
else:
logger.warning(
"Invalid feed in new config, keeping current",
extra={"new_feed": new_feed, "current": self._feed},
)
# Update region
region_dict = new_config.settings.get("region")
if region_dict:
self.region = RegionConfig(**region_dict)
self._region_box = shapely_box(
self.region.west,
self.region.south,
self.region.east,
self.region.north,
)
else:
self.region = None
self._region_box = None
logger.info(
"USGS quake config applied",
extra={
"region": region_dict,
"feed": self._feed,
},
)
async def startup(self) -> None:
"""Initialize HTTP session and dedup tracker."""
# Initialize HTTP session
self._session = aiohttp.ClientSession(
headers={"User-Agent": "Central/1.0 (earthquake monitoring)"},
timeout=aiohttp.ClientTimeout(total=30),
)
# Initialize dedup tracker (shared sqlite DB)
self._db = sqlite3.connect(str(self._cursor_db_path))
self._db.execute("""
CREATE TABLE IF NOT EXISTS published_ids (
adapter TEXT NOT NULL,
event_id TEXT NOT NULL,
first_seen TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_seen TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (adapter, event_id)
)
""")
self._db.execute("""
CREATE INDEX IF NOT EXISTS published_ids_last_seen
ON published_ids (last_seen)
""")
self._db.commit()
# Sweep old entries on startup (7 days for quakes)
self.sweep_old_ids()
logger.info(
"USGS quake adapter started",
extra={
"region": {
"north": self.region.north,
"south": self.region.south,
"east": self.region.east,
"west": self.region.west,
} if self.region else None,
"feed": self._feed,
},
)
async def shutdown(self) -> None:
"""Close HTTP session and database."""
if self._session:
await self._session.close()
self._session = None
if self._db:
self._db.close()
self._db = None
logger.info("USGS quake adapter shut down")
def is_published(self, event_id: str) -> bool:
"""Check if an event has already been published."""
if not self._db:
return False
cur = self._db.execute(
"SELECT 1 FROM published_ids WHERE adapter = ? AND event_id = ?",
(self.name, event_id),
)
return cur.fetchone() is not None
def mark_published(self, event_id: str) -> None:
"""Mark an event as published."""
if not self._db:
return
self._db.execute(
"""
INSERT INTO published_ids (adapter, event_id, first_seen, last_seen)
VALUES (?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
ON CONFLICT (adapter, event_id) DO UPDATE SET
last_seen = CURRENT_TIMESTAMP
""",
(self.name, event_id),
)
self._db.commit()
def sweep_old_ids(self) -> int:
"""Remove published_ids older than 7 days. Returns count deleted."""
if not self._db:
return 0
cur = self._db.execute(
"DELETE FROM published_ids WHERE adapter = ? AND last_seen < datetime('now', '-7 days')",
(self.name,),
)
self._db.commit()
count = cur.rowcount
if count > 0:
logger.info("USGS quake swept old dedup entries", extra={"count": count})
return count
def _build_url(self) -> str:
"""Build USGS GeoJSON feed URL."""
return f"{USGS_FEED_BASE}/{self._feed}.geojson"
@retry(
stop=stop_after_attempt(3),
wait=wait_exponential_jitter(initial=1, max=15),
retry=retry_if_exception_type((aiohttp.ClientError,)),
reraise=True,
)
async def _fetch_geojson(self) -> dict[str, Any]:
"""Fetch GeoJSON data from USGS."""
if not self._session:
raise RuntimeError("Session not initialized")
url = self._build_url()
async with self._session.get(url) as resp:
resp.raise_for_status()
return await resp.json()
def _point_in_region(self, lon: float, lat: float) -> bool:
"""Check if point intersects region bbox using shapely."""
if self._region_box is None:
return True
point = Point(lon, lat)
return self._region_box.intersects(point)
def _feature_to_event(self, feature: dict[str, Any]) -> Event | None:
"""Convert a GeoJSON feature to an Event."""
props = feature.get("properties", {})
geometry = feature.get("geometry", {})
coords = geometry.get("coordinates", [])
# Validate required fields
event_id = feature.get("id")
if not event_id:
logger.warning("Feature missing id", extra={"properties": props})
return None
# Get magnitude - skip if null/missing (PM decision)
mag = props.get("mag")
if mag is None:
logger.debug(
"Skipping event with null magnitude",
extra={"id": event_id, "place": props.get("place")},
)
return None
try:
mag = float(mag)
except (TypeError, ValueError):
logger.warning(
"Invalid magnitude value",
extra={"id": event_id, "mag": mag},
)
return None
# Get coordinates [lon, lat, depth]
if len(coords) < 2:
logger.warning("Feature missing coordinates", extra={"id": event_id})
return None
lon, lat = coords[0], coords[1]
depth = coords[2] if len(coords) > 2 else None
# Region filter
if not self._point_in_region(lon, lat):
return None
# Parse event time (milliseconds since epoch)
time_ms = props.get("time")
if time_ms is not None:
try:
event_time = datetime.fromtimestamp(time_ms / 1000, tz=timezone.utc)
except (TypeError, ValueError, OSError):
event_time = datetime.now(timezone.utc)
else:
event_time = datetime.now(timezone.utc)
# Build tier and severity
tier = magnitude_tier(mag)
severity = magnitude_to_severity(mag)
# Build geo
geo = Geo(
centroid=(lon, lat),
bbox=(lon, lat, lon, lat),
regions=[],
primary_region=None,
)
# Build data payload
data = {
"magnitude": mag,
"place": props.get("place"),
"time_ms": time_ms,
"updated_ms": props.get("updated"),
"tz": props.get("tz"),
"url": props.get("url"),
"detail": props.get("detail"),
"felt": props.get("felt"),
"cdi": props.get("cdi"),
"mmi": props.get("mmi"),
"alert": props.get("alert"),
"status": props.get("status"),
"tsunami": props.get("tsunami"),
"sig": props.get("sig"),
"net": props.get("net"),
"code": props.get("code"),
"ids": props.get("ids"),
"sources": props.get("sources"),
"types": props.get("types"),
"nst": props.get("nst"),
"dmin": props.get("dmin"),
"rms": props.get("rms"),
"gap": props.get("gap"),
"magType": props.get("magType"),
"type": props.get("type"),
"title": props.get("title"),
"longitude": lon,
"latitude": lat,
"depth": depth,
}
return Event(
id=event_id,
source="central/adapters/usgs_quake",
category=f"quake.event.{tier}",
time=event_time,
expires=None,
severity=severity,
geo=geo,
data=data,
)
async def poll(self) -> AsyncIterator[Event]:
"""Poll USGS for earthquake data."""
if not self.region:
logger.warning("USGS quake region not configured, skipping poll")
return
# Sweep old dedup entries periodically
self.sweep_old_ids()
try:
data = await self._fetch_geojson()
except Exception as e:
logger.error("Failed to fetch USGS data", extra={"error": str(e)})
raise
features = data.get("features", [])
metadata = data.get("metadata", {})
logger.info(
"USGS quake poll completed",
extra={
"feature_count": len(features),
"title": metadata.get("title"),
"generated": metadata.get("generated"),
},
)
new_count = 0
for feature in features:
event = self._feature_to_event(feature)
if event is None:
continue
if self.is_published(event.id):
continue
yield event
self.mark_published(event.id)
new_count += 1
logger.info("USGS quake yielded events", extra={"count": new_count})

706
src/central/archive.py
View file

@ -1,353 +1,353 @@
"""Central archive consumer - JetStream to TimescaleDB."""
import asyncio
import json
import logging
import signal
import sys
from datetime import datetime, timezone
from typing import Any
import asyncpg
import nats
from nats.js import JetStreamContext
from nats.js.api import ConsumerConfig, DeliverPolicy, AckPolicy
from central.bootstrap_config import get_settings
CONSUMER_NAME = "archive"
STREAM_NAME = "CENTRAL_WX"
SUBJECT_FILTER = "central.wx.>"
BATCH_SIZE = 100
FETCH_TIMEOUT = 5.0
ACK_WAIT = 30
class JsonFormatter(logging.Formatter):
"""JSON log formatter for structured logging."""
def format(self, record: logging.LogRecord) -> str:
log_obj: dict[str, Any] = {
"ts": datetime.now(timezone.utc).isoformat(),
"level": record.levelname,
"logger": record.name,
"msg": record.getMessage(),
}
if record.exc_info:
log_obj["exc"] = self.formatException(record.exc_info)
for key in record.__dict__:
if key not in (
"name", "msg", "args", "created", "filename", "funcName",
"levelname", "levelno", "lineno", "module", "msecs",
"pathname", "process", "processName", "relativeCreated",
"stack_info", "exc_info", "exc_text", "thread", "threadName",
"taskName", "message",
):
log_obj[key] = record.__dict__[key]
return json.dumps(log_obj)
def setup_logging() -> None:
"""Configure JSON logging to stdout."""
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(JsonFormatter())
logging.root.handlers = [handler]
logging.root.setLevel(logging.INFO)
logger = logging.getLogger("central.archive")
def _build_geom_sql(geo_data: dict[str, Any] | None) -> str | None:
"""Build PostGIS geometry from event geo data."""
if not geo_data:
return None
bbox = geo_data.get("bbox")
centroid = geo_data.get("centroid")
if bbox and len(bbox) == 4:
# Create polygon from bbox
min_lon, min_lat, max_lon, max_lat = bbox
return json.dumps({
"type": "Polygon",
"coordinates": [[
[min_lon, min_lat],
[max_lon, min_lat],
[max_lon, max_lat],
[min_lon, max_lat],
[min_lon, min_lat],
]]
})
elif centroid and len(centroid) == 2:
# Create point from centroid
return json.dumps({
"type": "Point",
"coordinates": centroid
})
return None
class ArchiveConsumer:
"""Archive consumer process."""
def __init__(self, nats_url: str, postgres_dsn: str) -> None:
self._nats_url = nats_url
self._postgres_dsn = postgres_dsn
self._nc: nats.NATS | None = None
self._js: JetStreamContext | None = None
self._pool: asyncpg.Pool | None = None
self._shutdown_event = asyncio.Event()
async def connect(self) -> None:
"""Connect to NATS and PostgreSQL."""
self._nc = await nats.connect(self._nats_url)
self._js = self._nc.jetstream()
logger.info("Connected to NATS", extra={"url": self._nats_url})
self._pool = await asyncpg.create_pool(
self._postgres_dsn,
min_size=1,
max_size=5,
)
logger.info("Connected to PostgreSQL")
async def disconnect(self) -> None:
"""Disconnect from NATS and PostgreSQL."""
if self._pool:
await self._pool.close()
self._pool = None
if self._nc:
await self._nc.drain()
await self._nc.close()
self._nc = None
self._js = None
logger.info("Disconnected")
async def _ensure_consumer(self) -> None:
"""Ensure the durable consumer exists."""
if not self._js:
return
try:
await self._js.consumer_info(STREAM_NAME, CONSUMER_NAME)
logger.info("Consumer exists", extra={"consumer": CONSUMER_NAME})
except nats.js.errors.NotFoundError:
consumer_config = ConsumerConfig(
durable_name=CONSUMER_NAME,
deliver_policy=DeliverPolicy.ALL,
ack_policy=AckPolicy.EXPLICIT,
ack_wait=ACK_WAIT,
filter_subject=SUBJECT_FILTER,
)
await self._js.add_consumer(STREAM_NAME, consumer_config)
logger.info("Consumer created", extra={"consumer": CONSUMER_NAME})
async def _process_message(self, msg: Any, conn: asyncpg.Connection) -> None:
"""Process a single message and insert into database."""
try:
envelope = json.loads(msg.data.decode())
except json.JSONDecodeError as e:
logger.warning("Invalid JSON in message", extra={"error": str(e)})
await msg.ack()
return
event_data = envelope.get("data", {})
geo_data = event_data.get("geo")
event_id = envelope.get("id")
source = event_data.get("source", "")
category = event_data.get("category", "")
time_str = event_data.get("time")
expires_str = event_data.get("expires")
severity = event_data.get("severity")
regions = event_data.get("geo", {}).get("regions", [])
primary_region = event_data.get("geo", {}).get("primary_region")
# Parse timestamps
event_time = None
if time_str:
try:
event_time = datetime.fromisoformat(time_str.replace("Z", "+00:00"))
except (ValueError, TypeError):
pass
expires_time = None
if expires_str:
try:
expires_time = datetime.fromisoformat(expires_str.replace("Z", "+00:00"))
except (ValueError, TypeError):
pass
if not event_id or not event_time:
logger.warning(
"Message missing required fields",
extra={"id": event_id, "time": time_str}
)
await msg.ack()
return
geom_json = _build_geom_sql(geo_data)
try:
if geom_json:
await conn.execute(
"""
INSERT INTO events (id, source, category, time, expires, severity,
geom, regions, primary_region, payload)
VALUES ($1, $2, $3, $4, $5, $6,
ST_GeomFromGeoJSON($7), $8, $9, $10)
ON CONFLICT (id, time) DO UPDATE SET
source = EXCLUDED.source,
category = EXCLUDED.category,
expires = EXCLUDED.expires,
severity = EXCLUDED.severity,
geom = EXCLUDED.geom,
regions = EXCLUDED.regions,
primary_region = EXCLUDED.primary_region,
payload = EXCLUDED.payload
""",
event_id, source, category, event_time, expires_time, severity,
geom_json, regions, primary_region, json.dumps(envelope)
)
else:
await conn.execute(
"""
INSERT INTO events (id, source, category, time, expires, severity,
geom, regions, primary_region, payload)
VALUES ($1, $2, $3, $4, $5, $6, NULL, $7, $8, $9)
ON CONFLICT (id, time) DO UPDATE SET
source = EXCLUDED.source,
category = EXCLUDED.category,
expires = EXCLUDED.expires,
severity = EXCLUDED.severity,
geom = EXCLUDED.geom,
regions = EXCLUDED.regions,
primary_region = EXCLUDED.primary_region,
payload = EXCLUDED.payload
""",
event_id, source, category, event_time, expires_time, severity,
regions, primary_region, json.dumps(envelope)
)
await msg.ack()
logger.info("Archived event", extra={"id": event_id, "category": category})
except Exception as e:
logger.error(
"Failed to insert event",
extra={"id": event_id, "error": str(e)}
)
# Don't ack - let it be redelivered
async def _consume_loop(self) -> None:
"""Main consume loop."""
if not self._js or not self._pool:
return
await self._ensure_consumer()
sub = await self._js.pull_subscribe(
SUBJECT_FILTER,
durable=CONSUMER_NAME,
stream=STREAM_NAME,
)
logger.info(
"Subscribed to stream",
extra={"stream": STREAM_NAME, "filter": SUBJECT_FILTER}
)
while not self._shutdown_event.is_set():
try:
msgs = await sub.fetch(
batch=BATCH_SIZE,
timeout=FETCH_TIMEOUT,
)
if msgs:
async with self._pool.acquire() as conn:
for msg in msgs:
await self._process_message(msg, conn)
except nats.errors.TimeoutError:
# No messages available, continue
pass
except asyncio.CancelledError:
break
except Exception as e:
logger.exception("Error in consume loop", extra={"error": str(e)})
await asyncio.sleep(1)
logger.info("Consume loop stopped")
async def start(self) -> None:
"""Start the consumer."""
await self.connect()
logger.info("Archive consumer ready")
async def run(self) -> None:
"""Run the consume loop until shutdown."""
await self._consume_loop()
async def stop(self) -> None:
"""Stop the consumer gracefully."""
logger.info("Archive consumer shutting down")
self._shutdown_event.set()
await self.disconnect()
logger.info("Archive consumer stopped")
async def async_main() -> None:
"""Async entry point."""
setup_logging()
settings = get_settings()
logger.info(
"Archive starting",
extra={
"nats_url": settings.nats_url,
},
)
consumer = ArchiveConsumer(
nats_url=settings.nats_url,
postgres_dsn=settings.db_dsn,
)
loop = asyncio.get_running_loop()
shutdown_event = asyncio.Event()
def handle_signal() -> None:
shutdown_event.set()
for sig in (signal.SIGTERM, signal.SIGINT):
loop.add_signal_handler(sig, handle_signal)
await consumer.start()
# Run consumer in background
consume_task = asyncio.create_task(consumer.run())
# Wait for shutdown signal
await shutdown_event.wait()
consumer._shutdown_event.set()
consume_task.cancel()
try:
await consume_task
except asyncio.CancelledError:
pass
await consumer.stop()
def main() -> None:
"""Entry point."""
asyncio.run(async_main())
if __name__ == "__main__":
main()
"""Central archive consumer - JetStream to TimescaleDB."""
import asyncio
import json
import logging
import signal
import sys
from datetime import datetime, timezone
from typing import Any
import asyncpg
import nats
from nats.js import JetStreamContext
from nats.js.api import ConsumerConfig, DeliverPolicy, AckPolicy
from central.bootstrap_config import get_settings
CONSUMER_NAME = "archive"
STREAM_NAME = "CENTRAL_WX"
SUBJECT_FILTER = "central.wx.>"
BATCH_SIZE = 100
FETCH_TIMEOUT = 5.0
ACK_WAIT = 30
class JsonFormatter(logging.Formatter):
"""JSON log formatter for structured logging."""
def format(self, record: logging.LogRecord) -> str:
log_obj: dict[str, Any] = {
"ts": datetime.now(timezone.utc).isoformat(),
"level": record.levelname,
"logger": record.name,
"msg": record.getMessage(),
}
if record.exc_info:
log_obj["exc"] = self.formatException(record.exc_info)
for key in record.__dict__:
if key not in (
"name", "msg", "args", "created", "filename", "funcName",
"levelname", "levelno", "lineno", "module", "msecs",
"pathname", "process", "processName", "relativeCreated",
"stack_info", "exc_info", "exc_text", "thread", "threadName",
"taskName", "message",
):
log_obj[key] = record.__dict__[key]
return json.dumps(log_obj)
def setup_logging() -> None:
"""Configure JSON logging to stdout."""
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(JsonFormatter())
logging.root.handlers = [handler]
logging.root.setLevel(logging.INFO)
logger = logging.getLogger("central.archive")
def _build_geom_sql(geo_data: dict[str, Any] | None) -> str | None:
"""Build PostGIS geometry from event geo data."""
if not geo_data:
return None
bbox = geo_data.get("bbox")
centroid = geo_data.get("centroid")
if bbox and len(bbox) == 4:
# Create polygon from bbox
min_lon, min_lat, max_lon, max_lat = bbox
return json.dumps({
"type": "Polygon",
"coordinates": [[
[min_lon, min_lat],
[max_lon, min_lat],
[max_lon, max_lat],
[min_lon, max_lat],
[min_lon, min_lat],
]]
})
elif centroid and len(centroid) == 2:
# Create point from centroid
return json.dumps({
"type": "Point",
"coordinates": centroid
})
return None
class ArchiveConsumer:
"""Archive consumer process."""
def __init__(self, nats_url: str, postgres_dsn: str) -> None:
self._nats_url = nats_url
self._postgres_dsn = postgres_dsn
self._nc: nats.NATS | None = None
self._js: JetStreamContext | None = None
self._pool: asyncpg.Pool | None = None
self._shutdown_event = asyncio.Event()
async def connect(self) -> None:
"""Connect to NATS and PostgreSQL."""
self._nc = await nats.connect(self._nats_url)
self._js = self._nc.jetstream()
logger.info("Connected to NATS", extra={"url": self._nats_url})
self._pool = await asyncpg.create_pool(
self._postgres_dsn,
min_size=1,
max_size=5,
)
logger.info("Connected to PostgreSQL")
async def disconnect(self) -> None:
"""Disconnect from NATS and PostgreSQL."""
if self._pool:
await self._pool.close()
self._pool = None
if self._nc:
await self._nc.drain()
await self._nc.close()
self._nc = None
self._js = None
logger.info("Disconnected")
async def _ensure_consumer(self) -> None:
"""Ensure the durable consumer exists."""
if not self._js:
return
try:
await self._js.consumer_info(STREAM_NAME, CONSUMER_NAME)
logger.info("Consumer exists", extra={"consumer": CONSUMER_NAME})
except nats.js.errors.NotFoundError:
consumer_config = ConsumerConfig(
durable_name=CONSUMER_NAME,
deliver_policy=DeliverPolicy.ALL,
ack_policy=AckPolicy.EXPLICIT,
ack_wait=ACK_WAIT,
filter_subject=SUBJECT_FILTER,
)
await self._js.add_consumer(STREAM_NAME, consumer_config)
logger.info("Consumer created", extra={"consumer": CONSUMER_NAME})
async def _process_message(self, msg: Any, conn: asyncpg.Connection) -> None:
"""Process a single message and insert into database."""
try:
envelope = json.loads(msg.data.decode())
except json.JSONDecodeError as e:
logger.warning("Invalid JSON in message", extra={"error": str(e)})
await msg.ack()
return
event_data = envelope.get("data", {})
geo_data = event_data.get("geo")
event_id = envelope.get("id")
source = event_data.get("source", "")
category = event_data.get("category", "")
time_str = event_data.get("time")
expires_str = event_data.get("expires")
severity = event_data.get("severity")
regions = event_data.get("geo", {}).get("regions", [])
primary_region = event_data.get("geo", {}).get("primary_region")
# Parse timestamps
event_time = None
if time_str:
try:
event_time = datetime.fromisoformat(time_str.replace("Z", "+00:00"))
except (ValueError, TypeError):
pass
expires_time = None
if expires_str:
try:
expires_time = datetime.fromisoformat(expires_str.replace("Z", "+00:00"))
except (ValueError, TypeError):
pass
if not event_id or not event_time:
logger.warning(
"Message missing required fields",
extra={"id": event_id, "time": time_str}
)
await msg.ack()
return
geom_json = _build_geom_sql(geo_data)
try:
if geom_json:
await conn.execute(
"""
INSERT INTO events (id, source, category, time, expires, severity,
geom, regions, primary_region, payload)
VALUES ($1, $2, $3, $4, $5, $6,
ST_GeomFromGeoJSON($7), $8, $9, $10)
ON CONFLICT (id, time) DO UPDATE SET
source = EXCLUDED.source,
category = EXCLUDED.category,
expires = EXCLUDED.expires,
severity = EXCLUDED.severity,
geom = EXCLUDED.geom,
regions = EXCLUDED.regions,
primary_region = EXCLUDED.primary_region,
payload = EXCLUDED.payload
""",
event_id, source, category, event_time, expires_time, severity,
geom_json, regions, primary_region, json.dumps(envelope)
)
else:
await conn.execute(
"""
INSERT INTO events (id, source, category, time, expires, severity,
geom, regions, primary_region, payload)
VALUES ($1, $2, $3, $4, $5, $6, NULL, $7, $8, $9)
ON CONFLICT (id, time) DO UPDATE SET
source = EXCLUDED.source,
category = EXCLUDED.category,
expires = EXCLUDED.expires,
severity = EXCLUDED.severity,
geom = EXCLUDED.geom,
regions = EXCLUDED.regions,
primary_region = EXCLUDED.primary_region,
payload = EXCLUDED.payload
""",
event_id, source, category, event_time, expires_time, severity,
regions, primary_region, json.dumps(envelope)
)
await msg.ack()
logger.info("Archived event", extra={"id": event_id, "category": category})
except Exception as e:
logger.error(
"Failed to insert event",
extra={"id": event_id, "error": str(e)}
)
# Don't ack - let it be redelivered
async def _consume_loop(self) -> None:
"""Main consume loop."""
if not self._js or not self._pool:
return
await self._ensure_consumer()
sub = await self._js.pull_subscribe(
SUBJECT_FILTER,
durable=CONSUMER_NAME,
stream=STREAM_NAME,
)
logger.info(
"Subscribed to stream",
extra={"stream": STREAM_NAME, "filter": SUBJECT_FILTER}
)
while not self._shutdown_event.is_set():
try:
msgs = await sub.fetch(
batch=BATCH_SIZE,
timeout=FETCH_TIMEOUT,
)
if msgs:
async with self._pool.acquire() as conn:
for msg in msgs:
await self._process_message(msg, conn)
except nats.errors.TimeoutError:
# No messages available, continue
pass
except asyncio.CancelledError:
break
except Exception as e:
logger.exception("Error in consume loop", extra={"error": str(e)})
await asyncio.sleep(1)
logger.info("Consume loop stopped")
async def start(self) -> None:
"""Start the consumer."""
await self.connect()
logger.info("Archive consumer ready")
async def run(self) -> None:
"""Run the consume loop until shutdown."""
await self._consume_loop()
async def stop(self) -> None:
"""Stop the consumer gracefully."""
logger.info("Archive consumer shutting down")
self._shutdown_event.set()
await self.disconnect()
logger.info("Archive consumer stopped")
async def async_main() -> None:
"""Async entry point."""
setup_logging()
settings = get_settings()
logger.info(
"Archive starting",
extra={
"nats_url": settings.nats_url,
},
)
consumer = ArchiveConsumer(
nats_url=settings.nats_url,
postgres_dsn=settings.db_dsn,
)
loop = asyncio.get_running_loop()
shutdown_event = asyncio.Event()
def handle_signal() -> None:
shutdown_event.set()
for sig in (signal.SIGTERM, signal.SIGINT):
loop.add_signal_handler(sig, handle_signal)
await consumer.start()
# Run consumer in background
consume_task = asyncio.create_task(consumer.run())
# Wait for shutdown signal
await shutdown_event.wait()
consumer._shutdown_event.set()
consume_task.cancel()
try:
await consume_task
except asyncio.CancelledError:
pass
await consumer.stop()
def main() -> None:
"""Entry point."""
asyncio.run(async_main())
if __name__ == "__main__":
main()

150
src/central/cli.py
View file

@ -1,75 +1,75 @@
"""Central CLI commands."""
import argparse
import asyncio
import sys
async def config_store_check() -> int:
"""Smoke test for config store connectivity.
Connects via bootstrap_config, lists adapters, and verifies crypto.
Returns 0 on success, 1 on failure.
"""
from central.bootstrap_config import get_settings
from central.config_store import ConfigStore
from central.crypto import decrypt, encrypt
settings = get_settings()
print(f"Connecting to: {settings.db_dsn.split('@')[1]}") # Hide password
try:
store = await ConfigStore.create(settings.db_dsn)
except Exception as e:
print(f"ERROR: Failed to connect to database: {e}")
return 1
try:
# List adapters
adapters = await store.list_adapters()
print(f"\nAdapters ({len(adapters)}):")
for adapter in adapters:
print(f" - {adapter.name}: enabled={adapter.enabled}, cadence_s={adapter.cadence_s}")
print(f" settings: {adapter.settings}")
# Test crypto
test_plaintext = b"config_store_check_test"
try:
ciphertext = encrypt(test_plaintext)
decrypted = decrypt(ciphertext)
if decrypted == test_plaintext:
print("\ncrypto: ok")
else:
print("\ncrypto: FAILED (round-trip mismatch)")
return 1
except Exception as e:
print(f"\ncrypto: FAILED ({e})")
return 1
print("\nAll checks passed.")
return 0
finally:
await store.close()
def main_config_store_check() -> None:
"""Entry point for central-cli config-store-check."""
sys.exit(asyncio.run(config_store_check()))
def main() -> None:
"""Main CLI entry point."""
parser = argparse.ArgumentParser(description="Central CLI")
subparsers = parser.add_subparsers(dest="command", required=True)
subparsers.add_parser("config-store-check", help="Test config store connectivity")
args = parser.parse_args()
if args.command == "config-store-check":
main_config_store_check()
if __name__ == "__main__":
main()
"""Central CLI commands."""
import argparse
import asyncio
import sys
async def config_store_check() -> int:
"""Smoke test for config store connectivity.
Connects via bootstrap_config, lists adapters, and verifies crypto.
Returns 0 on success, 1 on failure.
"""
from central.bootstrap_config import get_settings
from central.config_store import ConfigStore
from central.crypto import decrypt, encrypt
settings = get_settings()
print(f"Connecting to: {settings.db_dsn.split('@')[1]}") # Hide password
try:
store = await ConfigStore.create(settings.db_dsn)
except Exception as e:
print(f"ERROR: Failed to connect to database: {e}")
return 1
try:
# List adapters
adapters = await store.list_adapters()
print(f"\nAdapters ({len(adapters)}):")
for adapter in adapters:
print(f" - {adapter.name}: enabled={adapter.enabled}, cadence_s={adapter.cadence_s}")
print(f" settings: {adapter.settings}")
# Test crypto
test_plaintext = b"config_store_check_test"
try:
ciphertext = encrypt(test_plaintext)
decrypted = decrypt(ciphertext)
if decrypted == test_plaintext:
print("\ncrypto: ok")
else:
print("\ncrypto: FAILED (round-trip mismatch)")
return 1
except Exception as e:
print(f"\ncrypto: FAILED ({e})")
return 1
print("\nAll checks passed.")
return 0
finally:
await store.close()
def main_config_store_check() -> None:
"""Entry point for central-cli config-store-check."""
sys.exit(asyncio.run(config_store_check()))
def main() -> None:
"""Main CLI entry point."""
parser = argparse.ArgumentParser(description="Central CLI")
subparsers = parser.add_subparsers(dest="command", required=True)
subparsers.add_parser("config-store-check", help="Test config store connectivity")
args = parser.parse_args()
if args.command == "config-store-check":
main_config_store_check()
if __name__ == "__main__":
main()

664
src/central/config_store.py
View file

@ -1,332 +1,332 @@
"""Database-backed configuration store.
Provides async access to the config schema tables with support for
Postgres LISTEN/NOTIFY for real-time config change notifications.
"""
import asyncio
import json
import logging
from collections.abc import Awaitable, Callable
from typing import Any
import asyncpg
from central.config_models import AdapterConfig, StreamConfig
from central.crypto import decrypt, encrypt
logger = logging.getLogger(__name__)
async def _setup_json_codec(conn: asyncpg.Connection) -> None:
"""Set up JSON codec for asyncpg connection."""
await conn.set_type_codec(
"jsonb",
encoder=json.dumps,
decoder=json.loads,
schema="pg_catalog",
)
class ConfigStore:
"""Async interface to the config schema in Postgres."""
def __init__(self, pool: asyncpg.Pool) -> None:
self._pool = pool
@classmethod
async def create(cls, dsn: str, min_size: int = 1, max_size: int = 5) -> "ConfigStore":
"""Create a ConfigStore with a new connection pool."""
pool = await asyncpg.create_pool(
dsn,
min_size=min_size,
max_size=max_size,
init=_setup_json_codec,
)
return cls(pool)
async def close(self) -> None:
"""Close the connection pool."""
await self._pool.close()
# -------------------------------------------------------------------------
# Adapter configuration
# -------------------------------------------------------------------------
async def get_adapter(self, name: str) -> AdapterConfig | None:
"""Get configuration for a specific adapter."""
async with self._pool.acquire() as conn:
row = await conn.fetchrow(
"""
SELECT name, enabled, cadence_s, settings, paused_at, updated_at
FROM config.adapters
WHERE name = $1
""",
name,
)
if row is None:
return None
return AdapterConfig(**dict(row))
async def list_adapters(self) -> list[AdapterConfig]:
"""List all configured adapters."""
async with self._pool.acquire() as conn:
rows = await conn.fetch(
"""
SELECT name, enabled, cadence_s, settings, paused_at, updated_at
FROM config.adapters
ORDER BY name
"""
)
return [AdapterConfig(**dict(row)) for row in rows]
async def upsert_adapter(
self,
name: str,
enabled: bool,
cadence_s: int,
settings: dict[str, Any],
) -> None:
"""Insert or update an adapter configuration."""
async with self._pool.acquire() as conn:
await conn.execute(
"""
INSERT INTO config.adapters (name, enabled, cadence_s, settings, updated_at)
VALUES ($1, $2, $3, $4, now())
ON CONFLICT (name) DO UPDATE SET
enabled = EXCLUDED.enabled,
cadence_s = EXCLUDED.cadence_s,
settings = EXCLUDED.settings,
updated_at = now()
""",
name,
enabled,
cadence_s,
settings, # Will be encoded as JSON by the codec
)
async def pause_adapter(self, name: str) -> None:
"""Pause an adapter by setting paused_at."""
async with self._pool.acquire() as conn:
await conn.execute(
"""
UPDATE config.adapters
SET paused_at = now(), updated_at = now()
WHERE name = $1
""",
name,
)
async def unpause_adapter(self, name: str) -> None:
"""Unpause an adapter by clearing paused_at."""
async with self._pool.acquire() as conn:
await conn.execute(
"""
UPDATE config.adapters
SET paused_at = NULL, updated_at = now()
WHERE name = $1
""",
name,
)
# -------------------------------------------------------------------------
# Stream configuration
# -------------------------------------------------------------------------
async def get_stream(self, name: str) -> StreamConfig | None:
"""Get configuration for a specific stream."""
async with self._pool.acquire() as conn:
row = await conn.fetchrow(
"""
SELECT name, max_age_s, max_bytes, managed_max_bytes, updated_at
FROM config.streams
WHERE name = $1
""",
name,
)
if row is None:
return None
return StreamConfig(**dict(row))
async def list_streams(self) -> list[StreamConfig]:
"""List all configured streams."""
async with self._pool.acquire() as conn:
rows = await conn.fetch(
"""
SELECT name, max_age_s, max_bytes, managed_max_bytes, updated_at
FROM config.streams
ORDER BY name
"""
)
return [StreamConfig(**dict(row)) for row in rows]
async def upsert_stream(self, name: str, max_age_s: int) -> None:
"""Insert or update a stream's max_age_s (operator-facing)."""
async with self._pool.acquire() as conn:
await conn.execute(
"""
INSERT INTO config.streams (name, max_age_s, updated_at)
VALUES ($1, $2, now())
ON CONFLICT (name) DO UPDATE SET
max_age_s = EXCLUDED.max_age_s,
updated_at = now()
""",
name,
max_age_s,
)
async def update_stream_max_bytes(self, name: str, max_bytes: int) -> None:
"""Update a stream's max_bytes (supervisor-internal).
This update only touches max_bytes, which does NOT trigger
the column-filtered NOTIFY (only max_age_s changes fire NOTIFY).
"""
async with self._pool.acquire() as conn:
await conn.execute(
"""
UPDATE config.streams
SET max_bytes = $2, updated_at = now()
WHERE name = $1
""",
name,
max_bytes,
)
# -------------------------------------------------------------------------
# API key management
# -------------------------------------------------------------------------
async def set_api_key(self, alias: str, plaintext_value: str) -> None:
"""Store an API key, encrypting it with the master key."""
encrypted = encrypt(plaintext_value.encode("utf-8"))
async with self._pool.acquire() as conn:
await conn.execute(
"""
INSERT INTO config.api_keys (alias, encrypted_value)
VALUES ($1, $2)
ON CONFLICT (alias) DO UPDATE SET
encrypted_value = EXCLUDED.encrypted_value,
rotated_at = now()
""",
alias,
encrypted,
)
async def get_api_key(self, alias: str) -> str | None:
"""Retrieve and decrypt an API key by alias."""
async with self._pool.acquire() as conn:
row = await conn.fetchrow(
"""
SELECT encrypted_value FROM config.api_keys WHERE alias = $1
""",
alias,
)
if row is not None:
# Update last_used_at
await conn.execute(
"""
UPDATE config.api_keys SET last_used_at = now() WHERE alias = $1
""",
alias,
)
if row is None:
return None
return decrypt(row["encrypted_value"]).decode("utf-8")
async def delete_api_key(self, alias: str) -> bool:
"""Delete an API key. Returns True if key existed."""
async with self._pool.acquire() as conn:
result = await conn.execute(
"DELETE FROM config.api_keys WHERE alias = $1", alias
)
return result == "DELETE 1"
# -------------------------------------------------------------------------
# Change notifications
# -------------------------------------------------------------------------
async def listen_for_changes(
self,
callback: Callable[[str, str], Awaitable[None] | None],
) -> None:
"""Listen for config changes via Postgres NOTIFY.
Runs forever, calling callback(table, key) each time a change is
detected. The callback can be sync or async.
On connection loss, automatically reconnects with exponential backoff.
Cancellation (via task.cancel()) propagates cleanly.
Args:
callback: Function called with (table_name, row_key) on each change.
"""
backoff = 1.0
max_backoff = 30.0
while True:
conn = None
try:
conn = await self._pool.acquire()
logger.info("Config listener connected to database")
backoff = 1.0 # Reset backoff on successful connect
def notification_handler(
conn: asyncpg.Connection,
pid: int,
channel: str,
payload: str,
) -> None:
# payload format: "table_name:key"
if ":" in payload:
table, key = payload.split(":", 1)
else:
table, key = payload, ""
result = callback(table, key)
if asyncio.iscoroutine(result):
asyncio.create_task(result)
await conn.add_listener("config_changed", notification_handler)
try:
# Keep connection alive with periodic keepalive
while True:
await asyncio.sleep(60)
await conn.execute("SELECT 1")
finally:
await conn.remove_listener("config_changed", notification_handler)
except asyncio.CancelledError:
# Cancellation must propagate cleanly
logger.info("Config listener cancelled")
raise
except (
asyncpg.PostgresConnectionError,
asyncpg.InterfaceError,
ConnectionResetError,
OSError,
) as e:
logger.warning(
"Config listener connection lost, reconnecting in %.1fs: %s",
backoff,
e,
)
await asyncio.sleep(backoff)
backoff = min(backoff * 2, max_backoff)
except Exception as e:
# Unexpected error - log and retry with backoff
logger.exception(
"Config listener unexpected error, reconnecting in %.1fs",
backoff,
)
await asyncio.sleep(backoff)
backoff = min(backoff * 2, max_backoff)
finally:
if conn is not None:
try:
await self._pool.release(conn)
except Exception:
pass # Connection may already be invalid
"""Database-backed configuration store.
Provides async access to the config schema tables with support for
Postgres LISTEN/NOTIFY for real-time config change notifications.
"""
import asyncio
import json
import logging
from collections.abc import Awaitable, Callable
from typing import Any
import asyncpg
from central.config_models import AdapterConfig, StreamConfig
from central.crypto import decrypt, encrypt
logger = logging.getLogger(__name__)
async def _setup_json_codec(conn: asyncpg.Connection) -> None:
"""Set up JSON codec for asyncpg connection."""
await conn.set_type_codec(
"jsonb",
encoder=json.dumps,
decoder=json.loads,
schema="pg_catalog",
)
class ConfigStore:
"""Async interface to the config schema in Postgres."""
def __init__(self, pool: asyncpg.Pool) -> None:
self._pool = pool
@classmethod
async def create(cls, dsn: str, min_size: int = 1, max_size: int = 5) -> "ConfigStore":
"""Create a ConfigStore with a new connection pool."""
pool = await asyncpg.create_pool(
dsn,
min_size=min_size,
max_size=max_size,
init=_setup_json_codec,
)
return cls(pool)
async def close(self) -> None:
"""Close the connection pool."""
await self._pool.close()
# -------------------------------------------------------------------------
# Adapter configuration
# -------------------------------------------------------------------------
async def get_adapter(self, name: str) -> AdapterConfig | None:
"""Get configuration for a specific adapter."""
async with self._pool.acquire() as conn:
row = await conn.fetchrow(
"""
SELECT name, enabled, cadence_s, settings, paused_at, updated_at
FROM config.adapters
WHERE name = $1
""",
name,
)
if row is None:
return None
return AdapterConfig(**dict(row))
async def list_adapters(self) -> list[AdapterConfig]:
"""List all configured adapters."""
async with self._pool.acquire() as conn:
rows = await conn.fetch(
"""
SELECT name, enabled, cadence_s, settings, paused_at, updated_at
FROM config.adapters
ORDER BY name
"""
)
return [AdapterConfig(**dict(row)) for row in rows]
async def upsert_adapter(
self,
name: str,
enabled: bool,
cadence_s: int,
settings: dict[str, Any],
) -> None:
"""Insert or update an adapter configuration."""
async with self._pool.acquire() as conn:
await conn.execute(
"""
INSERT INTO config.adapters (name, enabled, cadence_s, settings, updated_at)
VALUES ($1, $2, $3, $4, now())
ON CONFLICT (name) DO UPDATE SET
enabled = EXCLUDED.enabled,
cadence_s = EXCLUDED.cadence_s,
settings = EXCLUDED.settings,
updated_at = now()
""",
name,
enabled,
cadence_s,
settings, # Will be encoded as JSON by the codec
)
async def pause_adapter(self, name: str) -> None:
"""Pause an adapter by setting paused_at."""
async with self._pool.acquire() as conn:
await conn.execute(
"""
UPDATE config.adapters
SET paused_at = now(), updated_at = now()
WHERE name = $1
""",
name,
)
async def unpause_adapter(self, name: str) -> None:
"""Unpause an adapter by clearing paused_at."""
async with self._pool.acquire() as conn:
await conn.execute(
"""
UPDATE config.adapters
SET paused_at = NULL, updated_at = now()
WHERE name = $1
""",
name,
)
# -------------------------------------------------------------------------
# Stream configuration
# -------------------------------------------------------------------------
async def get_stream(self, name: str) -> StreamConfig | None:
"""Get configuration for a specific stream."""
async with self._pool.acquire() as conn:
row = await conn.fetchrow(
"""
SELECT name, max_age_s, max_bytes, managed_max_bytes, updated_at
FROM config.streams
WHERE name = $1
""",
name,
)
if row is None:
return None
return StreamConfig(**dict(row))
async def list_streams(self) -> list[StreamConfig]:
"""List all configured streams."""
async with self._pool.acquire() as conn:
rows = await conn.fetch(
"""
SELECT name, max_age_s, max_bytes, managed_max_bytes, updated_at
FROM config.streams
ORDER BY name
"""
)
return [StreamConfig(**dict(row)) for row in rows]
async def upsert_stream(self, name: str, max_age_s: int) -> None:
"""Insert or update a stream's max_age_s (operator-facing)."""
async with self._pool.acquire() as conn:
await conn.execute(
"""
INSERT INTO config.streams (name, max_age_s, updated_at)
VALUES ($1, $2, now())
ON CONFLICT (name) DO UPDATE SET
max_age_s = EXCLUDED.max_age_s,
updated_at = now()
""",
name,
max_age_s,
)
async def update_stream_max_bytes(self, name: str, max_bytes: int) -> None:
"""Update a stream's max_bytes (supervisor-internal).
This update only touches max_bytes, which does NOT trigger
the column-filtered NOTIFY (only max_age_s changes fire NOTIFY).
"""
async with self._pool.acquire() as conn:
await conn.execute(
"""
UPDATE config.streams
SET max_bytes = $2, updated_at = now()
WHERE name = $1
""",
name,
max_bytes,
)
# -------------------------------------------------------------------------
# API key management
# -------------------------------------------------------------------------
async def set_api_key(self, alias: str, plaintext_value: str) -> None:
"""Store an API key, encrypting it with the master key."""
encrypted = encrypt(plaintext_value.encode("utf-8"))
async with self._pool.acquire() as conn:
await conn.execute(
"""
INSERT INTO config.api_keys (alias, encrypted_value)
VALUES ($1, $2)
ON CONFLICT (alias) DO UPDATE SET
encrypted_value = EXCLUDED.encrypted_value,
rotated_at = now()
""",
alias,
encrypted,
)
async def get_api_key(self, alias: str) -> str | None:
"""Retrieve and decrypt an API key by alias."""
async with self._pool.acquire() as conn:
row = await conn.fetchrow(
"""
SELECT encrypted_value FROM config.api_keys WHERE alias = $1
""",
alias,
)
if row is not None:
# Update last_used_at
await conn.execute(
"""
UPDATE config.api_keys SET last_used_at = now() WHERE alias = $1
""",
alias,
)
if row is None:
return None
return decrypt(row["encrypted_value"]).decode("utf-8")
async def delete_api_key(self, alias: str) -> bool:
"""Delete an API key. Returns True if key existed."""
async with self._pool.acquire() as conn:
result = await conn.execute(
"DELETE FROM config.api_keys WHERE alias = $1", alias
)
return result == "DELETE 1"
# -------------------------------------------------------------------------
# Change notifications
# -------------------------------------------------------------------------
async def listen_for_changes(
self,
callback: Callable[[str, str], Awaitable[None] | None],
) -> None:
"""Listen for config changes via Postgres NOTIFY.
Runs forever, calling callback(table, key) each time a change is
detected. The callback can be sync or async.
On connection loss, automatically reconnects with exponential backoff.
Cancellation (via task.cancel()) propagates cleanly.
Args:
callback: Function called with (table_name, row_key) on each change.
"""
backoff = 1.0
max_backoff = 30.0
while True:
conn = None
try:
conn = await self._pool.acquire()
logger.info("Config listener connected to database")
backoff = 1.0 # Reset backoff on successful connect
def notification_handler(
conn: asyncpg.Connection,
pid: int,
channel: str,
payload: str,
) -> None:
# payload format: "table_name:key"
if ":" in payload:
table, key = payload.split(":", 1)
else:
table, key = payload, ""
result = callback(table, key)
if asyncio.iscoroutine(result):
asyncio.create_task(result)
await conn.add_listener("config_changed", notification_handler)
try:
# Keep connection alive with periodic keepalive
while True:
await asyncio.sleep(60)
await conn.execute("SELECT 1")
finally:
await conn.remove_listener("config_changed", notification_handler)
except asyncio.CancelledError:
# Cancellation must propagate cleanly
logger.info("Config listener cancelled")
raise
except (
asyncpg.PostgresConnectionError,
asyncpg.InterfaceError,
ConnectionResetError,
OSError,
) as e:
logger.warning(
"Config listener connection lost, reconnecting in %.1fs: %s",
backoff,
e,
)
await asyncio.sleep(backoff)
backoff = min(backoff * 2, max_backoff)
except Exception as e:
# Unexpected error - log and retry with backoff
logger.exception(
"Config listener unexpected error, reconnecting in %.1fs",
backoff,
)
await asyncio.sleep(backoff)
backoff = min(backoff * 2, max_backoff)
finally:
if conn is not None:
try:
await self._pool.release(conn)
except Exception:
pass # Connection may already be invalid

222
src/central/crypto.py
View file

@ -1,111 +1,111 @@
"""Cryptographic primitives for secret storage.
Uses AES-256-GCM for authenticated encryption. The master key is read
from the path specified in bootstrap config on first use and cached.
"""
import base64
import os
from functools import lru_cache
from pathlib import Path
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
# AES-256 requires 32-byte key
KEY_SIZE = 32
# GCM nonce size (96 bits recommended by NIST)
NONCE_SIZE = 12
class CryptoError(Exception):
"""Base exception for crypto operations."""
class KeyLoadError(CryptoError):
"""Failed to load master key."""
class DecryptionError(CryptoError):
"""Failed to decrypt ciphertext (wrong key or tampered data)."""
@lru_cache
def _load_master_key(path: Path) -> bytes:
"""Load and decode the base64-encoded master key from file."""
try:
key_b64 = path.read_text().strip()
key = base64.b64decode(key_b64)
except FileNotFoundError:
raise KeyLoadError(f"Master key file not found: {path}")
except Exception as e:
raise KeyLoadError(f"Failed to read master key from {path}: {e}")
if len(key) != KEY_SIZE:
raise KeyLoadError(
f"Invalid master key size: expected {KEY_SIZE} bytes, got {len(key)}"
)
return key
def encrypt(plaintext: bytes, key_path: Path | None = None) -> bytes:
"""Encrypt plaintext using AES-256-GCM.
Args:
plaintext: Data to encrypt.
key_path: Path to master key file. If None, uses default from
bootstrap config.
Returns:
Ciphertext in format: nonce (12 bytes) || ciphertext || tag (16 bytes)
"""
if key_path is None:
from central.bootstrap_config import get_settings
key_path = get_settings().master_key_path
key = _load_master_key(key_path)
nonce = os.urandom(NONCE_SIZE)
aesgcm = AESGCM(key)
# GCM appends the 16-byte tag to the ciphertext
ciphertext_with_tag = aesgcm.encrypt(nonce, plaintext, associated_data=None)
return nonce + ciphertext_with_tag
def decrypt(ciphertext: bytes, key_path: Path | None = None) -> bytes:
"""Decrypt ciphertext using AES-256-GCM.
Args:
ciphertext: Data in format: nonce || ciphertext || tag
key_path: Path to master key file. If None, uses default from
bootstrap config.
Returns:
Decrypted plaintext.
Raises:
DecryptionError: If decryption fails (wrong key or tampered data).
"""
if key_path is None:
from central.bootstrap_config import get_settings
key_path = get_settings().master_key_path
if len(ciphertext) < NONCE_SIZE + 16: # nonce + minimum tag
raise DecryptionError("Ciphertext too short")
key = _load_master_key(key_path)
nonce = ciphertext[:NONCE_SIZE]
ciphertext_with_tag = ciphertext[NONCE_SIZE:]
aesgcm = AESGCM(key)
try:
plaintext = aesgcm.decrypt(nonce, ciphertext_with_tag, associated_data=None)
except Exception as e:
raise DecryptionError(f"Decryption failed: {e}")
return plaintext
def clear_key_cache() -> None:
"""Clear the cached master key. Use after key rotation."""
_load_master_key.cache_clear()
"""Cryptographic primitives for secret storage.
Uses AES-256-GCM for authenticated encryption. The master key is read
from the path specified in bootstrap config on first use and cached.
"""
import base64
import os
from functools import lru_cache
from pathlib import Path
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
# AES-256 requires 32-byte key
KEY_SIZE = 32
# GCM nonce size (96 bits recommended by NIST)
NONCE_SIZE = 12
class CryptoError(Exception):
"""Base exception for crypto operations."""
class KeyLoadError(CryptoError):
"""Failed to load master key."""
class DecryptionError(CryptoError):
"""Failed to decrypt ciphertext (wrong key or tampered data)."""
@lru_cache
def _load_master_key(path: Path) -> bytes:
"""Load and decode the base64-encoded master key from file."""
try:
key_b64 = path.read_text().strip()
key = base64.b64decode(key_b64)
except FileNotFoundError:
raise KeyLoadError(f"Master key file not found: {path}")
except Exception as e:
raise KeyLoadError(f"Failed to read master key from {path}: {e}")
if len(key) != KEY_SIZE:
raise KeyLoadError(
f"Invalid master key size: expected {KEY_SIZE} bytes, got {len(key)}"
)
return key
def encrypt(plaintext: bytes, key_path: Path | None = None) -> bytes:
"""Encrypt plaintext using AES-256-GCM.
Args:
plaintext: Data to encrypt.
key_path: Path to master key file. If None, uses default from
bootstrap config.
Returns:
Ciphertext in format: nonce (12 bytes) || ciphertext || tag (16 bytes)
"""
if key_path is None:
from central.bootstrap_config import get_settings
key_path = get_settings().master_key_path
key = _load_master_key(key_path)
nonce = os.urandom(NONCE_SIZE)
aesgcm = AESGCM(key)
# GCM appends the 16-byte tag to the ciphertext
ciphertext_with_tag = aesgcm.encrypt(nonce, plaintext, associated_data=None)
return nonce + ciphertext_with_tag
def decrypt(ciphertext: bytes, key_path: Path | None = None) -> bytes:
"""Decrypt ciphertext using AES-256-GCM.
Args:
ciphertext: Data in format: nonce || ciphertext || tag
key_path: Path to master key file. If None, uses default from
bootstrap config.
Returns:
Decrypted plaintext.
Raises:
DecryptionError: If decryption fails (wrong key or tampered data).
"""
if key_path is None:
from central.bootstrap_config import get_settings
key_path = get_settings().master_key_path
if len(ciphertext) < NONCE_SIZE + 16: # nonce + minimum tag
raise DecryptionError("Ciphertext too short")
key = _load_master_key(key_path)
nonce = ciphertext[:NONCE_SIZE]
ciphertext_with_tag = ciphertext[NONCE_SIZE:]
aesgcm = AESGCM(key)
try:
plaintext = aesgcm.decrypt(nonce, ciphertext_with_tag, associated_data=None)
except Exception as e:
raise DecryptionError(f"Decryption failed: {e}")
return plaintext
def clear_key_cache() -> None:
"""Clear the cached master key. Use after key rotation."""
_load_master_key.cache_clear()

250
src/central/migrate.py
View file

@ -1,125 +1,125 @@
"""Simple database migration runner.
Tracks applied migrations in a `schema_migrations` table. Migrations are
plain SQL files in `sql/migrations/` named with numeric prefixes:
001_create_config_schema.sql
002_add_operators_table.sql
...
Usage:
central-migrate [--dry-run]
"""
import argparse
import asyncio
import sys
from pathlib import Path
import asyncpg
MIGRATIONS_DIR = Path(__file__).parent.parent.parent / "sql" / "migrations"
async def ensure_migrations_table(conn: asyncpg.Connection) -> None:
"""Create the schema_migrations table if it doesn't exist."""
await conn.execute("""
CREATE TABLE IF NOT EXISTS schema_migrations (
version TEXT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
)
""")
async def get_applied_migrations(conn: asyncpg.Connection) -> set[str]:
"""Return set of already-applied migration versions."""
rows = await conn.fetch("SELECT version FROM schema_migrations")
return {row["version"] for row in rows}
def discover_migrations(migrations_dir: Path) -> list[tuple[str, Path]]:
"""Find all .sql files in migrations directory, sorted by name.
Returns list of (version, path) tuples where version is the filename
without extension.
"""
if not migrations_dir.exists():
return []
migrations = []
for f in sorted(migrations_dir.glob("*.sql")):
version = f.stem # e.g., "001_create_config_schema"
migrations.append((version, f))
return migrations
async def apply_migration(
conn: asyncpg.Connection, version: str, sql_path: Path, dry_run: bool = False
) -> None:
"""Apply a single migration."""
sql = sql_path.read_text()
if dry_run:
print(f"[DRY RUN] Would apply: {version}")
print(f" SQL: {sql[:200]}..." if len(sql) > 200 else f" SQL: {sql}")
return
async with conn.transaction():
await conn.execute(sql)
await conn.execute(
"INSERT INTO schema_migrations (version) VALUES ($1)", version
)
print(f"Applied: {version}")
async def run_migrations(dsn: str, dry_run: bool = False) -> int:
"""Run all pending migrations.
Returns number of migrations applied.
"""
conn = await asyncpg.connect(dsn)
try:
await ensure_migrations_table(conn)
applied = await get_applied_migrations(conn)
pending = [
(v, p) for v, p in discover_migrations(MIGRATIONS_DIR) if v not in applied
]
if not pending:
print("No pending migrations.")
return 0
print(f"Found {len(pending)} pending migration(s).")
for version, path in pending:
await apply_migration(conn, version, path, dry_run)
return len(pending)
finally:
await conn.close()
async def async_main() -> None:
"""Async entry point."""
parser = argparse.ArgumentParser(description="Run database migrations")
parser.add_argument(
"--dry-run",
action="store_true",
help="Show what would be applied without executing",
)
args = parser.parse_args()
from central.bootstrap_config import get_settings
settings = get_settings()
count = await run_migrations(settings.db_dsn, dry_run=args.dry_run)
if count > 0 and not args.dry_run:
print(f"Successfully applied {count} migration(s).")
def main() -> None:
"""Entry point."""
asyncio.run(async_main())
if __name__ == "__main__":
main()
"""Simple database migration runner.
Tracks applied migrations in a `schema_migrations` table. Migrations are
plain SQL files in `sql/migrations/` named with numeric prefixes:
001_create_config_schema.sql
002_add_operators_table.sql
...
Usage:
central-migrate [--dry-run]
"""
import argparse
import asyncio
import sys
from pathlib import Path
import asyncpg
MIGRATIONS_DIR = Path(__file__).parent.parent.parent / "sql" / "migrations"
async def ensure_migrations_table(conn: asyncpg.Connection) -> None:
"""Create the schema_migrations table if it doesn't exist."""
await conn.execute("""
CREATE TABLE IF NOT EXISTS schema_migrations (
version TEXT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
)
""")
async def get_applied_migrations(conn: asyncpg.Connection) -> set[str]:
"""Return set of already-applied migration versions."""
rows = await conn.fetch("SELECT version FROM schema_migrations")
return {row["version"] for row in rows}
def discover_migrations(migrations_dir: Path) -> list[tuple[str, Path]]:
"""Find all .sql files in migrations directory, sorted by name.
Returns list of (version, path) tuples where version is the filename
without extension.
"""
if not migrations_dir.exists():
return []
migrations = []
for f in sorted(migrations_dir.glob("*.sql")):
version = f.stem # e.g., "001_create_config_schema"
migrations.append((version, f))
return migrations
async def apply_migration(
conn: asyncpg.Connection, version: str, sql_path: Path, dry_run: bool = False
) -> None:
"""Apply a single migration."""
sql = sql_path.read_text()
if dry_run:
print(f"[DRY RUN] Would apply: {version}")
print(f" SQL: {sql[:200]}..." if len(sql) > 200 else f" SQL: {sql}")
return
async with conn.transaction():
await conn.execute(sql)
await conn.execute(
"INSERT INTO schema_migrations (version) VALUES ($1)", version
)
print(f"Applied: {version}")
async def run_migrations(dsn: str, dry_run: bool = False) -> int:
"""Run all pending migrations.
Returns number of migrations applied.
"""
conn = await asyncpg.connect(dsn)
try:
await ensure_migrations_table(conn)
applied = await get_applied_migrations(conn)
pending = [
(v, p) for v, p in discover_migrations(MIGRATIONS_DIR) if v not in applied
]
if not pending:
print("No pending migrations.")
return 0
print(f"Found {len(pending)} pending migration(s).")
for version, path in pending:
await apply_migration(conn, version, path, dry_run)
return len(pending)
finally:
await conn.close()
async def async_main() -> None:
"""Async entry point."""
parser = argparse.ArgumentParser(description="Run database migrations")
parser.add_argument(
"--dry-run",
action="store_true",
help="Show what would be applied without executing",
)
args = parser.parse_args()
from central.bootstrap_config import get_settings
settings = get_settings()
count = await run_migrations(settings.db_dsn, dry_run=args.dry_run)
if count > 0 and not args.dry_run:
print(f"Successfully applied {count} migration(s).")
def main() -> None:
"""Entry point."""
asyncio.run(async_main())
if __name__ == "__main__":
main()

8
src/central/models.py
View file

@ -38,7 +38,6 @@ def subject_for_event(ev: Event) -> str:
Dispatch by category prefix:
- fire.*: returns central.<category> directly
- quake.*: returns central.<category> directly
- wx.*: uses weather alert subject logic
Weather alert subjects:
@ -49,18 +48,11 @@ def subject_for_event(ev: Event) -> str:
Fire hotspot subjects:
central.fire.hotspot.<satellite>.<confidence>
Quake event subjects:
central.quake.event.<magnitude_tier>
"""
# Fire events: subject is just central.<category>
if ev.category.startswith("fire."):
return f"central.{ev.category}"
# Quake events: subject is just central.<category>
if ev.category.startswith("quake."):
return f"central.{ev.category}"
# Weather events: use geo-based subject logic
prefix = "central.wx"

524
src/central/stream_manager.py
View file

@ -1,262 +1,262 @@
"""JetStream stream manager for retention configuration."""
import logging
import re
from pathlib import Path
from typing import Any
from nats.js import JetStreamContext
from nats.js.api import StreamConfig, DiscardPolicy, RetentionPolicy
from central.config_models import StreamConfig as StreamConfigModel
logger = logging.getLogger(__name__)
# Constants
ONE_GB = 1024 * 1024 * 1024 # 1 GiB in bytes
NATS_CONFIG_PATH = Path("/etc/nats/nats-server.conf")
class StreamManager:
"""Manages JetStream stream configuration and retention."""
def __init__(self, js: JetStreamContext) -> None:
self._js = js
self._server_max_file_store: int | None = None
async def server_max_file_store_bytes(self) -> int:
"""Get the server's max_file_store setting in bytes.
Parses the NATS server config file and caches the result.
Returns a default of 20GB if config cannot be read.
"""
if self._server_max_file_store is not None:
return self._server_max_file_store
default_value = 20 * ONE_GB # 20GB default
try:
config_text = NATS_CONFIG_PATH.read_text()
# Parse max_file_store value (supports GB/MB/KB suffixes)
match = re.search(r'max_file_store:\s*(\d+)(GB|MB|KB|G|M|K)?', config_text, re.IGNORECASE)
if match:
value = int(match.group(1))
suffix = (match.group(2) or "").upper()
if suffix in ("GB", "G"):
value *= ONE_GB
elif suffix in ("MB", "M"):
value *= 1024 * 1024
elif suffix in ("KB", "K"):
value *= 1024
# else: assume bytes
self._server_max_file_store = value
logger.info(
"Parsed server max_file_store",
extra={"max_file_store_bytes": value},
)
return value
logger.warning(
"max_file_store not found in config, using default",
extra={"default": default_value},
)
self._server_max_file_store = default_value
return default_value
except Exception as e:
logger.warning(
"Failed to read NATS config, using default",
extra={"error": str(e), "default": default_value},
)
self._server_max_file_store = default_value
return default_value
def _compute_ceiling(self, server_max: int) -> int:
"""Compute per-stream ceiling as 30% of server max_file_store."""
return int(server_max * 0.30)
async def ensure_stream(
self,
name: str,
subjects: list[str],
config: StreamConfigModel,
) -> None:
"""Ensure a stream exists with the given configuration.
Creates the stream if it doesn't exist, or updates it if it does.
Always enforces: discard=old, max_msgs=-1 (unlimited).
"""
server_max = await self.server_max_file_store_bytes()
ceiling = self._compute_ceiling(server_max)
# Clamp max_bytes to [1GB, ceiling]
max_bytes = max(ONE_GB, min(config.max_bytes, ceiling))
stream_config = StreamConfig(
name=name,
subjects=subjects,
retention=RetentionPolicy.LIMITS,
discard=DiscardPolicy.OLD,
max_age=config.max_age_s,
max_bytes=max_bytes,
max_msgs=-1, # Unlimited messages
)
try:
# Try to get existing stream
existing = await self._js.stream_info(name)
# Update if config differs
await self._js.update_stream(config=stream_config)
logger.info(
"Updated stream",
extra={
"stream": name,
"max_age_s": config.max_age_s,
"max_bytes": max_bytes,
},
)
except Exception as e:
if "stream not found" in str(e).lower():
# Create new stream
await self._js.add_stream(config=stream_config)
logger.info(
"Created stream",
extra={
"stream": name,
"subjects": subjects,
"max_age_s": config.max_age_s,
"max_bytes": max_bytes,
},
)
else:
raise
async def apply_retention(self, name: str, config: StreamConfigModel) -> None:
"""Apply retention settings to an existing stream.
Updates max_age and max_bytes. Always enforces discard=old, max_msgs=-1.
"""
server_max = await self.server_max_file_store_bytes()
ceiling = self._compute_ceiling(server_max)
# Clamp max_bytes to [1GB, ceiling]
max_bytes = max(ONE_GB, min(config.max_bytes, ceiling))
try:
# Get current stream config
info = await self._js.stream_info(name)
current = info.config
# Build updated config
updated = StreamConfig(
name=name,
subjects=current.subjects,
retention=RetentionPolicy.LIMITS,
discard=DiscardPolicy.OLD,
max_age=config.max_age_s,
max_bytes=max_bytes,
max_msgs=-1,
)
await self._js.update_stream(config=updated)
logger.info(
"Applied retention",
extra={
"stream": name,
"max_age_s": config.max_age_s,
"max_bytes": max_bytes,
},
)
except Exception as e:
logger.error(
"Failed to apply retention",
extra={"stream": name, "error": str(e)},
)
raise
async def recompute_max_bytes(self, name: str, max_age_s: int) -> int:
"""Recompute max_bytes based on observed throughput.
Formula: rate × max_age × 1.5 safety margin, clamped to [1GB, ceiling].
Returns the computed max_bytes value.
"""
server_max = await self.server_max_file_store_bytes()
ceiling = self._compute_ceiling(server_max)
try:
info = await self._js.stream_info(name)
current_bytes = info.state.bytes
current_msgs = info.state.messages
# Get stream age from first message
first_seq = info.state.first_seq
last_seq = info.state.last_seq
if current_msgs == 0 or last_seq == 0:
# No messages yet, use floor
return ONE_GB
# Estimate message age span (approximation)
# Use stream's configured max_age as the observation window
configured_max_age = info.config.max_age
if configured_max_age > 0:
# Rate = current_bytes / configured_max_age (in seconds)
rate_per_second = current_bytes / configured_max_age
else:
# Fallback: assume 1 day of data
rate_per_second = current_bytes / 86400
# Project bytes needed for new max_age with 1.5x safety margin
projected = int(rate_per_second * max_age_s * 1.5)
# Clamp to [1GB, ceiling]
result = max(ONE_GB, min(projected, ceiling))
logger.info(
"Recomputed max_bytes",
extra={
"stream": name,
"current_bytes": current_bytes,
"rate_per_second": rate_per_second,
"max_age_s": max_age_s,
"projected": projected,
"result": result,
"ceiling": ceiling,
},
)
return result
except Exception as e:
logger.error(
"Failed to recompute max_bytes, using floor",
extra={"stream": name, "error": str(e)},
)
return ONE_GB
async def get_stream_stats(self, name: str) -> dict[str, Any]:
"""Get current stream statistics for monitoring."""
try:
info = await self._js.stream_info(name)
return {
"stream": name,
"bytes": info.state.bytes,
"messages": info.state.messages,
"max_bytes": info.config.max_bytes,
"max_age_s": info.config.max_age,
"consumers": info.state.consumer_count,
}
except Exception as e:
logger.error(
"Failed to get stream stats",
extra={"stream": name, "error": str(e)},
)
return {"stream": name, "error": str(e)}
"""JetStream stream manager for retention configuration."""
import logging
import re
from pathlib import Path
from typing import Any
from nats.js import JetStreamContext
from nats.js.api import StreamConfig, DiscardPolicy, RetentionPolicy
from central.config_models import StreamConfig as StreamConfigModel
logger = logging.getLogger(__name__)
# Constants
ONE_GB = 1024 * 1024 * 1024 # 1 GiB in bytes
NATS_CONFIG_PATH = Path("/etc/nats/nats-server.conf")
class StreamManager:
"""Manages JetStream stream configuration and retention."""
def __init__(self, js: JetStreamContext) -> None:
self._js = js
self._server_max_file_store: int | None = None
async def server_max_file_store_bytes(self) -> int:
"""Get the server's max_file_store setting in bytes.
Parses the NATS server config file and caches the result.
Returns a default of 20GB if config cannot be read.
"""
if self._server_max_file_store is not None:
return self._server_max_file_store
default_value = 20 * ONE_GB # 20GB default
try:
config_text = NATS_CONFIG_PATH.read_text()
# Parse max_file_store value (supports GB/MB/KB suffixes)
match = re.search(r'max_file_store:\s*(\d+)(GB|MB|KB|G|M|K)?', config_text, re.IGNORECASE)
if match:
value = int(match.group(1))
suffix = (match.group(2) or "").upper()
if suffix in ("GB", "G"):
value *= ONE_GB
elif suffix in ("MB", "M"):
value *= 1024 * 1024
elif suffix in ("KB", "K"):
value *= 1024
# else: assume bytes
self._server_max_file_store = value
logger.info(
"Parsed server max_file_store",
extra={"max_file_store_bytes": value},
)
return value
logger.warning(
"max_file_store not found in config, using default",
extra={"default": default_value},
)
self._server_max_file_store = default_value
return default_value
except Exception as e:
logger.warning(
"Failed to read NATS config, using default",
extra={"error": str(e), "default": default_value},
)
self._server_max_file_store = default_value
return default_value
def _compute_ceiling(self, server_max: int) -> int:
"""Compute per-stream ceiling as 30% of server max_file_store."""
return int(server_max * 0.30)
async def ensure_stream(
self,
name: str,
subjects: list[str],
config: StreamConfigModel,
) -> None:
"""Ensure a stream exists with the given configuration.
Creates the stream if it doesn't exist, or updates it if it does.
Always enforces: discard=old, max_msgs=-1 (unlimited).
"""
server_max = await self.server_max_file_store_bytes()
ceiling = self._compute_ceiling(server_max)
# Clamp max_bytes to [1GB, ceiling]
max_bytes = max(ONE_GB, min(config.max_bytes, ceiling))
stream_config = StreamConfig(
name=name,
subjects=subjects,
retention=RetentionPolicy.LIMITS,
discard=DiscardPolicy.OLD,
max_age=config.max_age_s,
max_bytes=max_bytes,
max_msgs=-1, # Unlimited messages
)
try:
# Try to get existing stream
existing = await self._js.stream_info(name)
# Update if config differs
await self._js.update_stream(config=stream_config)
logger.info(
"Updated stream",
extra={
"stream": name,
"max_age_s": config.max_age_s,
"max_bytes": max_bytes,
},
)
except Exception as e:
if "stream not found" in str(e).lower():
# Create new stream
await self._js.add_stream(config=stream_config)
logger.info(
"Created stream",
extra={
"stream": name,
"subjects": subjects,
"max_age_s": config.max_age_s,
"max_bytes": max_bytes,
},
)
else:
raise
async def apply_retention(self, name: str, config: StreamConfigModel) -> None:
"""Apply retention settings to an existing stream.
Updates max_age and max_bytes. Always enforces discard=old, max_msgs=-1.
"""
server_max = await self.server_max_file_store_bytes()
ceiling = self._compute_ceiling(server_max)
# Clamp max_bytes to [1GB, ceiling]
max_bytes = max(ONE_GB, min(config.max_bytes, ceiling))
try:
# Get current stream config
info = await self._js.stream_info(name)
current = info.config
# Build updated config
updated = StreamConfig(
name=name,
subjects=current.subjects,
retention=RetentionPolicy.LIMITS,
discard=DiscardPolicy.OLD,
max_age=config.max_age_s,
max_bytes=max_bytes,
max_msgs=-1,
)
await self._js.update_stream(config=updated)
logger.info(
"Applied retention",
extra={
"stream": name,
"max_age_s": config.max_age_s,
"max_bytes": max_bytes,
},
)
except Exception as e:
logger.error(
"Failed to apply retention",
extra={"stream": name, "error": str(e)},
)
raise
async def recompute_max_bytes(self, name: str, max_age_s: int) -> int:
"""Recompute max_bytes based on observed throughput.
Formula: rate × max_age × 1.5 safety margin, clamped to [1GB, ceiling].
Returns the computed max_bytes value.
"""
server_max = await self.server_max_file_store_bytes()
ceiling = self._compute_ceiling(server_max)
try:
info = await self._js.stream_info(name)
current_bytes = info.state.bytes
current_msgs = info.state.messages
# Get stream age from first message
first_seq = info.state.first_seq
last_seq = info.state.last_seq
if current_msgs == 0 or last_seq == 0:
# No messages yet, use floor
return ONE_GB
# Estimate message age span (approximation)
# Use stream's configured max_age as the observation window
configured_max_age = info.config.max_age
if configured_max_age > 0:
# Rate = current_bytes / configured_max_age (in seconds)
rate_per_second = current_bytes / configured_max_age
else:
# Fallback: assume 1 day of data
rate_per_second = current_bytes / 86400
# Project bytes needed for new max_age with 1.5x safety margin
projected = int(rate_per_second * max_age_s * 1.5)
# Clamp to [1GB, ceiling]
result = max(ONE_GB, min(projected, ceiling))
logger.info(
"Recomputed max_bytes",
extra={
"stream": name,
"current_bytes": current_bytes,
"rate_per_second": rate_per_second,
"max_age_s": max_age_s,
"projected": projected,
"result": result,
"ceiling": ceiling,
},
)
return result
except Exception as e:
logger.error(
"Failed to recompute max_bytes, using floor",
extra={"stream": name, "error": str(e)},
)
return ONE_GB
async def get_stream_stats(self, name: str) -> dict[str, Any]:
"""Get current stream statistics for monitoring."""
try:
info = await self._js.stream_info(name)
return {
"stream": name,
"bytes": info.state.bytes,
"messages": info.state.messages,
"max_bytes": info.config.max_bytes,
"max_age_s": info.config.max_age,
"consumers": info.state.consumer_count,
}
except Exception as e:
logger.error(
"Failed to get stream stats",
extra={"stream": name, "error": str(e)},
)
return {"stream": name, "error": str(e)}

36
tests/README.md
View file

@ -1,18 +1,18 @@
# Central Tests
## Test Database
Some tests (notably `test_config_store.py`) require a real PostgreSQL database.
By default, tests connect to:
```
postgresql://central_test:testpass@localhost/central_test
```
If your test database uses different credentials, set the `CENTRAL_TEST_DB_DSN`
environment variable:
```bash
export CENTRAL_TEST_DB_DSN="postgresql://myuser:mypass@localhost/mydb"
uv run pytest tests/test_config_store.py
```
# Central Tests
## Test Database
Some tests (notably `test_config_store.py`) require a real PostgreSQL database.
By default, tests connect to:
```
postgresql://central_test:testpass@localhost/central_test
```
If your test database uses different credentials, set the `CENTRAL_TEST_DB_DSN`
environment variable:
```bash
export CENTRAL_TEST_DB_DSN="postgresql://myuser:mypass@localhost/mydb"
uv run pytest tests/test_config_store.py
```

246
tests/test_bootstrap_config.py
View file

@ -1,123 +1,123 @@
"""Tests for bootstrap configuration."""
import os
from pathlib import Path
from tempfile import NamedTemporaryFile
import pytest
from central.bootstrap_config import Settings, get_settings
class TestSettingsFromEnv:
"""Test loading settings from environment variables."""
def test_reads_from_env(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Settings are read from CENTRAL_* environment variables."""
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://test:pass@localhost/testdb")
monkeypatch.setenv("CENTRAL_NATS_URL", "nats://10.0.0.1:4222")
monkeypatch.setenv("CENTRAL_MASTER_KEY_PATH", "/tmp/test.key")
monkeypatch.setenv("CENTRAL_LOG_LEVEL", "DEBUG")
settings = Settings()
assert settings.db_dsn == "postgresql://test:pass@localhost/testdb"
assert settings.nats_url == "nats://10.0.0.1:4222"
assert settings.master_key_path == Path("/tmp/test.key")
assert settings.log_level == "DEBUG"
def test_defaults_applied(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Default values are used when env vars not set."""
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://x:y@localhost/db")
# Clear any existing env vars that might interfere
monkeypatch.delenv("CENTRAL_NATS_URL", raising=False)
monkeypatch.delenv("CENTRAL_MASTER_KEY_PATH", raising=False)
monkeypatch.delenv("CENTRAL_LOG_LEVEL", raising=False)
settings = Settings()
assert settings.nats_url == "nats://localhost:4222"
assert settings.master_key_path == Path("/etc/central/master.key")
assert settings.log_level == "INFO"
class TestSettingsFromFile:
"""Test loading settings from .env file."""
def test_reads_from_env_file(self, tmp_path: Path) -> None:
"""Settings are read from .env file when env vars not present."""
env_file = tmp_path / ".env"
env_file.write_text(
"CENTRAL_DB_DSN=postgresql://file:pass@localhost/filedb\n"
"CENTRAL_NATS_URL=nats://file.local:4222\n"
"CENTRAL_LOG_LEVEL=WARNING\n"
)
# Create settings pointing to the temp .env file
settings = Settings(_env_file=env_file)
assert settings.db_dsn == "postgresql://file:pass@localhost/filedb"
assert settings.nats_url == "nats://file.local:4222"
assert settings.log_level == "WARNING"
def test_env_vars_override_file(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Environment variables take precedence over .env file."""
env_file = tmp_path / ".env"
env_file.write_text("CENTRAL_DB_DSN=postgresql://file@localhost/filedb\n")
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://env@localhost/envdb")
settings = Settings(_env_file=env_file)
assert settings.db_dsn == "postgresql://env@localhost/envdb"
class TestSettingsValidation:
"""Test settings validation and error handling."""
def test_fails_if_required_var_missing(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Clear error when required CENTRAL_DB_DSN is missing."""
# Ensure no env vars or .env file provides the DSN
monkeypatch.delenv("CENTRAL_DB_DSN", raising=False)
with pytest.raises(Exception) as exc_info:
# Use a non-existent .env file path to ensure no fallback
Settings(_env_file=Path("/nonexistent/.env"))
# pydantic-settings raises ValidationError for missing required fields
assert "db_dsn" in str(exc_info.value).lower() or "validation" in str(exc_info.value).lower()
def test_invalid_log_level_rejected(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Invalid log level values are rejected."""
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://x@localhost/db")
monkeypatch.setenv("CENTRAL_LOG_LEVEL", "INVALID")
with pytest.raises(Exception):
Settings()
class TestGetSettings:
"""Test the cached settings loader."""
def test_caches_result(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""get_settings() returns cached instance."""
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://cached@localhost/db")
get_settings.cache_clear()
s1 = get_settings()
s2 = get_settings()
assert s1 is s2
def test_cache_clear_reloads(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""cache_clear() forces reload on next call."""
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://first@localhost/db")
get_settings.cache_clear()
s1 = get_settings()
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://second@localhost/db")
get_settings.cache_clear()
s2 = get_settings()
assert s1.db_dsn != s2.db_dsn
"""Tests for bootstrap configuration."""
import os
from pathlib import Path
from tempfile import NamedTemporaryFile
import pytest
from central.bootstrap_config import Settings, get_settings
class TestSettingsFromEnv:
"""Test loading settings from environment variables."""
def test_reads_from_env(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Settings are read from CENTRAL_* environment variables."""
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://test:pass@localhost/testdb")
monkeypatch.setenv("CENTRAL_NATS_URL", "nats://10.0.0.1:4222")
monkeypatch.setenv("CENTRAL_MASTER_KEY_PATH", "/tmp/test.key")
monkeypatch.setenv("CENTRAL_LOG_LEVEL", "DEBUG")
settings = Settings()
assert settings.db_dsn == "postgresql://test:pass@localhost/testdb"
assert settings.nats_url == "nats://10.0.0.1:4222"
assert settings.master_key_path == Path("/tmp/test.key")
assert settings.log_level == "DEBUG"
def test_defaults_applied(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Default values are used when env vars not set."""
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://x:y@localhost/db")
# Clear any existing env vars that might interfere
monkeypatch.delenv("CENTRAL_NATS_URL", raising=False)
monkeypatch.delenv("CENTRAL_MASTER_KEY_PATH", raising=False)
monkeypatch.delenv("CENTRAL_LOG_LEVEL", raising=False)
settings = Settings()
assert settings.nats_url == "nats://localhost:4222"
assert settings.master_key_path == Path("/etc/central/master.key")
assert settings.log_level == "INFO"
class TestSettingsFromFile:
"""Test loading settings from .env file."""
def test_reads_from_env_file(self, tmp_path: Path) -> None:
"""Settings are read from .env file when env vars not present."""
env_file = tmp_path / ".env"
env_file.write_text(
"CENTRAL_DB_DSN=postgresql://file:pass@localhost/filedb\n"
"CENTRAL_NATS_URL=nats://file.local:4222\n"
"CENTRAL_LOG_LEVEL=WARNING\n"
)
# Create settings pointing to the temp .env file
settings = Settings(_env_file=env_file)
assert settings.db_dsn == "postgresql://file:pass@localhost/filedb"
assert settings.nats_url == "nats://file.local:4222"
assert settings.log_level == "WARNING"
def test_env_vars_override_file(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Environment variables take precedence over .env file."""
env_file = tmp_path / ".env"
env_file.write_text("CENTRAL_DB_DSN=postgresql://file@localhost/filedb\n")
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://env@localhost/envdb")
settings = Settings(_env_file=env_file)
assert settings.db_dsn == "postgresql://env@localhost/envdb"
class TestSettingsValidation:
"""Test settings validation and error handling."""
def test_fails_if_required_var_missing(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Clear error when required CENTRAL_DB_DSN is missing."""
# Ensure no env vars or .env file provides the DSN
monkeypatch.delenv("CENTRAL_DB_DSN", raising=False)
with pytest.raises(Exception) as exc_info:
# Use a non-existent .env file path to ensure no fallback
Settings(_env_file=Path("/nonexistent/.env"))
# pydantic-settings raises ValidationError for missing required fields
assert "db_dsn" in str(exc_info.value).lower() or "validation" in str(exc_info.value).lower()
def test_invalid_log_level_rejected(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Invalid log level values are rejected."""
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://x@localhost/db")
monkeypatch.setenv("CENTRAL_LOG_LEVEL", "INVALID")
with pytest.raises(Exception):
Settings()
class TestGetSettings:
"""Test the cached settings loader."""
def test_caches_result(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""get_settings() returns cached instance."""
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://cached@localhost/db")
get_settings.cache_clear()
s1 = get_settings()
s2 = get_settings()
assert s1 is s2
def test_cache_clear_reloads(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""cache_clear() forces reload on next call."""
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://first@localhost/db")
get_settings.cache_clear()
s1 = get_settings()
monkeypatch.setenv("CENTRAL_DB_DSN", "postgresql://second@localhost/db")
get_settings.cache_clear()
s2 = get_settings()
assert s1.db_dsn != s2.db_dsn

264
tests/test_config_source.py
View file

@ -1,132 +1,132 @@
"""Tests for configuration source abstraction."""
import base64
import os
from pathlib import Path
import asyncpg
import pytest
import pytest_asyncio
from central.config_source import (
ConfigSource,
DbConfigSource,
)
from central.crypto import KEY_SIZE, clear_key_cache
# Test database DSN
TEST_DB_DSN = os.environ.get(
"CENTRAL_TEST_DB_DSN",
"postgresql://central_test:testpass@localhost/central_test",
)
@pytest.fixture(scope="session")
def master_key_path(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""Create a master key file for the test session."""
key = os.urandom(KEY_SIZE)
key_path = tmp_path_factory.mktemp("keys") / "master.key"
key_path.write_text(base64.b64encode(key).decode())
return key_path
@pytest.fixture(autouse=True)
def setup_master_key(master_key_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Configure master key path for all tests."""
clear_key_cache()
monkeypatch.setenv("CENTRAL_DB_DSN", TEST_DB_DSN)
monkeypatch.setenv("CENTRAL_MASTER_KEY_PATH", str(master_key_path))
@pytest_asyncio.fixture
async def db_conn() -> asyncpg.Connection:
"""Get a direct database connection for setup/teardown."""
conn = await asyncpg.connect(TEST_DB_DSN)
yield conn
await conn.close()
@pytest_asyncio.fixture
async def clean_config_schema(db_conn: asyncpg.Connection) -> None:
"""Ensure config schema exists and is clean before each test."""
await db_conn.execute("CREATE SCHEMA IF NOT EXISTS config")
await db_conn.execute("""
CREATE TABLE IF NOT EXISTS config.adapters (
name TEXT PRIMARY KEY,
enabled BOOLEAN NOT NULL DEFAULT true,
cadence_s INTEGER NOT NULL,
settings JSONB NOT NULL DEFAULT '{}'::jsonb,
paused_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
)
""")
await db_conn.execute("DELETE FROM config.adapters")
class TestDbConfigSource:
"""Tests for database-backed config source."""
@pytest_asyncio.fixture
async def db_source(self, clean_config_schema: None) -> DbConfigSource:
"""Create a DbConfigSource for testing."""
source = await DbConfigSource.create(TEST_DB_DSN)
yield source
await source.close()
@pytest.mark.asyncio
async def test_list_enabled_adapters_empty(self, db_source: DbConfigSource) -> None:
"""list_enabled_adapters returns empty list when no adapters."""
adapters = await db_source.list_enabled_adapters()
assert adapters == []
@pytest.mark.asyncio
async def test_list_enabled_adapters(
self, db_source: DbConfigSource, db_conn: asyncpg.Connection
) -> None:
"""list_enabled_adapters returns only enabled, non-paused adapters."""
# Insert test adapters
await db_conn.execute("""
INSERT INTO config.adapters (name, enabled, cadence_s, settings)
VALUES
('enabled_adapter', true, 60, '{"key": "value"}'::jsonb),
('disabled_adapter', false, 60, '{}'::jsonb),
('paused_adapter', true, 60, '{}'::jsonb)
""")
await db_conn.execute("""
UPDATE config.adapters
SET paused_at = now()
WHERE name = 'paused_adapter'
""")
adapters = await db_source.list_enabled_adapters()
assert len(adapters) == 1
assert adapters[0].name == "enabled_adapter"
@pytest.mark.asyncio
async def test_get_adapter(
self, db_source: DbConfigSource, db_conn: asyncpg.Connection
) -> None:
"""get_adapter returns correct adapter config."""
await db_conn.execute("""
INSERT INTO config.adapters (name, enabled, cadence_s, settings)
VALUES ('test_adapter', true, 120, '{"states": ["ID"]}'::jsonb)
""")
adapter = await db_source.get_adapter("test_adapter")
assert adapter is not None
assert adapter.name == "test_adapter"
assert adapter.cadence_s == 120
assert adapter.settings == {"states": ["ID"]}
@pytest.mark.asyncio
async def test_get_nonexistent_adapter(self, db_source: DbConfigSource) -> None:
"""get_adapter returns None for nonexistent adapter."""
adapter = await db_source.get_adapter("does_not_exist")
assert adapter is None
@pytest.mark.asyncio
async def test_implements_protocol(self, db_source: DbConfigSource) -> None:
"""DbConfigSource implements ConfigSource protocol."""
assert isinstance(db_source, ConfigSource)
"""Tests for configuration source abstraction."""
import base64
import os
from pathlib import Path
import asyncpg
import pytest
import pytest_asyncio
from central.config_source import (
ConfigSource,
DbConfigSource,
)
from central.crypto import KEY_SIZE, clear_key_cache
# Test database DSN
TEST_DB_DSN = os.environ.get(
"CENTRAL_TEST_DB_DSN",
"postgresql://central_test:testpass@localhost/central_test",
)
@pytest.fixture(scope="session")
def master_key_path(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""Create a master key file for the test session."""
key = os.urandom(KEY_SIZE)
key_path = tmp_path_factory.mktemp("keys") / "master.key"
key_path.write_text(base64.b64encode(key).decode())
return key_path
@pytest.fixture(autouse=True)
def setup_master_key(master_key_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Configure master key path for all tests."""
clear_key_cache()
monkeypatch.setenv("CENTRAL_DB_DSN", TEST_DB_DSN)
monkeypatch.setenv("CENTRAL_MASTER_KEY_PATH", str(master_key_path))
@pytest_asyncio.fixture
async def db_conn() -> asyncpg.Connection:
"""Get a direct database connection for setup/teardown."""
conn = await asyncpg.connect(TEST_DB_DSN)
yield conn
await conn.close()
@pytest_asyncio.fixture
async def clean_config_schema(db_conn: asyncpg.Connection) -> None:
"""Ensure config schema exists and is clean before each test."""
await db_conn.execute("CREATE SCHEMA IF NOT EXISTS config")
await db_conn.execute("""
CREATE TABLE IF NOT EXISTS config.adapters (
name TEXT PRIMARY KEY,
enabled BOOLEAN NOT NULL DEFAULT true,
cadence_s INTEGER NOT NULL,
settings JSONB NOT NULL DEFAULT '{}'::jsonb,
paused_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
)
""")
await db_conn.execute("DELETE FROM config.adapters")
class TestDbConfigSource:
"""Tests for database-backed config source."""
@pytest_asyncio.fixture
async def db_source(self, clean_config_schema: None) -> DbConfigSource:
"""Create a DbConfigSource for testing."""
source = await DbConfigSource.create(TEST_DB_DSN)
yield source
await source.close()
@pytest.mark.asyncio
async def test_list_enabled_adapters_empty(self, db_source: DbConfigSource) -> None:
"""list_enabled_adapters returns empty list when no adapters."""
adapters = await db_source.list_enabled_adapters()
assert adapters == []
@pytest.mark.asyncio
async def test_list_enabled_adapters(
self, db_source: DbConfigSource, db_conn: asyncpg.Connection
) -> None:
"""list_enabled_adapters returns only enabled, non-paused adapters."""
# Insert test adapters
await db_conn.execute("""
INSERT INTO config.adapters (name, enabled, cadence_s, settings)
VALUES
('enabled_adapter', true, 60, '{"key": "value"}'::jsonb),
('disabled_adapter', false, 60, '{}'::jsonb),
('paused_adapter', true, 60, '{}'::jsonb)
""")
await db_conn.execute("""
UPDATE config.adapters
SET paused_at = now()
WHERE name = 'paused_adapter'
""")
adapters = await db_source.list_enabled_adapters()
assert len(adapters) == 1
assert adapters[0].name == "enabled_adapter"
@pytest.mark.asyncio
async def test_get_adapter(
self, db_source: DbConfigSource, db_conn: asyncpg.Connection
) -> None:
"""get_adapter returns correct adapter config."""
await db_conn.execute("""
INSERT INTO config.adapters (name, enabled, cadence_s, settings)
VALUES ('test_adapter', true, 120, '{"states": ["ID"]}'::jsonb)
""")
adapter = await db_source.get_adapter("test_adapter")
assert adapter is not None
assert adapter.name == "test_adapter"
assert adapter.cadence_s == 120
assert adapter.settings == {"states": ["ID"]}
@pytest.mark.asyncio
async def test_get_nonexistent_adapter(self, db_source: DbConfigSource) -> None:
"""get_adapter returns None for nonexistent adapter."""
adapter = await db_source.get_adapter("does_not_exist")
assert adapter is None
@pytest.mark.asyncio
async def test_implements_protocol(self, db_source: DbConfigSource) -> None:
"""DbConfigSource implements ConfigSource protocol."""
assert isinstance(db_source, ConfigSource)

678
tests/test_config_store.py
View file

@ -1,339 +1,339 @@
"""Tests for database-backed configuration store.
These tests require a real Postgres database. Set CENTRAL_TEST_DB_DSN
environment variable to override the default test database connection.
"""
import asyncio
import base64
import os
from pathlib import Path
import asyncpg
import pytest
import pytest_asyncio
from central.config_store import ConfigStore
from central.crypto import KEY_SIZE, clear_key_cache
# Test database DSN - uses central_test database with well-known test password.
# Override via CENTRAL_TEST_DB_DSN env var if your test DB differs.
TEST_DB_DSN = os.environ.get(
"CENTRAL_TEST_DB_DSN",
"postgresql://central_test:testpass@localhost/central_test",
)
@pytest.fixture(scope="session")
def master_key_path(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""Create a master key file for the test session."""
key = os.urandom(KEY_SIZE)
key_path = tmp_path_factory.mktemp("keys") / "master.key"
key_path.write_text(base64.b64encode(key).decode())
return key_path
@pytest.fixture(autouse=True)
def setup_master_key(master_key_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Configure master key path for all tests."""
clear_key_cache()
monkeypatch.setenv("CENTRAL_DB_DSN", TEST_DB_DSN)
monkeypatch.setenv("CENTRAL_MASTER_KEY_PATH", str(master_key_path))
@pytest_asyncio.fixture
async def db_conn() -> asyncpg.Connection:
"""Get a direct database connection for setup/teardown."""
conn = await asyncpg.connect(TEST_DB_DSN)
yield conn
await conn.close()
@pytest_asyncio.fixture
async def clean_config_schema(db_conn: asyncpg.Connection) -> None:
"""Ensure config schema exists and is clean before each test."""
# Create schema if not exists
await db_conn.execute("CREATE SCHEMA IF NOT EXISTS config")
# Create tables if not exist
await db_conn.execute("""
CREATE TABLE IF NOT EXISTS config.adapters (
name TEXT PRIMARY KEY,
enabled BOOLEAN NOT NULL DEFAULT true,
cadence_s INTEGER NOT NULL,
settings JSONB NOT NULL DEFAULT '{}'::jsonb,
paused_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
)
""")
await db_conn.execute("""
CREATE TABLE IF NOT EXISTS config.api_keys (
alias TEXT PRIMARY KEY,
encrypted_value BYTEA NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
rotated_at TIMESTAMPTZ,
last_used_at TIMESTAMPTZ
)
""")
# Create notify function with proper key detection
await db_conn.execute("""
CREATE OR REPLACE FUNCTION config.notify_config_change()
RETURNS trigger AS $$
DECLARE
key_value TEXT;
BEGIN
IF TG_TABLE_NAME = 'adapters' THEN
key_value := COALESCE(NEW.name, OLD.name, '');
ELSIF TG_TABLE_NAME = 'api_keys' THEN
key_value := COALESCE(NEW.alias, OLD.alias, '');
ELSE
key_value := '';
END IF;
PERFORM pg_notify('config_changed', TG_TABLE_NAME || ':' || key_value);
RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql
""")
# Create triggers if not exist
await db_conn.execute("""
DROP TRIGGER IF EXISTS adapters_notify ON config.adapters;
CREATE TRIGGER adapters_notify
AFTER INSERT OR UPDATE OR DELETE ON config.adapters
FOR EACH ROW EXECUTE FUNCTION config.notify_config_change()
""")
await db_conn.execute("""
DROP TRIGGER IF EXISTS api_keys_notify ON config.api_keys;
CREATE TRIGGER api_keys_notify
AFTER INSERT OR UPDATE OR DELETE ON config.api_keys
FOR EACH ROW EXECUTE FUNCTION config.notify_config_change()
""")
# Clean tables
await db_conn.execute("DELETE FROM config.adapters")
await db_conn.execute("DELETE FROM config.api_keys")
@pytest_asyncio.fixture
async def config_store(clean_config_schema: None) -> ConfigStore:
"""Create a ConfigStore connected to the test database."""
store = await ConfigStore.create(TEST_DB_DSN)
yield store
await store.close()
class TestAdapterConfig:
"""Tests for adapter configuration operations."""
@pytest.mark.asyncio
async def test_upsert_and_get(self, config_store: ConfigStore) -> None:
"""Can insert and retrieve adapter config."""
await config_store.upsert_adapter(
name="test_adapter",
enabled=True,
cadence_s=120,
settings={"key": "value"},
)
adapter = await config_store.get_adapter("test_adapter")
assert adapter is not None
assert adapter.name == "test_adapter"
assert adapter.enabled is True
assert adapter.cadence_s == 120
assert adapter.settings == {"key": "value"}
@pytest.mark.asyncio
async def test_get_nonexistent(self, config_store: ConfigStore) -> None:
"""Getting nonexistent adapter returns None."""
adapter = await config_store.get_adapter("does_not_exist")
assert adapter is None
@pytest.mark.asyncio
async def test_list_adapters(self, config_store: ConfigStore) -> None:
"""Can list all adapters."""
await config_store.upsert_adapter("adapter_a", True, 60, {})
await config_store.upsert_adapter("adapter_b", False, 300, {"x": 1})
adapters = await config_store.list_adapters()
assert len(adapters) == 2
names = [a.name for a in adapters]
assert "adapter_a" in names
assert "adapter_b" in names
@pytest.mark.asyncio
async def test_upsert_updates_existing(self, config_store: ConfigStore) -> None:
"""Upsert updates existing adapter."""
await config_store.upsert_adapter("updater", True, 60, {"v": 1})
await config_store.upsert_adapter("updater", False, 120, {"v": 2})
adapter = await config_store.get_adapter("updater")
assert adapter is not None
assert adapter.enabled is False
assert adapter.cadence_s == 120
assert adapter.settings == {"v": 2}
@pytest.mark.asyncio
async def test_pause_unpause(self, config_store: ConfigStore) -> None:
"""Can pause and unpause adapter."""
await config_store.upsert_adapter("pausable", True, 60, {})
await config_store.pause_adapter("pausable")
adapter = await config_store.get_adapter("pausable")
assert adapter is not None
assert adapter.is_paused is True
await config_store.unpause_adapter("pausable")
adapter = await config_store.get_adapter("pausable")
assert adapter is not None
assert adapter.is_paused is False
class TestApiKeys:
"""Tests for API key operations."""
@pytest.mark.asyncio
async def test_set_and_get_key(self, config_store: ConfigStore) -> None:
"""Can store and retrieve encrypted API key."""
await config_store.set_api_key("test_key", "super_secret_value")
value = await config_store.get_api_key("test_key")
assert value == "super_secret_value"
@pytest.mark.asyncio
async def test_get_nonexistent_key(self, config_store: ConfigStore) -> None:
"""Getting nonexistent key returns None."""
value = await config_store.get_api_key("does_not_exist")
assert value is None
@pytest.mark.asyncio
async def test_key_rotation(self, config_store: ConfigStore) -> None:
"""Updating key sets rotated_at."""
await config_store.set_api_key("rotate_me", "value1")
await config_store.set_api_key("rotate_me", "value2")
value = await config_store.get_api_key("rotate_me")
assert value == "value2"
@pytest.mark.asyncio
async def test_delete_key(self, config_store: ConfigStore) -> None:
"""Can delete API key."""
await config_store.set_api_key("delete_me", "value")
deleted = await config_store.delete_api_key("delete_me")
assert deleted is True
value = await config_store.get_api_key("delete_me")
assert value is None
@pytest.mark.asyncio
async def test_delete_nonexistent(self, config_store: ConfigStore) -> None:
"""Deleting nonexistent key returns False."""
deleted = await config_store.delete_api_key("never_existed")
assert deleted is False
class TestNotifications:
"""Tests for LISTEN/NOTIFY functionality."""
@pytest.mark.asyncio
async def test_notify_on_adapter_change(self, config_store: ConfigStore) -> None:
"""NOTIFY fires when adapter is changed."""
notifications: list[tuple[str, str]] = []
notification_received = asyncio.Event()
async def callback(table: str, key: str) -> None:
notifications.append((table, key))
notification_received.set()
# Start listener in background
listen_task = asyncio.create_task(config_store.listen_for_changes(callback))
try:
# Give listener time to subscribe
await asyncio.sleep(0.1)
# Trigger a change
await config_store.upsert_adapter("notify_test", True, 60, {})
# Wait for notification (with timeout)
try:
await asyncio.wait_for(notification_received.wait(), timeout=5.0)
except asyncio.TimeoutError:
pytest.fail("Notification not received within timeout")
assert len(notifications) >= 1
assert notifications[0][0] == "adapters"
assert notifications[0][1] == "notify_test"
finally:
listen_task.cancel()
try:
await listen_task
except asyncio.CancelledError:
pass
@pytest.mark.asyncio
async def test_notify_on_api_key_change(self, config_store: ConfigStore) -> None:
"""NOTIFY fires when API key is changed."""
notifications: list[tuple[str, str]] = []
notification_received = asyncio.Event()
async def callback(table: str, key: str) -> None:
notifications.append((table, key))
notification_received.set()
listen_task = asyncio.create_task(config_store.listen_for_changes(callback))
try:
await asyncio.sleep(0.1)
await config_store.set_api_key("notify_key", "secret")
try:
await asyncio.wait_for(notification_received.wait(), timeout=5.0)
except asyncio.TimeoutError:
pytest.fail("Notification not received within timeout")
assert len(notifications) >= 1
assert notifications[0][0] == "api_keys"
assert notifications[0][1] == "notify_key"
finally:
listen_task.cancel()
try:
await listen_task
except asyncio.CancelledError:
pass
class TestListenerReconnect:
"""Tests for listener reconnection on connection loss."""
@pytest.mark.asyncio
async def test_listener_cancellation_propagates(
self, config_store: ConfigStore
) -> None:
"""Cancellation cleanly stops the listener without reconnect loop."""
async def callback(table: str, key: str) -> None:
pass
listen_task = asyncio.create_task(config_store.listen_for_changes(callback))
# Give listener time to start
await asyncio.sleep(0.1)
# Cancel and verify it stops
listen_task.cancel()
try:
await asyncio.wait_for(listen_task, timeout=2.0)
except asyncio.CancelledError:
pass # Expected
except asyncio.TimeoutError:
pytest.fail("Listener did not stop after cancellation")
assert listen_task.cancelled() or listen_task.done()
"""Tests for database-backed configuration store.
These tests require a real Postgres database. Set CENTRAL_TEST_DB_DSN
environment variable to override the default test database connection.
"""
import asyncio
import base64
import os
from pathlib import Path
import asyncpg
import pytest
import pytest_asyncio
from central.config_store import ConfigStore
from central.crypto import KEY_SIZE, clear_key_cache
# Test database DSN - uses central_test database with well-known test password.
# Override via CENTRAL_TEST_DB_DSN env var if your test DB differs.
TEST_DB_DSN = os.environ.get(
"CENTRAL_TEST_DB_DSN",
"postgresql://central_test:testpass@localhost/central_test",
)
@pytest.fixture(scope="session")
def master_key_path(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""Create a master key file for the test session."""
key = os.urandom(KEY_SIZE)
key_path = tmp_path_factory.mktemp("keys") / "master.key"
key_path.write_text(base64.b64encode(key).decode())
return key_path
@pytest.fixture(autouse=True)
def setup_master_key(master_key_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Configure master key path for all tests."""
clear_key_cache()
monkeypatch.setenv("CENTRAL_DB_DSN", TEST_DB_DSN)
monkeypatch.setenv("CENTRAL_MASTER_KEY_PATH", str(master_key_path))
@pytest_asyncio.fixture
async def db_conn() -> asyncpg.Connection:
"""Get a direct database connection for setup/teardown."""
conn = await asyncpg.connect(TEST_DB_DSN)
yield conn
await conn.close()
@pytest_asyncio.fixture
async def clean_config_schema(db_conn: asyncpg.Connection) -> None:
"""Ensure config schema exists and is clean before each test."""
# Create schema if not exists
await db_conn.execute("CREATE SCHEMA IF NOT EXISTS config")
# Create tables if not exist
await db_conn.execute("""
CREATE TABLE IF NOT EXISTS config.adapters (
name TEXT PRIMARY KEY,
enabled BOOLEAN NOT NULL DEFAULT true,
cadence_s INTEGER NOT NULL,
settings JSONB NOT NULL DEFAULT '{}'::jsonb,
paused_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
)
""")
await db_conn.execute("""
CREATE TABLE IF NOT EXISTS config.api_keys (
alias TEXT PRIMARY KEY,
encrypted_value BYTEA NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
rotated_at TIMESTAMPTZ,
last_used_at TIMESTAMPTZ
)
""")
# Create notify function with proper key detection
await db_conn.execute("""
CREATE OR REPLACE FUNCTION config.notify_config_change()
RETURNS trigger AS $$
DECLARE
key_value TEXT;
BEGIN
IF TG_TABLE_NAME = 'adapters' THEN
key_value := COALESCE(NEW.name, OLD.name, '');
ELSIF TG_TABLE_NAME = 'api_keys' THEN
key_value := COALESCE(NEW.alias, OLD.alias, '');
ELSE
key_value := '';
END IF;
PERFORM pg_notify('config_changed', TG_TABLE_NAME || ':' || key_value);
RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql
""")
# Create triggers if not exist
await db_conn.execute("""
DROP TRIGGER IF EXISTS adapters_notify ON config.adapters;
CREATE TRIGGER adapters_notify
AFTER INSERT OR UPDATE OR DELETE ON config.adapters
FOR EACH ROW EXECUTE FUNCTION config.notify_config_change()
""")
await db_conn.execute("""
DROP TRIGGER IF EXISTS api_keys_notify ON config.api_keys;
CREATE TRIGGER api_keys_notify
AFTER INSERT OR UPDATE OR DELETE ON config.api_keys
FOR EACH ROW EXECUTE FUNCTION config.notify_config_change()
""")
# Clean tables
await db_conn.execute("DELETE FROM config.adapters")
await db_conn.execute("DELETE FROM config.api_keys")
@pytest_asyncio.fixture
async def config_store(clean_config_schema: None) -> ConfigStore:
"""Create a ConfigStore connected to the test database."""
store = await ConfigStore.create(TEST_DB_DSN)
yield store
await store.close()
class TestAdapterConfig:
"""Tests for adapter configuration operations."""
@pytest.mark.asyncio
async def test_upsert_and_get(self, config_store: ConfigStore) -> None:
"""Can insert and retrieve adapter config."""
await config_store.upsert_adapter(
name="test_adapter",
enabled=True,
cadence_s=120,
settings={"key": "value"},
)
adapter = await config_store.get_adapter("test_adapter")
assert adapter is not None
assert adapter.name == "test_adapter"
assert adapter.enabled is True
assert adapter.cadence_s == 120
assert adapter.settings == {"key": "value"}
@pytest.mark.asyncio
async def test_get_nonexistent(self, config_store: ConfigStore) -> None:
"""Getting nonexistent adapter returns None."""
adapter = await config_store.get_adapter("does_not_exist")
assert adapter is None
@pytest.mark.asyncio
async def test_list_adapters(self, config_store: ConfigStore) -> None:
"""Can list all adapters."""
await config_store.upsert_adapter("adapter_a", True, 60, {})
await config_store.upsert_adapter("adapter_b", False, 300, {"x": 1})
adapters = await config_store.list_adapters()
assert len(adapters) == 2
names = [a.name for a in adapters]
assert "adapter_a" in names
assert "adapter_b" in names
@pytest.mark.asyncio
async def test_upsert_updates_existing(self, config_store: ConfigStore) -> None:
"""Upsert updates existing adapter."""
await config_store.upsert_adapter("updater", True, 60, {"v": 1})
await config_store.upsert_adapter("updater", False, 120, {"v": 2})
adapter = await config_store.get_adapter("updater")
assert adapter is not None
assert adapter.enabled is False
assert adapter.cadence_s == 120
assert adapter.settings == {"v": 2}
@pytest.mark.asyncio
async def test_pause_unpause(self, config_store: ConfigStore) -> None:
"""Can pause and unpause adapter."""
await config_store.upsert_adapter("pausable", True, 60, {})
await config_store.pause_adapter("pausable")
adapter = await config_store.get_adapter("pausable")
assert adapter is not None
assert adapter.is_paused is True
await config_store.unpause_adapter("pausable")
adapter = await config_store.get_adapter("pausable")
assert adapter is not None
assert adapter.is_paused is False
class TestApiKeys:
"""Tests for API key operations."""
@pytest.mark.asyncio
async def test_set_and_get_key(self, config_store: ConfigStore) -> None:
"""Can store and retrieve encrypted API key."""
await config_store.set_api_key("test_key", "super_secret_value")
value = await config_store.get_api_key("test_key")
assert value == "super_secret_value"
@pytest.mark.asyncio
async def test_get_nonexistent_key(self, config_store: ConfigStore) -> None:
"""Getting nonexistent key returns None."""
value = await config_store.get_api_key("does_not_exist")
assert value is None
@pytest.mark.asyncio
async def test_key_rotation(self, config_store: ConfigStore) -> None:
"""Updating key sets rotated_at."""
await config_store.set_api_key("rotate_me", "value1")
await config_store.set_api_key("rotate_me", "value2")
value = await config_store.get_api_key("rotate_me")
assert value == "value2"
@pytest.mark.asyncio
async def test_delete_key(self, config_store: ConfigStore) -> None:
"""Can delete API key."""
await config_store.set_api_key("delete_me", "value")
deleted = await config_store.delete_api_key("delete_me")
assert deleted is True
value = await config_store.get_api_key("delete_me")
assert value is None
@pytest.mark.asyncio
async def test_delete_nonexistent(self, config_store: ConfigStore) -> None:
"""Deleting nonexistent key returns False."""
deleted = await config_store.delete_api_key("never_existed")
assert deleted is False
class TestNotifications:
"""Tests for LISTEN/NOTIFY functionality."""
@pytest.mark.asyncio
async def test_notify_on_adapter_change(self, config_store: ConfigStore) -> None:
"""NOTIFY fires when adapter is changed."""
notifications: list[tuple[str, str]] = []
notification_received = asyncio.Event()
async def callback(table: str, key: str) -> None:
notifications.append((table, key))
notification_received.set()
# Start listener in background
listen_task = asyncio.create_task(config_store.listen_for_changes(callback))
try:
# Give listener time to subscribe
await asyncio.sleep(0.1)
# Trigger a change
await config_store.upsert_adapter("notify_test", True, 60, {})
# Wait for notification (with timeout)
try:
await asyncio.wait_for(notification_received.wait(), timeout=5.0)
except asyncio.TimeoutError:
pytest.fail("Notification not received within timeout")
assert len(notifications) >= 1
assert notifications[0][0] == "adapters"
assert notifications[0][1] == "notify_test"
finally:
listen_task.cancel()
try:
await listen_task
except asyncio.CancelledError:
pass
@pytest.mark.asyncio
async def test_notify_on_api_key_change(self, config_store: ConfigStore) -> None:
"""NOTIFY fires when API key is changed."""
notifications: list[tuple[str, str]] = []
notification_received = asyncio.Event()
async def callback(table: str, key: str) -> None:
notifications.append((table, key))
notification_received.set()
listen_task = asyncio.create_task(config_store.listen_for_changes(callback))
try:
await asyncio.sleep(0.1)
await config_store.set_api_key("notify_key", "secret")
try:
await asyncio.wait_for(notification_received.wait(), timeout=5.0)
except asyncio.TimeoutError:
pytest.fail("Notification not received within timeout")
assert len(notifications) >= 1
assert notifications[0][0] == "api_keys"
assert notifications[0][1] == "notify_key"
finally:
listen_task.cancel()
try:
await listen_task
except asyncio.CancelledError:
pass
class TestListenerReconnect:
"""Tests for listener reconnection on connection loss."""
@pytest.mark.asyncio
async def test_listener_cancellation_propagates(
self, config_store: ConfigStore
) -> None:
"""Cancellation cleanly stops the listener without reconnect loop."""
async def callback(table: str, key: str) -> None:
pass
listen_task = asyncio.create_task(config_store.listen_for_changes(callback))
# Give listener time to start
await asyncio.sleep(0.1)
# Cancel and verify it stops
listen_task.cancel()
try:
await asyncio.wait_for(listen_task, timeout=2.0)
except asyncio.CancelledError:
pass # Expected
except asyncio.TimeoutError:
pytest.fail("Listener did not stop after cancellation")
assert listen_task.cancelled() or listen_task.done()

350
tests/test_crypto.py
View file

@ -1,175 +1,175 @@
"""Tests for cryptographic primitives."""
import base64
import os
from pathlib import Path
import pytest
from central.crypto import (
KEY_SIZE,
DecryptionError,
KeyLoadError,
clear_key_cache,
decrypt,
encrypt,
)
@pytest.fixture
def master_key(tmp_path: Path) -> Path:
"""Create a valid master key file."""
key = os.urandom(KEY_SIZE)
key_path = tmp_path / "master.key"
key_path.write_text(base64.b64encode(key).decode())
clear_key_cache()
return key_path
@pytest.fixture
def wrong_key(tmp_path: Path) -> Path:
"""Create a different master key file."""
key = os.urandom(KEY_SIZE)
key_path = tmp_path / "wrong.key"
key_path.write_text(base64.b64encode(key).decode())
return key_path
class TestEncryptDecrypt:
"""Test encrypt/decrypt round-trip."""
def test_round_trip(self, master_key: Path) -> None:
"""Encrypting then decrypting returns original plaintext."""
plaintext = b"Hello, Central!"
ciphertext = encrypt(plaintext, key_path=master_key)
decrypted = decrypt(ciphertext, key_path=master_key)
assert decrypted == plaintext
def test_round_trip_empty(self, master_key: Path) -> None:
"""Empty plaintext encrypts and decrypts correctly."""
plaintext = b""
ciphertext = encrypt(plaintext, key_path=master_key)
decrypted = decrypt(ciphertext, key_path=master_key)
assert decrypted == plaintext
def test_round_trip_large(self, master_key: Path) -> None:
"""Large plaintext encrypts and decrypts correctly."""
plaintext = os.urandom(1024 * 1024) # 1MB
ciphertext = encrypt(plaintext, key_path=master_key)
decrypted = decrypt(ciphertext, key_path=master_key)
assert decrypted == plaintext
def test_ciphertext_different_each_time(self, master_key: Path) -> None:
"""Same plaintext produces different ciphertext (random nonce)."""
plaintext = b"test"
ct1 = encrypt(plaintext, key_path=master_key)
ct2 = encrypt(plaintext, key_path=master_key)
assert ct1 != ct2
# But both decrypt to same plaintext
assert decrypt(ct1, key_path=master_key) == plaintext
assert decrypt(ct2, key_path=master_key) == plaintext
class TestDecryptionFailures:
"""Test AEAD authentication catches tampering."""
def test_wrong_key_fails(self, master_key: Path, wrong_key: Path) -> None:
"""Decryption with wrong key raises DecryptionError."""
plaintext = b"secret"
ciphertext = encrypt(plaintext, key_path=master_key)
clear_key_cache() # Clear cache so wrong_key is loaded
with pytest.raises(DecryptionError):
decrypt(ciphertext, key_path=wrong_key)
def test_tampered_ciphertext_fails(self, master_key: Path) -> None:
"""Modified ciphertext is detected and rejected."""
plaintext = b"secret"
ciphertext = encrypt(plaintext, key_path=master_key)
# Flip a bit in the ciphertext (after nonce, before tag)
tampered = bytearray(ciphertext)
tampered[15] ^= 0x01 # Flip one bit
tampered = bytes(tampered)
with pytest.raises(DecryptionError):
decrypt(tampered, key_path=master_key)
def test_tampered_tag_fails(self, master_key: Path) -> None:
"""Modified authentication tag is detected and rejected."""
plaintext = b"secret"
ciphertext = encrypt(plaintext, key_path=master_key)
# Flip a bit in the last byte (part of the tag)
tampered = bytearray(ciphertext)
tampered[-1] ^= 0x01
tampered = bytes(tampered)
with pytest.raises(DecryptionError):
decrypt(tampered, key_path=master_key)
def test_truncated_ciphertext_fails(self, master_key: Path) -> None:
"""Truncated ciphertext is rejected."""
ciphertext = b"tooshort"
with pytest.raises(DecryptionError, match="too short"):
decrypt(ciphertext, key_path=master_key)
class TestKeyLoading:
"""Test master key loading."""
def test_missing_key_file(self, tmp_path: Path) -> None:
"""Missing key file raises KeyLoadError."""
clear_key_cache()
missing = tmp_path / "nonexistent.key"
with pytest.raises(KeyLoadError, match="not found"):
encrypt(b"test", key_path=missing)
def test_invalid_key_size(self, tmp_path: Path) -> None:
"""Key file with wrong size raises KeyLoadError."""
clear_key_cache()
bad_key = tmp_path / "bad.key"
bad_key.write_text(base64.b64encode(b"tooshort").decode())
with pytest.raises(KeyLoadError, match="Invalid master key size"):
encrypt(b"test", key_path=bad_key)
def test_invalid_base64(self, tmp_path: Path) -> None:
"""Invalid base64 in key file raises KeyLoadError."""
clear_key_cache()
bad_key = tmp_path / "bad.key"
bad_key.write_text("not valid base64!!!")
with pytest.raises(KeyLoadError):
encrypt(b"test", key_path=bad_key)
def test_key_cached(self, master_key: Path) -> None:
"""Key is cached after first load."""
# First encryption loads the key
encrypt(b"test1", key_path=master_key)
# Delete the file
master_key.unlink()
# Second encryption should still work (cached)
ciphertext = encrypt(b"test2", key_path=master_key)
assert len(ciphertext) > 0
def test_cache_clear(self, master_key: Path) -> None:
"""clear_key_cache forces reload."""
encrypt(b"test", key_path=master_key)
master_key.unlink()
clear_key_cache()
with pytest.raises(KeyLoadError, match="not found"):
encrypt(b"test", key_path=master_key)
"""Tests for cryptographic primitives."""
import base64
import os
from pathlib import Path
import pytest
from central.crypto import (
KEY_SIZE,
DecryptionError,
KeyLoadError,
clear_key_cache,
decrypt,
encrypt,
)
@pytest.fixture
def master_key(tmp_path: Path) -> Path:
"""Create a valid master key file."""
key = os.urandom(KEY_SIZE)
key_path = tmp_path / "master.key"
key_path.write_text(base64.b64encode(key).decode())
clear_key_cache()
return key_path
@pytest.fixture
def wrong_key(tmp_path: Path) -> Path:
"""Create a different master key file."""
key = os.urandom(KEY_SIZE)
key_path = tmp_path / "wrong.key"
key_path.write_text(base64.b64encode(key).decode())
return key_path
class TestEncryptDecrypt:
"""Test encrypt/decrypt round-trip."""
def test_round_trip(self, master_key: Path) -> None:
"""Encrypting then decrypting returns original plaintext."""
plaintext = b"Hello, Central!"
ciphertext = encrypt(plaintext, key_path=master_key)
decrypted = decrypt(ciphertext, key_path=master_key)
assert decrypted == plaintext
def test_round_trip_empty(self, master_key: Path) -> None:
"""Empty plaintext encrypts and decrypts correctly."""
plaintext = b""
ciphertext = encrypt(plaintext, key_path=master_key)
decrypted = decrypt(ciphertext, key_path=master_key)
assert decrypted == plaintext
def test_round_trip_large(self, master_key: Path) -> None:
"""Large plaintext encrypts and decrypts correctly."""
plaintext = os.urandom(1024 * 1024) # 1MB
ciphertext = encrypt(plaintext, key_path=master_key)
decrypted = decrypt(ciphertext, key_path=master_key)
assert decrypted == plaintext
def test_ciphertext_different_each_time(self, master_key: Path) -> None:
"""Same plaintext produces different ciphertext (random nonce)."""
plaintext = b"test"
ct1 = encrypt(plaintext, key_path=master_key)
ct2 = encrypt(plaintext, key_path=master_key)
assert ct1 != ct2
# But both decrypt to same plaintext
assert decrypt(ct1, key_path=master_key) == plaintext
assert decrypt(ct2, key_path=master_key) == plaintext
class TestDecryptionFailures:
"""Test AEAD authentication catches tampering."""
def test_wrong_key_fails(self, master_key: Path, wrong_key: Path) -> None:
"""Decryption with wrong key raises DecryptionError."""
plaintext = b"secret"
ciphertext = encrypt(plaintext, key_path=master_key)
clear_key_cache() # Clear cache so wrong_key is loaded
with pytest.raises(DecryptionError):
decrypt(ciphertext, key_path=wrong_key)
def test_tampered_ciphertext_fails(self, master_key: Path) -> None:
"""Modified ciphertext is detected and rejected."""
plaintext = b"secret"
ciphertext = encrypt(plaintext, key_path=master_key)
# Flip a bit in the ciphertext (after nonce, before tag)
tampered = bytearray(ciphertext)
tampered[15] ^= 0x01 # Flip one bit
tampered = bytes(tampered)
with pytest.raises(DecryptionError):
decrypt(tampered, key_path=master_key)
def test_tampered_tag_fails(self, master_key: Path) -> None:
"""Modified authentication tag is detected and rejected."""
plaintext = b"secret"
ciphertext = encrypt(plaintext, key_path=master_key)
# Flip a bit in the last byte (part of the tag)
tampered = bytearray(ciphertext)
tampered[-1] ^= 0x01
tampered = bytes(tampered)
with pytest.raises(DecryptionError):
decrypt(tampered, key_path=master_key)
def test_truncated_ciphertext_fails(self, master_key: Path) -> None:
"""Truncated ciphertext is rejected."""
ciphertext = b"tooshort"
with pytest.raises(DecryptionError, match="too short"):
decrypt(ciphertext, key_path=master_key)
class TestKeyLoading:
"""Test master key loading."""
def test_missing_key_file(self, tmp_path: Path) -> None:
"""Missing key file raises KeyLoadError."""
clear_key_cache()
missing = tmp_path / "nonexistent.key"
with pytest.raises(KeyLoadError, match="not found"):
encrypt(b"test", key_path=missing)
def test_invalid_key_size(self, tmp_path: Path) -> None:
"""Key file with wrong size raises KeyLoadError."""
clear_key_cache()
bad_key = tmp_path / "bad.key"
bad_key.write_text(base64.b64encode(b"tooshort").decode())
with pytest.raises(KeyLoadError, match="Invalid master key size"):
encrypt(b"test", key_path=bad_key)
def test_invalid_base64(self, tmp_path: Path) -> None:
"""Invalid base64 in key file raises KeyLoadError."""
clear_key_cache()
bad_key = tmp_path / "bad.key"
bad_key.write_text("not valid base64!!!")
with pytest.raises(KeyLoadError):
encrypt(b"test", key_path=bad_key)
def test_key_cached(self, master_key: Path) -> None:
"""Key is cached after first load."""
# First encryption loads the key
encrypt(b"test1", key_path=master_key)
# Delete the file
master_key.unlink()
# Second encryption should still work (cached)
ciphertext = encrypt(b"test2", key_path=master_key)
assert len(ciphertext) > 0
def test_cache_clear(self, master_key: Path) -> None:
"""clear_key_cache forces reload."""
encrypt(b"test", key_path=master_key)
master_key.unlink()
clear_key_cache()
with pytest.raises(KeyLoadError, match="not found"):
encrypt(b"test", key_path=master_key)

322
tests/test_models.py
View file

@ -1,161 +1,161 @@
"""Smoke tests for Central models and CloudEvents wire format."""
from datetime import datetime, timezone
import pytest
from central.models import Event, Geo, subject_for_event
from central.config import NWSAdapterConfig, CloudEventsConfig, NATSConfig, PostgresConfig, Config
from central.cloudevents_wire import wrap_event
@pytest.fixture
def sample_geo() -> Geo:
"""Sample Geo object for testing."""
return Geo(
centroid=(-116.2, 43.6),
bbox=(-116.5, 43.4, -115.9, 43.8),
regions=["US-ID-Ada", "US-ID-Canyon"],
primary_region="US-ID-Ada",
)
@pytest.fixture
def sample_event(sample_geo: Geo) -> Event:
"""Sample Event object for testing."""
return Event(
id="urn:central:nws:alert:KBOI-202401151200-SVR",
source="central/adapters/nws",
category="wx.alert.severe_thunderstorm_warning",
time=datetime(2024, 1, 15, 12, 0, 0, tzinfo=timezone.utc),
expires=datetime(2024, 1, 15, 13, 0, 0, tzinfo=timezone.utc),
severity=3,
geo=sample_geo,
data={"headline": "Severe Thunderstorm Warning", "urgency": "Immediate"},
)
@pytest.fixture
def sample_config() -> Config:
"""Sample Config object for testing."""
return Config(
adapters={
"nws": NWSAdapterConfig(
enabled=True,
cadence_s=60,
states=["ID", "MT"],
contact_email="test@example.com",
)
},
cloudevents=CloudEventsConfig(
type_prefix="central",
source="central.local",
schema_version="1.0",
),
nats=NATSConfig(url="nats://localhost:4222"),
postgres=PostgresConfig(dsn="postgresql://user:pass@localhost/db"),
)
class TestSubjectForEvent:
"""Tests for subject_for_event helper."""
def test_county_subject(self, sample_event: Event) -> None:
"""County codes produce county subject."""
subject = subject_for_event(sample_event)
assert subject == "central.wx.alert.us.id.county.ada"
def test_zone_subject(self, sample_geo: Geo) -> None:
"""Zone codes produce zone subject."""
geo = Geo(
centroid=sample_geo.centroid,
bbox=sample_geo.bbox,
regions=["US-ID-Z033"],
primary_region="US-ID-Z033",
)
event = Event(
id="test-zone",
source="test",
category="wx.alert.winter_storm_warning",
time=datetime(2024, 1, 15, 12, 0, 0, tzinfo=timezone.utc),
geo=geo,
data={},
)
subject = subject_for_event(event)
assert subject == "central.wx.alert.us.id.zone.z033"
def test_unknown_subject(self, sample_event: Event) -> None:
"""Missing primary_region produces unknown subject."""
geo = Geo(regions=[], primary_region=None)
event = Event(
id="test-unknown",
source="test",
category="wx.alert.test",
time=datetime(2024, 1, 15, 12, 0, 0, tzinfo=timezone.utc),
geo=geo,
data={},
)
subject = subject_for_event(event)
assert subject == "central.wx.alert.us.unknown"
def test_custom_prefix(self, sample_event: Event) -> None:
"""Custom prefix is used in subject."""
subject = subject_for_event(sample_event, prefix="myapp.events")
assert subject == "myapp.events.alert.us.id.county.ada"
class TestCloudEventsWire:
"""Tests for CloudEvents wire format."""
def test_required_fields_present(
self, sample_event: Event, sample_config: Config
) -> None:
"""Required CloudEvents fields are present."""
envelope, msg_id = wrap_event(sample_event, sample_config)
assert msg_id == sample_event.id
assert envelope["id"] == sample_event.id
assert envelope["source"] == sample_config.cloudevents.source
assert envelope["type"] == "central.wx.alert.severe_thunderstorm_warning.v1"
assert envelope["specversion"] == "1.0"
assert "time" in envelope
assert envelope["datacontenttype"] == "application/json"
assert "data" in envelope
def test_extension_attributes_lowercase(
self, sample_event: Event, sample_config: Config
) -> None:
"""Extension attributes are lowercase with no underscores."""
envelope, _ = wrap_event(sample_event, sample_config)
# Check that extension attributes exist and are lowercase
assert envelope["centralschemaversion"] == "1.0"
assert envelope["centralcategory"] == "wx.alert.severe_thunderstorm_warning"
assert envelope["centralseverity"] == 3
# Verify no uppercase or underscores in extension names
for key in ["centralschemaversion", "centralcategory", "centralseverity"]:
assert key.islower()
assert "_" not in key
def test_severity_none_omits_centralseverity(
self, sample_geo: Geo, sample_config: Config
) -> None:
"""When severity is None, centralseverity is omitted entirely."""
event = Event(
id="test-no-severity",
source="test",
category="wx.alert.test",
time=datetime(2024, 1, 15, 12, 0, 0, tzinfo=timezone.utc),
severity=None, # Explicitly None
geo=sample_geo,
data={},
)
envelope, _ = wrap_event(event, sample_config)
# centralseverity should not be present at all
assert "centralseverity" not in envelope
# Other extensions should still be present
assert "centralschemaversion" in envelope
assert "centralcategory" in envelope
"""Smoke tests for Central models and CloudEvents wire format."""
from datetime import datetime, timezone
import pytest
from central.models import Event, Geo, subject_for_event
from central.config import NWSAdapterConfig, CloudEventsConfig, NATSConfig, PostgresConfig, Config
from central.cloudevents_wire import wrap_event
@pytest.fixture
def sample_geo() -> Geo:
"""Sample Geo object for testing."""
return Geo(
centroid=(-116.2, 43.6),
bbox=(-116.5, 43.4, -115.9, 43.8),
regions=["US-ID-Ada", "US-ID-Canyon"],
primary_region="US-ID-Ada",
)
@pytest.fixture
def sample_event(sample_geo: Geo) -> Event:
"""Sample Event object for testing."""
return Event(
id="urn:central:nws:alert:KBOI-202401151200-SVR",
source="central/adapters/nws",
category="wx.alert.severe_thunderstorm_warning",
time=datetime(2024, 1, 15, 12, 0, 0, tzinfo=timezone.utc),
expires=datetime(2024, 1, 15, 13, 0, 0, tzinfo=timezone.utc),
severity=3,
geo=sample_geo,
data={"headline": "Severe Thunderstorm Warning", "urgency": "Immediate"},
)
@pytest.fixture
def sample_config() -> Config:
"""Sample Config object for testing."""
return Config(
adapters={
"nws": NWSAdapterConfig(
enabled=True,
cadence_s=60,
states=["ID", "MT"],
contact_email="test@example.com",
)
},
cloudevents=CloudEventsConfig(
type_prefix="central",
source="central.local",
schema_version="1.0",
),
nats=NATSConfig(url="nats://localhost:4222"),
postgres=PostgresConfig(dsn="postgresql://user:pass@localhost/db"),
)
class TestSubjectForEvent:
"""Tests for subject_for_event helper."""
def test_county_subject(self, sample_event: Event) -> None:
"""County codes produce county subject."""
subject = subject_for_event(sample_event)
assert subject == "central.wx.alert.us.id.county.ada"
def test_zone_subject(self, sample_geo: Geo) -> None:
"""Zone codes produce zone subject."""
geo = Geo(
centroid=sample_geo.centroid,
bbox=sample_geo.bbox,
regions=["US-ID-Z033"],
primary_region="US-ID-Z033",
)
event = Event(
id="test-zone",
source="test",
category="wx.alert.winter_storm_warning",
time=datetime(2024, 1, 15, 12, 0, 0, tzinfo=timezone.utc),
geo=geo,
data={},
)
subject = subject_for_event(event)
assert subject == "central.wx.alert.us.id.zone.z033"
def test_unknown_subject(self, sample_event: Event) -> None:
"""Missing primary_region produces unknown subject."""
geo = Geo(regions=[], primary_region=None)
event = Event(
id="test-unknown",
source="test",
category="wx.alert.test",
time=datetime(2024, 1, 15, 12, 0, 0, tzinfo=timezone.utc),
geo=geo,
data={},
)
subject = subject_for_event(event)
assert subject == "central.wx.alert.us.unknown"
def test_custom_prefix(self, sample_event: Event) -> None:
"""Custom prefix is used in subject."""
subject = subject_for_event(sample_event, prefix="myapp.events")
assert subject == "myapp.events.alert.us.id.county.ada"
class TestCloudEventsWire:
"""Tests for CloudEvents wire format."""
def test_required_fields_present(
self, sample_event: Event, sample_config: Config
) -> None:
"""Required CloudEvents fields are present."""
envelope, msg_id = wrap_event(sample_event, sample_config)
assert msg_id == sample_event.id
assert envelope["id"] == sample_event.id
assert envelope["source"] == sample_config.cloudevents.source
assert envelope["type"] == "central.wx.alert.severe_thunderstorm_warning.v1"
assert envelope["specversion"] == "1.0"
assert "time" in envelope
assert envelope["datacontenttype"] == "application/json"
assert "data" in envelope
def test_extension_attributes_lowercase(
self, sample_event: Event, sample_config: Config
) -> None:
"""Extension attributes are lowercase with no underscores."""
envelope, _ = wrap_event(sample_event, sample_config)
# Check that extension attributes exist and are lowercase
assert envelope["centralschemaversion"] == "1.0"
assert envelope["centralcategory"] == "wx.alert.severe_thunderstorm_warning"
assert envelope["centralseverity"] == 3
# Verify no uppercase or underscores in extension names
for key in ["centralschemaversion", "centralcategory", "centralseverity"]:
assert key.islower()
assert "_" not in key
def test_severity_none_omits_centralseverity(
self, sample_geo: Geo, sample_config: Config
) -> None:
"""When severity is None, centralseverity is omitted entirely."""
event = Event(
id="test-no-severity",
source="test",
category="wx.alert.test",
time=datetime(2024, 1, 15, 12, 0, 0, tzinfo=timezone.utc),
severity=None, # Explicitly None
geo=sample_geo,
data={},
)
envelope, _ = wrap_event(event, sample_config)
# centralseverity should not be present at all
assert "centralseverity" not in envelope
# Other extensions should still be present
assert "centralschemaversion" in envelope
assert "centralcategory" in envelope

714
tests/test_supervisor_hotreload.py
View file

@ -1,357 +1,357 @@
"""Tests for supervisor hot-reload and rate-limiting behavior."""
import asyncio
import base64
import os
from datetime import datetime, timedelta, timezone
from pathlib import Path
from unittest.mock import AsyncMock, MagicMock, patch
import asyncpg
import pytest
import pytest_asyncio
from central.config_models import AdapterConfig
from central.config_source import DbConfigSource
from central.config_store import ConfigStore
from central.crypto import KEY_SIZE, clear_key_cache
# Test database DSN
TEST_DB_DSN = os.environ.get(
"CENTRAL_TEST_DB_DSN",
"postgresql://central_test:testpass@localhost/central_test",
)
@pytest.fixture(scope="session")
def master_key_path(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""Create a master key file for the test session."""
key = os.urandom(KEY_SIZE)
key_path = tmp_path_factory.mktemp("keys") / "master.key"
key_path.write_text(base64.b64encode(key).decode())
return key_path
@pytest.fixture(autouse=True)
def setup_master_key(master_key_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Configure master key path for all tests."""
clear_key_cache()
monkeypatch.setenv("CENTRAL_DB_DSN", TEST_DB_DSN)
monkeypatch.setenv("CENTRAL_MASTER_KEY_PATH", str(master_key_path))
@pytest_asyncio.fixture
async def db_conn() -> asyncpg.Connection:
"""Get a direct database connection for setup/teardown."""
conn = await asyncpg.connect(TEST_DB_DSN)
yield conn
await conn.close()
@pytest_asyncio.fixture
async def clean_config_schema(db_conn: asyncpg.Connection) -> None:
"""Ensure config schema exists and is clean before each test."""
await db_conn.execute("CREATE SCHEMA IF NOT EXISTS config")
await db_conn.execute("""
CREATE TABLE IF NOT EXISTS config.adapters (
name TEXT PRIMARY KEY,
enabled BOOLEAN NOT NULL DEFAULT true,
cadence_s INTEGER NOT NULL,
settings JSONB NOT NULL DEFAULT '{}'::jsonb,
paused_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
)
""")
# Create notify trigger
await db_conn.execute("""
CREATE OR REPLACE FUNCTION config.notify_config_change()
RETURNS trigger AS $$
DECLARE
key_value TEXT;
BEGIN
IF TG_TABLE_NAME = 'adapters' THEN
key_value := COALESCE(NEW.name, OLD.name, '');
ELSE
key_value := '';
END IF;
PERFORM pg_notify('config_changed', TG_TABLE_NAME || ':' || key_value);
RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql
""")
await db_conn.execute("""
DROP TRIGGER IF EXISTS adapters_notify ON config.adapters;
CREATE TRIGGER adapters_notify
AFTER INSERT OR UPDATE OR DELETE ON config.adapters
FOR EACH ROW EXECUTE FUNCTION config.notify_config_change()
""")
await db_conn.execute("DELETE FROM config.adapters")
@pytest_asyncio.fixture
async def config_store(clean_config_schema: None) -> ConfigStore:
"""Create a ConfigStore connected to the test database."""
store = await ConfigStore.create(TEST_DB_DSN)
yield store
await store.close()
class TestDbConfigSourceNotifications:
"""Tests for DbConfigSource NOTIFY integration."""
@pytest.mark.asyncio
async def test_watch_receives_notifications(
self,
config_store: ConfigStore,
db_conn: asyncpg.Connection,
) -> None:
"""watch_for_changes receives NOTIFY when adapter changes."""
source = DbConfigSource(config_store)
notifications: list[tuple[str, str]] = []
notification_received = asyncio.Event()
async def callback(table: str, key: str) -> None:
notifications.append((table, key))
notification_received.set()
# Start watching in background
watch_task = asyncio.create_task(source.watch_for_changes(callback))
try:
# Wait for listener to connect
await asyncio.sleep(0.2)
# Insert an adapter via direct connection (not through store)
# This triggers the NOTIFY
await db_conn.execute("""
INSERT INTO config.adapters (name, enabled, cadence_s, settings)
VALUES ('test_adapter', true, 60, '{}'::jsonb)
""")
# Wait for notification
await asyncio.wait_for(notification_received.wait(), timeout=5.0)
assert len(notifications) >= 1
assert notifications[0] == ("adapters", "test_adapter")
finally:
watch_task.cancel()
try:
await watch_task
except asyncio.CancelledError:
pass
class TestRateLimitGuarantee:
"""Tests for rate-limit guarantees during hot-reload.
These tests verify the critical invariant: cadence changes must not
cause extra API calls before (last_poll + new_cadence).
"""
@pytest.mark.asyncio
async def test_cadence_change_respects_last_poll_time(self) -> None:
"""Changing cadence mid-cycle schedules next poll at last_poll + new_cadence.
This is the core rate-limit guarantee test (gate 3).
"""
# Import supervisor module to access AdapterState
from central.supervisor import AdapterState
# Mock adapter
mock_adapter = MagicMock()
mock_adapter.name = "test"
mock_adapter.cadence_s = 60
# Create adapter state with a known last_completed_poll time
last_poll = datetime.now(timezone.utc) - timedelta(seconds=30)
config = AdapterConfig(
name="test",
enabled=True,
cadence_s=60, # Original cadence
settings={},
paused_at=None,
updated_at=datetime.now(timezone.utc),
)
state = AdapterState(
name="test",
adapter=mock_adapter,
config=config,
last_completed_poll=last_poll,
)
# Simulate cadence change to 90 seconds
new_config = AdapterConfig(
name="test",
enabled=True,
cadence_s=90, # New cadence
settings={},
paused_at=None,
updated_at=datetime.now(timezone.utc),
)
# Update state as reschedule would
state.config = new_config
state.adapter.cadence_s = 90
# Calculate expected next poll time
expected_next_poll = last_poll + timedelta(seconds=90)
now = datetime.now(timezone.utc)
expected_wait = max(0, (expected_next_poll - now).total_seconds())
# The wait time should be based on last_poll + new_cadence
# Since last_poll was 30 seconds ago and new cadence is 90,
# we should wait 60 more seconds (90 - 30 = 60)
actual_next_poll = last_poll.timestamp() + new_config.cadence_s
actual_wait = max(0, actual_next_poll - now.timestamp())
# Allow 1 second tolerance for timing
assert abs(actual_wait - 60) < 2, (
f"Expected ~60s wait, got {actual_wait}s. "
f"Rate limit violated: poll would happen before last_poll + new_cadence"
)
@pytest.mark.asyncio
async def test_cadence_increase_after_gap_polls_immediately(self) -> None:
"""When last_poll + new_cadence is already past, poll immediately.
If operator increases cadence to 120s after a gap of 150s,
the poll should happen now (not wait another 120s).
"""
from central.supervisor import AdapterState
mock_adapter = MagicMock()
mock_adapter.name = "test"
mock_adapter.cadence_s = 60
# Last poll was 150 seconds ago
last_poll = datetime.now(timezone.utc) - timedelta(seconds=150)
config = AdapterConfig(
name="test",
enabled=True,
cadence_s=120, # Increased cadence
settings={},
paused_at=None,
updated_at=datetime.now(timezone.utc),
)
state = AdapterState(
name="test",
adapter=mock_adapter,
config=config,
last_completed_poll=last_poll,
)
# Calculate next poll time
now = datetime.now(timezone.utc)
next_poll_at = last_poll.timestamp() + config.cadence_s
wait_time = max(0, next_poll_at - now.timestamp())
# Since 150 > 120, next poll should be immediate (wait_time ~= 0)
assert wait_time < 1, (
f"Expected immediate poll (wait ~0s), got {wait_time}s. "
f"After a gap exceeding new cadence, poll should happen now."
)
@pytest.mark.asyncio
async def test_enable_disable_enable_respects_rate_limit(self) -> None:
"""Re-enabling adapter schedules poll at last_poll + cadence.
If adapter was disabled for a while and then re-enabled, the next
poll should be at (last_completed_poll + cadence_s), not immediately
(unless that time has already passed).
"""
from central.supervisor import AdapterState
mock_adapter = MagicMock()
mock_adapter.name = "test"
mock_adapter.cadence_s = 60
# Last poll was 30 seconds ago, then adapter was disabled
last_poll = datetime.now(timezone.utc) - timedelta(seconds=30)
# Re-enabled config
config = AdapterConfig(
name="test",
enabled=True,
cadence_s=60,
settings={},
paused_at=None,
updated_at=datetime.now(timezone.utc),
)
state = AdapterState(
name="test",
adapter=mock_adapter,
config=config,
last_completed_poll=last_poll,
)
# Calculate next poll time
now = datetime.now(timezone.utc)
next_poll_at = last_poll.timestamp() + config.cadence_s
wait_time = max(0, next_poll_at - now.timestamp())
# Should wait ~30 more seconds (60 - 30 = 30)
assert abs(wait_time - 30) < 2, (
f"Expected ~30s wait after re-enable, got {wait_time}s. "
f"Rate limit violated on enable→disable→enable sequence."
)
@pytest.mark.asyncio
async def test_multiple_rapid_cadence_changes_no_extra_polls(self) -> None:
"""Multiple rapid cadence changes don't cause extra polls.
If NOTIFY fires rapidly (60→90→120→90), the final schedule should
still be based on last_completed_poll + final_cadence.
"""
from central.supervisor import AdapterState
mock_adapter = MagicMock()
mock_adapter.name = "test"
mock_adapter.cadence_s = 60
# Last poll was 20 seconds ago
last_poll = datetime.now(timezone.utc) - timedelta(seconds=20)
state = AdapterState(
name="test",
adapter=mock_adapter,
config=AdapterConfig(
name="test",
enabled=True,
cadence_s=60,
settings={},
paused_at=None,
updated_at=datetime.now(timezone.utc),
),
last_completed_poll=last_poll,
)
# Simulate rapid cadence changes
for cadence in [90, 120, 90]: # Final cadence is 90
state.config = AdapterConfig(
name="test",
enabled=True,
cadence_s=cadence,
settings={},
paused_at=None,
updated_at=datetime.now(timezone.utc),
)
state.adapter.cadence_s = cadence
# Final schedule should be last_poll + 90
now = datetime.now(timezone.utc)
final_cadence = 90
next_poll_at = last_poll.timestamp() + final_cadence
wait_time = max(0, next_poll_at - now.timestamp())
# Should wait ~70 seconds (90 - 20 = 70)
assert abs(wait_time - 70) < 2, (
f"Expected ~70s wait after rapid changes, got {wait_time}s. "
f"Multiple NOTIFYs should not cause extra polls."
)
"""Tests for supervisor hot-reload and rate-limiting behavior."""
import asyncio
import base64
import os
from datetime import datetime, timedelta, timezone
from pathlib import Path
from unittest.mock import AsyncMock, MagicMock, patch
import asyncpg
import pytest
import pytest_asyncio
from central.config_models import AdapterConfig
from central.config_source import DbConfigSource
from central.config_store import ConfigStore
from central.crypto import KEY_SIZE, clear_key_cache
# Test database DSN
TEST_DB_DSN = os.environ.get(
"CENTRAL_TEST_DB_DSN",
"postgresql://central_test:testpass@localhost/central_test",
)
@pytest.fixture(scope="session")
def master_key_path(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""Create a master key file for the test session."""
key = os.urandom(KEY_SIZE)
key_path = tmp_path_factory.mktemp("keys") / "master.key"
key_path.write_text(base64.b64encode(key).decode())
return key_path
@pytest.fixture(autouse=True)
def setup_master_key(master_key_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Configure master key path for all tests."""
clear_key_cache()
monkeypatch.setenv("CENTRAL_DB_DSN", TEST_DB_DSN)
monkeypatch.setenv("CENTRAL_MASTER_KEY_PATH", str(master_key_path))
@pytest_asyncio.fixture
async def db_conn() -> asyncpg.Connection:
"""Get a direct database connection for setup/teardown."""
conn = await asyncpg.connect(TEST_DB_DSN)
yield conn
await conn.close()
@pytest_asyncio.fixture
async def clean_config_schema(db_conn: asyncpg.Connection) -> None:
"""Ensure config schema exists and is clean before each test."""
await db_conn.execute("CREATE SCHEMA IF NOT EXISTS config")
await db_conn.execute("""
CREATE TABLE IF NOT EXISTS config.adapters (
name TEXT PRIMARY KEY,
enabled BOOLEAN NOT NULL DEFAULT true,
cadence_s INTEGER NOT NULL,
settings JSONB NOT NULL DEFAULT '{}'::jsonb,
paused_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
)
""")
# Create notify trigger
await db_conn.execute("""
CREATE OR REPLACE FUNCTION config.notify_config_change()
RETURNS trigger AS $$
DECLARE
key_value TEXT;
BEGIN
IF TG_TABLE_NAME = 'adapters' THEN
key_value := COALESCE(NEW.name, OLD.name, '');
ELSE
key_value := '';
END IF;
PERFORM pg_notify('config_changed', TG_TABLE_NAME || ':' || key_value);
RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql
""")
await db_conn.execute("""
DROP TRIGGER IF EXISTS adapters_notify ON config.adapters;
CREATE TRIGGER adapters_notify
AFTER INSERT OR UPDATE OR DELETE ON config.adapters
FOR EACH ROW EXECUTE FUNCTION config.notify_config_change()
""")
await db_conn.execute("DELETE FROM config.adapters")
@pytest_asyncio.fixture
async def config_store(clean_config_schema: None) -> ConfigStore:
"""Create a ConfigStore connected to the test database."""
store = await ConfigStore.create(TEST_DB_DSN)
yield store
await store.close()
class TestDbConfigSourceNotifications:
"""Tests for DbConfigSource NOTIFY integration."""
@pytest.mark.asyncio
async def test_watch_receives_notifications(
self,
config_store: ConfigStore,
db_conn: asyncpg.Connection,
) -> None:
"""watch_for_changes receives NOTIFY when adapter changes."""
source = DbConfigSource(config_store)
notifications: list[tuple[str, str]] = []
notification_received = asyncio.Event()
async def callback(table: str, key: str) -> None:
notifications.append((table, key))
notification_received.set()
# Start watching in background
watch_task = asyncio.create_task(source.watch_for_changes(callback))
try:
# Wait for listener to connect
await asyncio.sleep(0.2)
# Insert an adapter via direct connection (not through store)
# This triggers the NOTIFY
await db_conn.execute("""
INSERT INTO config.adapters (name, enabled, cadence_s, settings)
VALUES ('test_adapter', true, 60, '{}'::jsonb)
""")
# Wait for notification
await asyncio.wait_for(notification_received.wait(), timeout=5.0)
assert len(notifications) >= 1
assert notifications[0] == ("adapters", "test_adapter")
finally:
watch_task.cancel()
try:
await watch_task
except asyncio.CancelledError:
pass
class TestRateLimitGuarantee:
"""Tests for rate-limit guarantees during hot-reload.
These tests verify the critical invariant: cadence changes must not
cause extra API calls before (last_poll + new_cadence).
"""
@pytest.mark.asyncio
async def test_cadence_change_respects_last_poll_time(self) -> None:
"""Changing cadence mid-cycle schedules next poll at last_poll + new_cadence.
This is the core rate-limit guarantee test (gate 3).
"""
# Import supervisor module to access AdapterState
from central.supervisor import AdapterState
# Mock adapter
mock_adapter = MagicMock()
mock_adapter.name = "test"
mock_adapter.cadence_s = 60
# Create adapter state with a known last_completed_poll time
last_poll = datetime.now(timezone.utc) - timedelta(seconds=30)
config = AdapterConfig(
name="test",
enabled=True,
cadence_s=60, # Original cadence
settings={},
paused_at=None,
updated_at=datetime.now(timezone.utc),
)
state = AdapterState(
name="test",
adapter=mock_adapter,
config=config,
last_completed_poll=last_poll,
)
# Simulate cadence change to 90 seconds
new_config = AdapterConfig(
name="test",
enabled=True,
cadence_s=90, # New cadence
settings={},
paused_at=None,
updated_at=datetime.now(timezone.utc),
)
# Update state as reschedule would
state.config = new_config
state.adapter.cadence_s = 90
# Calculate expected next poll time
expected_next_poll = last_poll + timedelta(seconds=90)
now = datetime.now(timezone.utc)
expected_wait = max(0, (expected_next_poll - now).total_seconds())
# The wait time should be based on last_poll + new_cadence
# Since last_poll was 30 seconds ago and new cadence is 90,
# we should wait 60 more seconds (90 - 30 = 60)
actual_next_poll = last_poll.timestamp() + new_config.cadence_s
actual_wait = max(0, actual_next_poll - now.timestamp())
# Allow 1 second tolerance for timing
assert abs(actual_wait - 60) < 2, (
f"Expected ~60s wait, got {actual_wait}s. "
f"Rate limit violated: poll would happen before last_poll + new_cadence"
)
@pytest.mark.asyncio
async def test_cadence_increase_after_gap_polls_immediately(self) -> None:
"""When last_poll + new_cadence is already past, poll immediately.
If operator increases cadence to 120s after a gap of 150s,
the poll should happen now (not wait another 120s).
"""
from central.supervisor import AdapterState
mock_adapter = MagicMock()
mock_adapter.name = "test"
mock_adapter.cadence_s = 60
# Last poll was 150 seconds ago
last_poll = datetime.now(timezone.utc) - timedelta(seconds=150)
config = AdapterConfig(
name="test",
enabled=True,
cadence_s=120, # Increased cadence
settings={},
paused_at=None,
updated_at=datetime.now(timezone.utc),
)
state = AdapterState(
name="test",
adapter=mock_adapter,
config=config,
last_completed_poll=last_poll,
)
# Calculate next poll time
now = datetime.now(timezone.utc)
next_poll_at = last_poll.timestamp() + config.cadence_s
wait_time = max(0, next_poll_at - now.timestamp())
# Since 150 > 120, next poll should be immediate (wait_time ~= 0)
assert wait_time < 1, (
f"Expected immediate poll (wait ~0s), got {wait_time}s. "
f"After a gap exceeding new cadence, poll should happen now."
)
@pytest.mark.asyncio
async def test_enable_disable_enable_respects_rate_limit(self) -> None:
"""Re-enabling adapter schedules poll at last_poll + cadence.
If adapter was disabled for a while and then re-enabled, the next
poll should be at (last_completed_poll + cadence_s), not immediately
(unless that time has already passed).
"""
from central.supervisor import AdapterState
mock_adapter = MagicMock()
mock_adapter.name = "test"
mock_adapter.cadence_s = 60
# Last poll was 30 seconds ago, then adapter was disabled
last_poll = datetime.now(timezone.utc) - timedelta(seconds=30)
# Re-enabled config
config = AdapterConfig(
name="test",
enabled=True,
cadence_s=60,
settings={},
paused_at=None,
updated_at=datetime.now(timezone.utc),
)
state = AdapterState(
name="test",
adapter=mock_adapter,
config=config,
last_completed_poll=last_poll,
)
# Calculate next poll time
now = datetime.now(timezone.utc)
next_poll_at = last_poll.timestamp() + config.cadence_s
wait_time = max(0, next_poll_at - now.timestamp())
# Should wait ~30 more seconds (60 - 30 = 30)
assert abs(wait_time - 30) < 2, (
f"Expected ~30s wait after re-enable, got {wait_time}s. "
f"Rate limit violated on enable→disable→enable sequence."
)
@pytest.mark.asyncio
async def test_multiple_rapid_cadence_changes_no_extra_polls(self) -> None:
"""Multiple rapid cadence changes don't cause extra polls.
If NOTIFY fires rapidly (60→90→120→90), the final schedule should
still be based on last_completed_poll + final_cadence.
"""
from central.supervisor import AdapterState
mock_adapter = MagicMock()
mock_adapter.name = "test"
mock_adapter.cadence_s = 60
# Last poll was 20 seconds ago
last_poll = datetime.now(timezone.utc) - timedelta(seconds=20)
state = AdapterState(
name="test",
adapter=mock_adapter,
config=AdapterConfig(
name="test",
enabled=True,
cadence_s=60,
settings={},
paused_at=None,
updated_at=datetime.now(timezone.utc),
),
last_completed_poll=last_poll,
)
# Simulate rapid cadence changes
for cadence in [90, 120, 90]: # Final cadence is 90
state.config = AdapterConfig(
name="test",
enabled=True,
cadence_s=cadence,
settings={},
paused_at=None,
updated_at=datetime.now(timezone.utc),
)
state.adapter.cadence_s = cadence
# Final schedule should be last_poll + 90
now = datetime.now(timezone.utc)
final_cadence = 90
next_poll_at = last_poll.timestamp() + final_cadence
wait_time = max(0, next_poll_at - now.timestamp())
# Should wait ~70 seconds (90 - 20 = 70)
assert abs(wait_time - 70) < 2, (
f"Expected ~70s wait after rapid changes, got {wait_time}s. "
f"Multiple NOTIFYs should not cause extra polls."
)

1092
tests/test_supervisor_integration.py
View file

File diff suppressed because it is too large Load diff

964
tests/test_usgs_quake.py
View file

@ -1,482 +1,482 @@
"""Tests for USGS earthquake adapter."""
import pytest
from datetime import datetime, timezone
from unittest.mock import AsyncMock, MagicMock, patch
from pathlib import Path
import tempfile
from central.adapters.usgs_quake import (
USGSQuakeAdapter,
magnitude_tier,
magnitude_to_severity,
)
from central.config_models import AdapterConfig, RegionConfig
from central.models import Event, Geo
# Sample USGS GeoJSON response
SAMPLE_GEOJSON = {
"type": "FeatureCollection",
"metadata": {
"generated": 1715878800000,
"url": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_hour.geojson",
"title": "USGS All Earthquakes, Past Hour",
"status": 200,
"api": "1.10.3",
"count": 3
},
"features": [
{
"type": "Feature",
"properties": {
"mag": 2.5,
"place": "10km N of Boise, Idaho",
"time": 1715878500000,
"updated": 1715878600000,
"tz": None,
"url": "https://earthquake.usgs.gov/earthquakes/eventpage/us1234",
"detail": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us1234.geojson",
"felt": None,
"cdi": None,
"mmi": None,
"alert": None,
"status": "automatic",
"tsunami": 0,
"sig": 100,
"net": "us",
"code": "1234",
"ids": ",us1234,",
"sources": ",us,",
"types": ",origin,",
"nst": 10,
"dmin": 0.5,
"rms": 0.3,
"gap": 100,
"magType": "ml",
"type": "earthquake",
"title": "M 2.5 - 10km N of Boise, Idaho"
},
"geometry": {
"type": "Point",
"coordinates": [-116.2, 43.7, 10.5]
},
"id": "us1234"
},
{
"type": "Feature",
"properties": {
"mag": 4.5,
"place": "20km S of Portland, Oregon",
"time": 1715878400000,
"updated": 1715878500000,
"tz": None,
"url": "https://earthquake.usgs.gov/earthquakes/eventpage/us5678",
"detail": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us5678.geojson",
"felt": 50,
"cdi": 4.0,
"mmi": 3.5,
"alert": "green",
"status": "reviewed",
"tsunami": 0,
"sig": 300,
"net": "us",
"code": "5678",
"ids": ",us5678,",
"sources": ",us,",
"types": ",origin,shakemap,",
"nst": 25,
"dmin": 0.2,
"rms": 0.2,
"gap": 50,
"magType": "mw",
"type": "earthquake",
"title": "M 4.5 - 20km S of Portland, Oregon"
},
"geometry": {
"type": "Point",
"coordinates": [-122.6, 45.3, 15.0]
},
"id": "us5678"
},
{
"type": "Feature",
"properties": {
"mag": 3.0,
"place": "50km E of San Francisco, California",
"time": 1715878300000,
"updated": 1715878400000,
"tz": None,
"url": "https://earthquake.usgs.gov/earthquakes/eventpage/us9999",
"detail": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us9999.geojson",
"felt": None,
"cdi": None,
"mmi": None,
"alert": None,
"status": "automatic",
"tsunami": 0,
"sig": 150,
"net": "us",
"code": "9999",
"ids": ",us9999,",
"sources": ",us,",
"types": ",origin,",
"nst": 15,
"dmin": 0.3,
"rms": 0.25,
"gap": 80,
"magType": "ml",
"type": "earthquake",
"title": "M 3.0 - 50km E of San Francisco, California"
},
"geometry": {
"type": "Point",
"coordinates": [-121.5, 37.8, 8.0]
},
"id": "us9999"
}
]
}
# Sample with null magnitude
SAMPLE_NULL_MAG = {
"type": "FeatureCollection",
"metadata": {"count": 1},
"features": [
{
"type": "Feature",
"properties": {
"mag": None,
"place": "Quarry blast",
"time": 1715878500000,
"type": "quarry blast"
},
"geometry": {
"type": "Point",
"coordinates": [-116.0, 44.0, 0.0]
},
"id": "usquarry1"
}
]
}
def make_adapter_config(
region: dict | None = None,
feed: str = "all_hour",
) -> AdapterConfig:
"""Create an AdapterConfig for testing."""
settings = {"feed": feed}
if region:
settings["region"] = region
else:
settings["region"] = {
"north": 49.5,
"south": 40.0,
"east": -110.0,
"west": -125.0,
}
return AdapterConfig(
name="usgs_quake",
enabled=True,
cadence_s=60,
settings=settings,
updated_at=datetime.now(timezone.utc),
)
@pytest.fixture
def temp_db_path():
"""Create a temporary database path for testing."""
with tempfile.NamedTemporaryFile(suffix=".db", delete=False) as f:
yield Path(f.name)
@pytest.fixture
def mock_config_store():
"""Create a mock ConfigStore."""
return MagicMock()
class TestMagnitudeTier:
"""Test magnitude tier classification."""
def test_minor(self):
assert magnitude_tier(0.5) == "minor"
assert magnitude_tier(2.9) == "minor"
def test_light(self):
assert magnitude_tier(3.0) == "light"
assert magnitude_tier(3.9) == "light"
def test_moderate(self):
assert magnitude_tier(4.0) == "moderate"
assert magnitude_tier(4.9) == "moderate"
def test_strong(self):
assert magnitude_tier(5.0) == "strong"
assert magnitude_tier(5.9) == "strong"
def test_major(self):
assert magnitude_tier(6.0) == "major"
assert magnitude_tier(6.9) == "major"
def test_great(self):
assert magnitude_tier(7.0) == "great"
assert magnitude_tier(9.5) == "great"
class TestMagnitudeToSeverity:
"""Test magnitude to severity mapping."""
def test_severity_levels(self):
assert magnitude_to_severity(2.0) == 0
assert magnitude_to_severity(3.5) == 1
assert magnitude_to_severity(4.5) == 2
assert magnitude_to_severity(5.5) == 3
assert magnitude_to_severity(6.5) == 4
assert magnitude_to_severity(7.5) == 5
class TestRegionFiltering:
"""Test region/bbox filtering."""
@pytest.mark.asyncio
async def test_filters_out_of_bbox(self, temp_db_path, mock_config_store):
"""Test that quakes outside bbox are filtered."""
# Region covers PNW only (north of 40, west of -110)
config = make_adapter_config(
region={"north": 49.5, "south": 40.0, "east": -110.0, "west": -125.0}
)
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
with patch.object(adapter, "_fetch_geojson", new_callable=AsyncMock) as mock_fetch:
mock_fetch.return_value = SAMPLE_GEOJSON
events = []
async for event in adapter.poll():
events.append(event)
# us1234 (Boise) and us5678 (Portland) are in bbox
# us9999 (SF, lat 37.8) is outside bbox (south < 40)
assert len(events) == 2
event_ids = {e.id for e in events}
assert "us1234" in event_ids
assert "us5678" in event_ids
assert "us9999" not in event_ids
await adapter.shutdown()
class TestDeduplication:
"""Test deduplication logic."""
@pytest.mark.asyncio
async def test_dedup_marks_published(self, temp_db_path, mock_config_store):
config = make_adapter_config()
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
event_id = "us1234"
assert not adapter.is_published(event_id)
adapter.mark_published(event_id)
assert adapter.is_published(event_id)
await adapter.shutdown()
@pytest.mark.asyncio
async def test_second_poll_no_duplicates(self, temp_db_path, mock_config_store):
"""Test that second poll with same events yields nothing."""
config = make_adapter_config()
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
with patch.object(adapter, "_fetch_geojson", new_callable=AsyncMock) as mock_fetch:
mock_fetch.return_value = SAMPLE_GEOJSON
# First poll
events1 = []
async for event in adapter.poll():
events1.append(event)
# Second poll - same data
events2 = []
async for event in adapter.poll():
events2.append(event)
# First poll should have events (2 in bbox)
assert len(events1) == 2
# Second poll should have 0 (all deduped)
assert len(events2) == 0
await adapter.shutdown()
class TestNullMagnitude:
"""Test handling of null magnitude events."""
@pytest.mark.asyncio
async def test_skips_null_magnitude(self, temp_db_path, mock_config_store):
"""Test that events with null magnitude are skipped."""
config = make_adapter_config()
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
with patch.object(adapter, "_fetch_geojson", new_callable=AsyncMock) as mock_fetch:
mock_fetch.return_value = SAMPLE_NULL_MAG
events = []
async for event in adapter.poll():
events.append(event)
# Should skip the null-magnitude event
assert len(events) == 0
await adapter.shutdown()
class TestEventGeneration:
"""Test Event generation from features."""
@pytest.mark.asyncio
async def test_event_category(self, temp_db_path, mock_config_store):
config = make_adapter_config()
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
with patch.object(adapter, "_fetch_geojson", new_callable=AsyncMock) as mock_fetch:
mock_fetch.return_value = SAMPLE_GEOJSON
events = []
async for event in adapter.poll():
events.append(event)
# Check categories
categories = {e.category for e in events}
# us1234 is M2.5 -> minor, us5678 is M4.5 -> moderate
assert "quake.event.minor" in categories
assert "quake.event.moderate" in categories
await adapter.shutdown()
@pytest.mark.asyncio
async def test_event_severity(self, temp_db_path, mock_config_store):
config = make_adapter_config()
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
with patch.object(adapter, "_fetch_geojson", new_callable=AsyncMock) as mock_fetch:
mock_fetch.return_value = SAMPLE_GEOJSON
events = []
async for event in adapter.poll():
events.append(event)
# Find events by ID
events_by_id = {e.id: e for e in events}
# M2.5 -> severity 0
assert events_by_id["us1234"].severity == 0
# M4.5 -> severity 2
assert events_by_id["us5678"].severity == 2
await adapter.shutdown()
@pytest.mark.asyncio
async def test_event_geo(self, temp_db_path, mock_config_store):
config = make_adapter_config()
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
with patch.object(adapter, "_fetch_geojson", new_callable=AsyncMock) as mock_fetch:
mock_fetch.return_value = SAMPLE_GEOJSON
events = []
async for event in adapter.poll():
events.append(event)
events_by_id = {e.id: e for e in events}
# Check Boise quake coordinates
boise = events_by_id["us1234"]
assert boise.geo.centroid == (-116.2, 43.7)
await adapter.shutdown()
class TestApplyConfig:
"""Test hot-reload configuration application."""
@pytest.mark.asyncio
async def test_apply_config_updates_region(self, temp_db_path, mock_config_store):
config = make_adapter_config(
region={"north": 49.5, "south": 40.0, "east": -110.0, "west": -125.0}
)
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
assert adapter.region.north == 49.5
new_config = make_adapter_config(
region={"north": 48.0, "south": 45.0, "east": -115.0, "west": -125.0}
)
await adapter.apply_config(new_config)
assert adapter.region.north == 48.0
assert adapter.region.south == 45.0
await adapter.shutdown()
@pytest.mark.asyncio
async def test_apply_config_updates_feed(self, temp_db_path, mock_config_store):
config = make_adapter_config(feed="all_hour")
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
assert adapter._feed == "all_hour"
new_config = make_adapter_config(feed="all_day")
await adapter.apply_config(new_config)
assert adapter._feed == "all_day"
await adapter.shutdown()
"""Tests for USGS earthquake adapter."""
import pytest
from datetime import datetime, timezone
from unittest.mock import AsyncMock, MagicMock, patch
from pathlib import Path
import tempfile
from central.adapters.usgs_quake import (
USGSQuakeAdapter,
magnitude_tier,
magnitude_to_severity,
)
from central.config_models import AdapterConfig, RegionConfig
from central.models import Event, Geo
# Sample USGS GeoJSON response
SAMPLE_GEOJSON = {
"type": "FeatureCollection",
"metadata": {
"generated": 1715878800000,
"url": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_hour.geojson",
"title": "USGS All Earthquakes, Past Hour",
"status": 200,
"api": "1.10.3",
"count": 3
},
"features": [
{
"type": "Feature",
"properties": {
"mag": 2.5,
"place": "10km N of Boise, Idaho",
"time": 1715878500000,
"updated": 1715878600000,
"tz": None,
"url": "https://earthquake.usgs.gov/earthquakes/eventpage/us1234",
"detail": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us1234.geojson",
"felt": None,
"cdi": None,
"mmi": None,
"alert": None,
"status": "automatic",
"tsunami": 0,
"sig": 100,
"net": "us",
"code": "1234",
"ids": ",us1234,",
"sources": ",us,",
"types": ",origin,",
"nst": 10,
"dmin": 0.5,
"rms": 0.3,
"gap": 100,
"magType": "ml",
"type": "earthquake",
"title": "M 2.5 - 10km N of Boise, Idaho"
},
"geometry": {
"type": "Point",
"coordinates": [-116.2, 43.7, 10.5]
},
"id": "us1234"
},
{
"type": "Feature",
"properties": {
"mag": 4.5,
"place": "20km S of Portland, Oregon",
"time": 1715878400000,
"updated": 1715878500000,
"tz": None,
"url": "https://earthquake.usgs.gov/earthquakes/eventpage/us5678",
"detail": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us5678.geojson",
"felt": 50,
"cdi": 4.0,
"mmi": 3.5,
"alert": "green",
"status": "reviewed",
"tsunami": 0,
"sig": 300,
"net": "us",
"code": "5678",
"ids": ",us5678,",
"sources": ",us,",
"types": ",origin,shakemap,",
"nst": 25,
"dmin": 0.2,
"rms": 0.2,
"gap": 50,
"magType": "mw",
"type": "earthquake",
"title": "M 4.5 - 20km S of Portland, Oregon"
},
"geometry": {
"type": "Point",
"coordinates": [-122.6, 45.3, 15.0]
},
"id": "us5678"
},
{
"type": "Feature",
"properties": {
"mag": 3.0,
"place": "50km E of San Francisco, California",
"time": 1715878300000,
"updated": 1715878400000,
"tz": None,
"url": "https://earthquake.usgs.gov/earthquakes/eventpage/us9999",
"detail": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us9999.geojson",
"felt": None,
"cdi": None,
"mmi": None,
"alert": None,
"status": "automatic",
"tsunami": 0,
"sig": 150,
"net": "us",
"code": "9999",
"ids": ",us9999,",
"sources": ",us,",
"types": ",origin,",
"nst": 15,
"dmin": 0.3,
"rms": 0.25,
"gap": 80,
"magType": "ml",
"type": "earthquake",
"title": "M 3.0 - 50km E of San Francisco, California"
},
"geometry": {
"type": "Point",
"coordinates": [-121.5, 37.8, 8.0]
},
"id": "us9999"
}
]
}
# Sample with null magnitude
SAMPLE_NULL_MAG = {
"type": "FeatureCollection",
"metadata": {"count": 1},
"features": [
{
"type": "Feature",
"properties": {
"mag": None,
"place": "Quarry blast",
"time": 1715878500000,
"type": "quarry blast"
},
"geometry": {
"type": "Point",
"coordinates": [-116.0, 44.0, 0.0]
},
"id": "usquarry1"
}
]
}
def make_adapter_config(
region: dict | None = None,
feed: str = "all_hour",
) -> AdapterConfig:
"""Create an AdapterConfig for testing."""
settings = {"feed": feed}
if region:
settings["region"] = region
else:
settings["region"] = {
"north": 49.5,
"south": 40.0,
"east": -110.0,
"west": -125.0,
}
return AdapterConfig(
name="usgs_quake",
enabled=True,
cadence_s=60,
settings=settings,
updated_at=datetime.now(timezone.utc),
)
@pytest.fixture
def temp_db_path():
"""Create a temporary database path for testing."""
with tempfile.NamedTemporaryFile(suffix=".db", delete=False) as f:
yield Path(f.name)
@pytest.fixture
def mock_config_store():
"""Create a mock ConfigStore."""
return MagicMock()
class TestMagnitudeTier:
"""Test magnitude tier classification."""
def test_minor(self):
assert magnitude_tier(0.5) == "minor"
assert magnitude_tier(2.9) == "minor"
def test_light(self):
assert magnitude_tier(3.0) == "light"
assert magnitude_tier(3.9) == "light"
def test_moderate(self):
assert magnitude_tier(4.0) == "moderate"
assert magnitude_tier(4.9) == "moderate"
def test_strong(self):
assert magnitude_tier(5.0) == "strong"
assert magnitude_tier(5.9) == "strong"
def test_major(self):
assert magnitude_tier(6.0) == "major"
assert magnitude_tier(6.9) == "major"
def test_great(self):
assert magnitude_tier(7.0) == "great"
assert magnitude_tier(9.5) == "great"
class TestMagnitudeToSeverity:
"""Test magnitude to severity mapping."""
def test_severity_levels(self):
assert magnitude_to_severity(2.0) == 0
assert magnitude_to_severity(3.5) == 1
assert magnitude_to_severity(4.5) == 2
assert magnitude_to_severity(5.5) == 3
assert magnitude_to_severity(6.5) == 4
assert magnitude_to_severity(7.5) == 5
class TestRegionFiltering:
"""Test region/bbox filtering."""
@pytest.mark.asyncio
async def test_filters_out_of_bbox(self, temp_db_path, mock_config_store):
"""Test that quakes outside bbox are filtered."""
# Region covers PNW only (north of 40, west of -110)
config = make_adapter_config(
region={"north": 49.5, "south": 40.0, "east": -110.0, "west": -125.0}
)
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
with patch.object(adapter, "_fetch_geojson", new_callable=AsyncMock) as mock_fetch:
mock_fetch.return_value = SAMPLE_GEOJSON
events = []
async for event in adapter.poll():
events.append(event)
# us1234 (Boise) and us5678 (Portland) are in bbox
# us9999 (SF, lat 37.8) is outside bbox (south < 40)
assert len(events) == 2
event_ids = {e.id for e in events}
assert "us1234" in event_ids
assert "us5678" in event_ids
assert "us9999" not in event_ids
await adapter.shutdown()
class TestDeduplication:
"""Test deduplication logic."""
@pytest.mark.asyncio
async def test_dedup_marks_published(self, temp_db_path, mock_config_store):
config = make_adapter_config()
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
event_id = "us1234"
assert not adapter.is_published(event_id)
adapter.mark_published(event_id)
assert adapter.is_published(event_id)
await adapter.shutdown()
@pytest.mark.asyncio
async def test_second_poll_no_duplicates(self, temp_db_path, mock_config_store):
"""Test that second poll with same events yields nothing."""
config = make_adapter_config()
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
with patch.object(adapter, "_fetch_geojson", new_callable=AsyncMock) as mock_fetch:
mock_fetch.return_value = SAMPLE_GEOJSON
# First poll
events1 = []
async for event in adapter.poll():
events1.append(event)
# Second poll - same data
events2 = []
async for event in adapter.poll():
events2.append(event)
# First poll should have events (2 in bbox)
assert len(events1) == 2
# Second poll should have 0 (all deduped)
assert len(events2) == 0
await adapter.shutdown()
class TestNullMagnitude:
"""Test handling of null magnitude events."""
@pytest.mark.asyncio
async def test_skips_null_magnitude(self, temp_db_path, mock_config_store):
"""Test that events with null magnitude are skipped."""
config = make_adapter_config()
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
with patch.object(adapter, "_fetch_geojson", new_callable=AsyncMock) as mock_fetch:
mock_fetch.return_value = SAMPLE_NULL_MAG
events = []
async for event in adapter.poll():
events.append(event)
# Should skip the null-magnitude event
assert len(events) == 0
await adapter.shutdown()
class TestEventGeneration:
"""Test Event generation from features."""
@pytest.mark.asyncio
async def test_event_category(self, temp_db_path, mock_config_store):
config = make_adapter_config()
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
with patch.object(adapter, "_fetch_geojson", new_callable=AsyncMock) as mock_fetch:
mock_fetch.return_value = SAMPLE_GEOJSON
events = []
async for event in adapter.poll():
events.append(event)
# Check categories
categories = {e.category for e in events}
# us1234 is M2.5 -> minor, us5678 is M4.5 -> moderate
assert "quake.event.minor" in categories
assert "quake.event.moderate" in categories
await adapter.shutdown()
@pytest.mark.asyncio
async def test_event_severity(self, temp_db_path, mock_config_store):
config = make_adapter_config()
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
with patch.object(adapter, "_fetch_geojson", new_callable=AsyncMock) as mock_fetch:
mock_fetch.return_value = SAMPLE_GEOJSON
events = []
async for event in adapter.poll():
events.append(event)
# Find events by ID
events_by_id = {e.id: e for e in events}
# M2.5 -> severity 0
assert events_by_id["us1234"].severity == 0
# M4.5 -> severity 2
assert events_by_id["us5678"].severity == 2
await adapter.shutdown()
@pytest.mark.asyncio
async def test_event_geo(self, temp_db_path, mock_config_store):
config = make_adapter_config()
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
with patch.object(adapter, "_fetch_geojson", new_callable=AsyncMock) as mock_fetch:
mock_fetch.return_value = SAMPLE_GEOJSON
events = []
async for event in adapter.poll():
events.append(event)
events_by_id = {e.id: e for e in events}
# Check Boise quake coordinates
boise = events_by_id["us1234"]
assert boise.geo.centroid == (-116.2, 43.7)
await adapter.shutdown()
class TestApplyConfig:
"""Test hot-reload configuration application."""
@pytest.mark.asyncio
async def test_apply_config_updates_region(self, temp_db_path, mock_config_store):
config = make_adapter_config(
region={"north": 49.5, "south": 40.0, "east": -110.0, "west": -125.0}
)
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
assert adapter.region.north == 49.5
new_config = make_adapter_config(
region={"north": 48.0, "south": 45.0, "east": -115.0, "west": -125.0}
)
await adapter.apply_config(new_config)
assert adapter.region.north == 48.0
assert adapter.region.south == 45.0
await adapter.shutdown()
@pytest.mark.asyncio
async def test_apply_config_updates_feed(self, temp_db_path, mock_config_store):
config = make_adapter_config(feed="all_hour")
adapter = USGSQuakeAdapter(
config=config,
config_store=mock_config_store,
cursor_db_path=temp_db_path,
)
await adapter.startup()
assert adapter._feed == "all_hour"
new_config = make_adapter_config(feed="all_day")
await adapter.apply_config(new_config)
assert adapter._feed == "all_day"
await adapter.shutdown()