Commit graph

143 commits

Author SHA1 Message Date
Matt Johnson
12a66d45ba docs: add Phase 1B planning notes
- Stream retention GUI design
- Region picker for bbox selection
- API key management requirements

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 18:49:29 +00:00
93d86a9276
Merge pull request #5 from zvx-echo6/feature/remove-adapter-limiter
fix: remove NWSAdapter internal AsyncLimiter (cadence-decrease root cause)
2026-05-16 11:31:38 -06:00
Matt Johnson
2597153a9c docs: add final cadence-decrease fix verification
Documents production verification of the AsyncLimiter removal fix:
- Decrease 60-30s: poll at Tlast+30s (not 60s)
- Increase 30-60s: poll at Tlast+60s
- Decrease 60-15s: immediate poll (deadline passed)
- All subsequent intervals use new cadence

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 17:26:26 +00:00
Matt Johnson
5b028b38e8 test: remove ineffective hot-reload tests
These tests pass on both fixed and unfixed code, meaning they do
not actually exercise the cadence-decrease bug. The tests were
added as part of PR #4 but direct verification showed they
do not catch the issue they claim to test.

A follow-up issue should be filed for proper regression tests
that reproduce the actual bug (AsyncLimiter blocking).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 17:19:27 +00:00
Matt Johnson
c368f175a1 build: remove aiolimiter dependency
No longer needed after removing internal rate limiting from NWSAdapter.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 17:19:15 +00:00
Matt Johnson
0eba319071 refactor(supervisor): remove adapter.cadence_s update
The NWSAdapter no longer has a cadence_s attribute since the
internal limiter was removed. The supervisor's rate limiting
via state.config.cadence_s and last_completed_poll is sufficient.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 17:18:55 +00:00
Matt Johnson
9d4ba97537 refactor(nws): remove internal AsyncLimiter rate limiting
The NWSAdapter had an internal AsyncLimiter that duplicated the
supervisor's rate-limit guarantee. When cadence changed, only
state.adapter.cadence_s was updated, not the internal limiter,
causing the cadence-decrease bug.

Since the supervisor already enforces rate limiting via
last_completed_poll + cadence_s scheduling, the adapter-level
limiter was redundant and caused the 30-second blocking observed
in diagnostic logs.

Removes:
- aiolimiter import
- self.cadence_s attribute (unused elsewhere)
- self._limiter creation
- async with self._limiter context in _fetch_alerts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 17:17:11 +00:00
6deccf1cf8
Merge pull request #4 from zvx-echo6/feature/1a-4-cadence-decrease-fix
fix: cadence-decrease hot-reload + integration tests + env docs
2026-05-16 00:58:24 -06:00
Matt Johnson
d210c980fd docs: add environment reference and bug investigation report
environment.md:
- Documents CT104 as the active development location
- Lists SSH access, repository paths, and service commands
- Notes that cortex clone is parked, matt-desktop has no clones

BUG-CADENCE-DECREASE.md:
- Full investigation of the cadence-decrease hot-reload bug
- Root cause analysis: cancel_event.set() inside lock context
- Proposed fix (Option A - structural)
- Test gap identification
- Production verification steps

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 05:59:53 +00:00
Matt Johnson
4215744a30 fix: move cancel_event signal outside lock for immediate delivery
The cancel_event.set() call was inside the async lock context in
_on_config_change, causing delayed signal delivery to the sleeping
loop. This manifested as cadence decreases not applying without a
restart - the loop would sleep its full original timeout before
seeing the new cadence.

Fix: _reschedule_adapter now returns the AdapterState to signal,
and _on_config_change signals AFTER releasing the lock. This ensures
immediate event delivery per asyncio semantics.

The lock protects state consistency during config fetches and updates.
The cancel_event is a one-way notification that does not need lock
protection - it simply wakes the sleeping coroutine.

See docs/BUG-CADENCE-DECREASE.md for full investigation.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 05:59:45 +00:00
Matt Johnson
35de09ea93 test: add hot-reload integration tests for cadence changes
Add tests that exercise the ACTUAL running loop with cancel_event
signaling, not just AdapterState math in isolation.

Test cases:
- Test 1: Cadence decrease (60->30) wakes loop immediately
- Test 2: Cadence increase (10->20) extends wait correctly
- Test 3: Enable/disable/enable with gap > cadence polls immediately
- Test 4: Enable/disable/enable with gap < cadence waits

These tests verify the cancel_event mechanism properly interrupts
the sleeping loop when config changes occur via _on_config_change.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 05:59:35 +00:00
Matt Johnson
c4a65a2ad7 docs: Phase 1a-3 final close-out verification
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 04:04:02 +00:00
0b23cc4572
Merge pull request #3 from zvx-echo6/feature/1a-3-phase-c-toml-retirement
Phase 1a-3 Part C: Retire TOML config source
2026-05-15 22:00:02 -06:00
Matt Johnson
24d99f18e2 docs: append cadence revert to Phase 1a-3 verification
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 03:57:32 +00:00
Ubuntu
a362b7b93e test: remove TomlConfigSource and config_source flag tests
TOML-related tests no longer needed after Phase C retirement.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 03:42:43 +00:00
Ubuntu
c6f4f3b081 refactor: supervisor always uses DbConfigSource
Remove conditional config source loading, simplify async_main.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 03:42:38 +00:00
Ubuntu
a1e81bae8a refactor: remove config_source flag from bootstrap settings
Database config is now the only option, no need for feature flag.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 03:42:33 +00:00
Ubuntu
4376588baf refactor: remove TomlConfigSource, keep only DbConfigSource
TOML config is now retired. Database is the sole configuration source.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 03:42:28 +00:00
Matt Johnson
98f3e578a4 docs: add Phase 1a-3 verification evidence
Phase B operational cutover verification:
- Config source cutover from TOML to DB confirmed
- Hot-reload cadence test passed (rate-limit guarantee)
- Enable/disable cycle test passed (preserved_last_poll)
- 10-minute soak with zero errors
- Data integrity verified (all alerts in DB)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 03:36:14 +00:00
b3788d556d
Merge pull request #2 from zvx-echo6/feature/1a-service-cutover
feat(config): Phase 1a-3 service cutover to DB-backed config
2026-05-15 21:08:41 -06:00
Matt Johnson
41439c52b3 refactor: rename DEFAULT_CLOUDEVENTS_CONFIG to CLOUDEVENTS_CONFIG
These are protocol-level constants, not defaults that get overridden.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 03:08:09 +00:00
Matt Johnson
c6cbdb0825 docs: add systemd unit files with EnvironmentFile directive
Unit files load env vars from /etc/central/central.env using
EnvironmentFile directive. Includes README with installation
and configuration instructions.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 02:38:21 +00:00
Matt Johnson
73beb90b25 chore: remove unused ABC import from config_source.py
ConfigSource uses Protocol, not ABC. Removed unused import.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 02:38:15 +00:00
Matt Johnson
b16151abf1 refactor: move CloudEvents config to code constants
CloudEvents envelope format is protocol-level (not operator config).
When using DB config source without TOML, wrap_event() now uses
DEFAULT_CLOUDEVENTS_CONFIG from cloudevents_constants.py.

Changes:
- Add cloudevents_constants.py with DEFAULT_CLOUDEVENTS_CONFIG
- Update wrap_event() to accept Config, CloudEventsConfig, or None
- Simplify supervisor: always use wrap_event (has defaults)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 02:38:11 +00:00
Matt Johnson
c39e3174b8 fix: preserve last_completed_poll across adapter disable/enable
Previously, _stop_adapter() used pop() to remove adapter state,
which lost last_completed_poll. On re-enable, a fresh state was
created, causing immediate poll and violating rate-limit guarantee.

Changes:
- Add is_running property to AdapterState
- _stop_adapter: preserve state, just cancel task
- _start_adapter: reuse existing stopped state if present
- Add _remove_adapter for full cleanup when adapter is deleted
- _on_config_change: call _remove_adapter for deleted adapters

Integration tests verify:
- Test A: gap > cadence -> immediate poll (correct)
- Test B: gap < cadence -> wait until last_poll + cadence (was broken)
- Test C: delete + re-add -> fresh state (correct)

Tests-fail-before-fix verified: Test A/B failed on unfixed code
with "State was removed on stop!", pass with fix.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 02:37:57 +00:00
Ubuntu
1abdf45375 test: add ConfigSource and hot-reload rate-limit tests
- TomlConfigSource tests: list/get adapters, watch_for_changes no-op
- DbConfigSource tests: list/get adapters with DB fixtures
- create_config_source factory tests
- NOTIFY integration test for DbConfigSource
- Rate-limit guarantee tests:
  - Cadence change respects last_poll time
  - Gap exceeding new cadence polls immediately
  - Enable/disable/enable respects rate limit
  - Multiple rapid changes no extra polls
- Bootstrap flag validation tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 01:55:47 +00:00
Ubuntu
daa7852cc0 refactor(archive): use bootstrap_config for connection strings
Archive now reads NATS URL and Postgres DSN from bootstrap_config
instead of TOML file. This is sufficient for archive since it only
needs connection strings, not adapter configuration.

No ConfigSource wiring needed - archive just consumes from JetStream.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 01:55:39 +00:00
Ubuntu
29fa49c5c2 feat(supervisor): add hot-reload support with rate-limit guarantee
Refactors supervisor to use ConfigSource abstraction:
- AdapterState tracks last_completed_poll for rate limiting
- Hot-reload via NOTIFY: cadence/enable/disable changes take effect
- Rate-limit guarantee: next poll at last_poll + new_cadence, not now
- Logs config source at startup (toml or db)
- Logs reschedule decisions with next poll timestamp

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 01:55:33 +00:00
Ubuntu
ee593abc54 feat(config): add ConfigSource abstraction and CENTRAL_CONFIG_SOURCE flag
- ConfigSource protocol with list_enabled_adapters, get_adapter, watch_for_changes
- TomlConfigSource: loads from TOML file, watch_for_changes is no-op
- DbConfigSource: wraps ConfigStore with LISTEN/NOTIFY support
- CENTRAL_CONFIG_SOURCE bootstrap flag: toml (default) or db
- CENTRAL_CONFIG_TOML_PATH for specifying TOML file location

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 01:55:25 +00:00
ee081c9bc2
Merge pull request #1 from zvx-echo6/feature/1a-config-storage
feat(config): Phase 1a-2 config storage primitives
2026-05-15 19:49:21 -06:00
Ubuntu
25909b0f4d fix(tests): replace echo6 reference with central.local
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 01:36:44 +00:00
Ubuntu
826141c71a docs(tests): add README documenting CENTRAL_TEST_DB_DSN env var
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 01:36:40 +00:00
Ubuntu
b183a621bb feat(config_store): add listener reconnect with exponential backoff
Listener now automatically reconnects on connection loss with
exponential backoff (1s-30s). Cancellation propagates cleanly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 01:36:35 +00:00
Ubuntu
166268a44e feat(db): add migration 002 for updated_at trigger and enabled index
Adds auto-update trigger for updated_at column on adapters table
and partial index for efficient enabled adapter queries.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-16 01:36:30 +00:00
Ubuntu
3e392cad81 feat(config): add CLI smoke command and dependencies
Add central-cli with config-store-check command that:
- Connects via bootstrap config
- Lists adapters from config store
- Verifies crypto round-trip

Updates pyproject.toml with new dependencies:
- pydantic-settings>=2.7.0
- cryptography>=44.0.0

New entry points:
- central-migrate
- central-cli

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-15 23:08:03 +00:00
Ubuntu
8c5349c880 feat(config): add database-backed config store
Add ConfigStore class providing async access to config schema:

- get_adapter/list_adapters/upsert_adapter for adapter config
- pause_adapter/unpause_adapter for runtime control
- set_api_key/get_api_key with encryption via crypto.py
- listen_for_changes using Postgres LISTEN/NOTIFY

Includes Pydantic models (AdapterConfig, ApiKeyInfo) and tests
using real Postgres test database.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-15 23:07:56 +00:00
Ubuntu
a9b7dcab62 feat(config): add migration framework and config schema
Add simple SQL migration runner tracking applied migrations in
schema_migrations table. First migration creates:

- config schema
- config.adapters table (name, enabled, cadence_s, settings JSONB)
- config.api_keys table (alias, encrypted_value BYTEA)
- NOTIFY triggers for real-time config change detection
- Seeds NWS adapter row from current TOML config

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-15 23:07:49 +00:00
Ubuntu
fab452aa02 feat(config): add AES-256-GCM crypto primitives
Add encrypt/decrypt functions using AES-256-GCM for secret storage.
Master key loaded from file path specified in bootstrap config.

Features:
- 32-byte key from base64-encoded file
- 12-byte random nonce per encryption
- AEAD authentication (detects tampering)
- Key caching with clear_key_cache() for rotation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-15 23:07:41 +00:00
Ubuntu
e126569a4d feat(config): add bootstrap config from environment
Add pydantic-settings based Settings class for loading configuration
from environment variables or .env file. Provides early-stage config
before database-backed config store is available.

Includes:
- CENTRAL_DB_DSN, CENTRAL_NATS_URL, CENTRAL_MASTER_KEY_PATH, CENTRAL_LOG_LEVEL
- Cached loader with get_settings()
- Tests for env vars, .env file, validation, caching

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-15 23:07:33 +00:00
Matt Johnson
4a7f1a76c7 refactor: rename CloudEvents extension attributes hub* → central* v0.1.1
Rename extension attributes for consistency with project naming:
- hubschemaversion → centralschemaversion
- hubcategory → centralcategory
- hubseverity → centralseverity

Non-breaking change - no consumers depend on these names yet.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-15 21:47:50 +00:00
Matt Johnson
31be17430d runtime: NWS adapter, supervisor, archive consumer, systemd units
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-15 21:29:08 +00:00
Matt Johnson
714971fe99 foundation: models, adapter ABC, config, CE wire, schema
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-15 21:08:56 +00:00
Matt Johnson
36ebbcb250 scaffold: initial repository structure 2026-05-15 19:16:24 +00:00