Matt Johnson
44f0257376
- Authentik -> edge2 CT 105 (Postgres pg_dump/restore; SECRET_KEY carried verbatim; zero-downtime until ~2s cutover) - Multi-block Caddy cutover: auth.echo6.co + notes.echo6.co outpost/forward_auth -> 100.64.0.36:9000 - runbook: add reboot tailscale-before-docker gotcha; clarify dnsmasq must NOT be repointed (points at Caddy host) - source left stopped + intact on Contabo as cold rollback Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2.5 KiB
| type | title | status | created | origin | resume-on | tags | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| session-resume | Session Handoff — MeshAI radio-drop resilience test | open | 2026-06-17 | matt-desktop (WSL) | cortex |
|
Session Handoff — MeshAI radio-drop resilience test
Created: 2026-06-17, from matt-desktop (WSL). Resume from cortex.
What we were doing
Matt unplugged/replugged the network cable on aida-nebra (AIDA-N2 Meshtastic node) to reboot it. The link dropped for ~12s then recovered. Matt then realized this is an accidental resilience test for MeshAI — he wants to see "how it dumps": how MeshAI handled losing and regaining its radio TCP connection (clean reconnect vs. errors/stack traces vs. crash+restart).
Key facts
- aida-nebra: 192.168.1.253 / TS 100.64.0.9 — meshtasticd node !27780c47, Nebra 2W hat, TCP API port 4403. Confirmed back online (LAN, :4403, and Tailscale all UP after the replug).
- MeshAI: utility CT 108, 192.168.1.144 / TS 100.64.0.32.
Docker image
ghcr.io/zvx-echo6/meshai:latest. Connects to the radio at 192.168.1.253:4403. Compose:/home/zvx/meshai/docker-compose.yml. - cortex and its host toc had a brief outage earlier this session; both back UP.
Why this stalled (the blocker)
matt-desktop's WSL SSH key is NOT authorized on utility (192.168.1.241):
Permission denied (publickey) — key SHA256:QL0Tm72T7RYVZfeDhPHJDUYnstRNFx2EF5/k8tKv3AM.
Could ping everything but not SSH in. Per policy, stopped instead of routing around.
cortex has working keys — that's why we're resuming there.
Next step on cortex (read-only diagnostic — do NOT restart/redeploy anything)
# find the meshai container + current status (did it restart through the blip?)
ssh zvx@192.168.1.241 "pct exec 108 -- docker ps --format '{{.Names}}\t{{.Image}}\t{{.Status}}'"
# pull logs across the blip window with timestamps
ssh zvx@192.168.1.241 "pct exec 108 -- docker logs --since 30m --timestamps <name> 2>&1 | tail -200"
Report: container uptime/restart count, verbatim disconnect→error→reconnect lines, and a one-line verdict (clean recover / crashed+auto-restarted / stuck).
NOTE: the blip was ~16:xx MT on 2026-06-17 (a few min before this file's timestamp). Logs roll — pull soon to still catch the disconnect/reconnect dump.
Working model reminder
Matt guides → Opus orchestrates (dispatch Sonnet for the SSH/log pull) → Sonnet executes the tight prompt above.