docs: migrate Matrix + Headscale to edge2 (CT 106/107)
- Matrix -> edge2 CT 106 (5 containers, 3 Postgres DBs, Signal bridge state preserved) - Headscale -> edge2 CT 107 (noise_private.key carried; tailnet data-plane survived cutover) - runbook: tailnet-control-plane pattern + self-verifying auto-rollback tmux cutover - both sources stopped + intact on Contabo as cold rollback Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
75df23b89e
commit
74a0aaae43
4 changed files with 55 additions and 45 deletions
|
|
@ -10,7 +10,7 @@ related:
|
|||
- [[caddy]]
|
||||
- [[ct-runbook]]
|
||||
- [[proxmox-create-ubuntu-vm]]
|
||||
updated: 2026-06-18
|
||||
updated: 2026-06-19
|
||||
---
|
||||
# Echo6 Environment Reference
|
||||
|
||||
|
|
@ -102,7 +102,7 @@ Five nodes running Proxmox VE:
|
|||
| Contabo Server | 5.189.158.149 | 100.64.0.1 | External VPS: Mail, [[authentik]], Headscale, Forge, Matrix |
|
||||
| edge2 | 184.174.35.153 | 100.64.0.26 | Contabo Cloud VPS 30 NVMe — Proxmox VE 8.4.19 (LXC-only), 8c/24GB/400GB |
|
||||
|
||||
*Last updated: 2026-06-18 — Added central (utility CT 104, 192.168.1.104, 100.64.0.12); also added edge2 CT 105 (authentik, 10.10.10.23, 100.64.0.36, node 48, migrated 2026-06-18); previously added edge2 CT 104 (livesync), CT 103 (forgejo), CT 102 (vaultwarden), pdm CT 100, wordpress CT 101*
|
||||
*Last updated: 2026-06-19 — Added edge2 CT 106 (matrix, 10.10.10.24, 100.64.0.37, migrated 2026-06-18) and CT 107 (headscale, 10.10.10.25, 100.64.0.38, migrated 2026-06-19); previously added CT 105 (authentik), CT 104 (livesync), CT 103 (forgejo), CT 102 (vaultwarden), pdm CT 100, wordpress CT 101, central (utility CT 104)*
|
||||
|
||||
## LXC Containers
|
||||
|
||||
|
|
@ -127,8 +127,10 @@ Five nodes running Proxmox VE:
|
|||
| forgejo | edge2 (CT 103) | 10.10.10.21 | 100.64.0.34 | Forgejo git server (migrated from Contabo 2026-06-16) |
|
||||
| livesync | edge2 (CT 104) | 10.10.10.22 | 100.64.0.35 | LiveSync Obsidian sync (CouchDB + provisioner; migrated from Contabo 2026-06-16) |
|
||||
| authentik | edge2 (CT 105) | 10.10.10.23 | 100.64.0.36 | Authentik SSO platform (migrated from Contabo 2026-06-18) |
|
||||
| matrix | edge2 (CT 106) | 10.10.10.24 | 100.64.0.37 | Matrix stack (Synapse + MAS + Element + mautrix-signal; migrated from Contabo 2026-06-18) |
|
||||
| headscale | edge2 (CT 107) | 10.10.10.25 | 100.64.0.38 | Headscale + Headplane tailnet control plane (migrated from Contabo 2026-06-19) |
|
||||
|
||||
> **Note (2026-06-18):** edge2 CT placements above (CT 102–105) are sourced from migration git log. edge2 was not directly SSH-reachable during the 2026-06-18 fleet audit — placements pending live confirmation.
|
||||
> **Note (2026-06-18):** edge2 CT placements above (CT 102–105) are sourced from migration git log. edge2 was not directly SSH-reachable during the 2026-06-18 fleet audit — placements pending live confirmation. CT 106 (matrix) and CT 107 (headscale) confirmed migrated 2026-06-18/19.
|
||||
|
||||
## IP Allocation Scheme
|
||||
|
||||
|
|
@ -181,6 +183,8 @@ Current registered nodes (26 total):
|
|||
| forgejo | 100.64.0.34 | LXC (edge2 CT 103) — node id 46 |
|
||||
| livesync | 100.64.0.35 | LXC (edge2 CT 104) — migrated 2026-06-16 |
|
||||
| authentik | 100.64.0.36 | LXC (edge2 CT 105) — node id 48, migrated 2026-06-18 |
|
||||
| matrix | 100.64.0.37 | LXC (edge2 CT 106) — migrated 2026-06-18 |
|
||||
| headscale | 100.64.0.38 | LXC (edge2 CT 107) — migrated 2026-06-19 |
|
||||
|
||||
## IdahoMesh Headscale Node List
|
||||
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ related:
|
|||
- [[caddy]]
|
||||
- [[headscale-onboard-node]]
|
||||
- [[meshtastic-headscale-runbook]]
|
||||
updated: 2026-06-18
|
||||
updated: 2026-06-19
|
||||
---
|
||||
# Echo6 Network IP Allocation
|
||||
|
||||
|
|
@ -73,6 +73,8 @@ edge2 (Contabo Cloud VPS 184.174.35.153 / Tailscale 100.64.0.26) uses a separate
|
|||
| 10.10.10.21 | forgejo | CT 103 | 100.64.0.34 | Forgejo git server (migrated from Contabo 2026-06-16) |
|
||||
| 10.10.10.22 | livesync | CT 104 | 100.64.0.35 | LiveSync Obsidian sync (CouchDB + provisioner; migrated from Contabo 2026-06-16) |
|
||||
| 10.10.10.23 | [[authentik]] | CT 105 | 100.64.0.36 | Authentik SSO platform (migrated from Contabo 2026-06-18) |
|
||||
| 10.10.10.24 | matrix | CT 106 | 100.64.0.37 | Matrix stack (Synapse + MAS + Element + mautrix-signal; migrated from Contabo 2026-06-18) |
|
||||
| 10.10.10.25 | headscale | CT 107 | 100.64.0.38 | Headscale + Headplane tailnet control plane (migrated from Contabo 2026-06-19) |
|
||||
|
||||
### VMs (.150-.199)
|
||||
| IP | VM | Host | Purpose |
|
||||
|
|
@ -126,6 +128,8 @@ edge2 (Contabo Cloud VPS 184.174.35.153 / Tailscale 100.64.0.26) uses a separate
|
|||
| 100.64.0.34 | forgejo (CT 103 on edge2) — node id 46 | 10.10.10.21 (vmbr0) |
|
||||
| 100.64.0.35 | livesync (CT 104 on edge2) — hostname `livesync` | 10.10.10.22 (vmbr0) |
|
||||
| 100.64.0.36 | authentik (CT 105 on edge2) — node id 48 | 10.10.10.23 (vmbr0) |
|
||||
| 100.64.0.37 | matrix (CT 106 on edge2) — hostname `matrix` | 10.10.10.24 (vmbr0) |
|
||||
| 100.64.0.38 | headscale (CT 107 on edge2) — hostname `headscale` | 10.10.10.25 (vmbr0) |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -138,4 +142,4 @@ edge2 (Contabo Cloud VPS 184.174.35.153 / Tailscale 100.64.0.26) uses a separate
|
|||
|
||||
---
|
||||
|
||||
*Last updated: 2026-06-18 — Added edge2 CT 105 (authentik) at 10.10.10.23 / 100.64.0.36 (node id 48); previously added CT 104 (livesync), CT 103 (forgejo), CT 102 (vaultwarden)*
|
||||
*Last updated: 2026-06-19 — Added edge2 CT 106 (matrix) at 10.10.10.24 / 100.64.0.37 and CT 107 (headscale) at 10.10.10.25 / 100.64.0.38; previously added CT 105 (authentik), CT 104 (livesync), CT 103 (forgejo), CT 102 (vaultwarden)*
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ related:
|
|||
- [[glossary]]
|
||||
- [[meshtastic-headscale-runbook]]
|
||||
- [[lxc-service-migration]]
|
||||
updated: 2026-06-18
|
||||
updated: 2026-06-19
|
||||
---
|
||||
# Current Services Inventory
|
||||
|
||||
|
|
@ -33,8 +33,8 @@ updated: 2026-06-18
|
|||
| TimescaleDB/PostGIS (central) | utility (CT 104) | 192.168.1.104:5432 | Internal | Central backend time-series + geospatial database (PostgreSQL 16 + TimescaleDB + PostGIS) |
|
||||
| [[authentik]] | edge2 (CT 105) | 100.64.0.36:9000 | https://auth.echo6.co | SSO provider (Echo6 branded, custom CSS, dark theme) — fronted by Contabo Caddy (reverse_proxy 100.64.0.36:9000); **migrated from Contabo 2026-06-18** |
|
||||
| Forge (Forgejo) | edge2 (CT 103) | 100.64.0.34:3001 HTTP / :2222 SSH (via Contabo DNAT) | https://forge.echo6.co | Git server — fronted by Contabo Caddy (reverse_proxy 100.64.0.34:3001); git SSH via iptables DNAT on Contabo (forgejo-ssh-dnat.service) — **migrated from Contabo 2026-06-16** |
|
||||
| Headscale | Contabo | 5.189.158.149 | https://vpn.echo6.co | Tailscale coordination (OIDC enabled) |
|
||||
| Headplane | Contabo | 127.0.0.1:3100 | https://vpn.echo6.co/admin | Headscale web UI (OIDC via Authentik) |
|
||||
| Headscale | edge2 (CT 107) | 100.64.0.38:8084 | https://vpn.echo6.co | Tailscale coordination (OIDC enabled) — **migrated from Contabo 2026-06-19** |
|
||||
| Headplane | edge2 (CT 107) | 100.64.0.38:3100 | https://vpn.echo6.co/admin | Headscale web UI (OIDC via Authentik) — **migrated from Contabo 2026-06-19** |
|
||||
| Mailcow | Contabo | 5.189.158.149 | https://mail.echo6.co | Email server |
|
||||
| Vaultwarden | edge2 (CT 102) | 100.64.0.33:8086 | https://vault.echo6.co | Password manager (SSO enabled) — fronted by Contabo Caddy (reverse_proxy 100.64.0.33:8086) |
|
||||
| Syncthing | Contabo | 100.64.0.1:22000 | Internal (Tailscale) | File sync — ~/.claude/, ~/projects/ |
|
||||
|
|
@ -68,14 +68,14 @@ updated: 2026-06-18
|
|||
| PostgreSQL/PostGIS | data (VM 1130) | 192.168.1.130:5432 | Internal | navi geo backend (padus, overture DBs) — see [[navi]] |
|
||||
| Files | data (VM 1130) | 192.168.1.130:8888 | https://files.echo6.co | PDF library (nginx, Authentik forward auth) |
|
||||
| Samba | data | 192.168.1.240:445 | Internal | SMB file sharing — `//data/library` → /mnt/data/library (guest access) |
|
||||
| Matrix [[synapse]] | Contabo | 127.0.0.1:8008 | https://matrix.echo6.co | Matrix homeserver (Docker, SSO) |
|
||||
| Element Web | Contabo | 127.0.0.1:8088 | https://element.echo6.co | Matrix web client (Docker) |
|
||||
| [[mautrix_signal]] | Contabo | internal (29328) | DM @signalbot:echo6.co | Signal bridge (Docker, E2BE, MSC4190, double puppeting) |
|
||||
| Matrix [[synapse]] | edge2 (CT 106) | 100.64.0.37:8008 | https://matrix.echo6.co | Matrix homeserver (Docker, SSO) — **migrated from Contabo 2026-06-18** |
|
||||
| Element Web | edge2 (CT 106) | 100.64.0.37:8088 | https://element.echo6.co | Matrix web client (Docker) — **migrated from Contabo 2026-06-18** |
|
||||
| [[mautrix_signal]] | edge2 (CT 106) | internal (29328) | DM @signalbot:echo6.co | Signal bridge (Docker, E2BE, MSC4190, double puppeting) — **migrated from Contabo 2026-06-18** |
|
||||
| LiveSync | edge2 (CT 104) | 100.64.0.35:5984 (CouchDB) / :5985 (provisioner) | https://notes.echo6.co | Obsidian sync (CouchDB + provisioner, Docker, JWT auth) — fronted by Contabo Caddy (reverse_proxy 100.64.0.35:5984 / :5985); **migrated from Contabo 2026-06-16** |
|
||||
| OpenTAKServer (OTS) | utility (CT 109) | 192.168.1.109:443 | https://ots.k7zvx.com | Live TAK server (native install, nginx+RabbitMQ+PostgreSQL, Meshtastic MQTT gateway on port 8883) — see [[ots-setup]] |
|
||||
| Echo6 Cortex Agent | cortex (VM 150) | N/A (Matrix bot) | #cortex:echo6.co in echo6-ops space | Claude Code bridge — @cortex:echo6.co, session continuity, E2EE (systemd) |
|
||||
| mautrix-signal | Contabo | 29328 (internal) | Internal (matrix-net) | Signal bridge — @signalbot:echo6.co, E2BE, MSC4190, auto-portals |
|
||||
| Matrix MAS | Contabo | 127.0.0.1:8085 | Internal (via Caddy) | Matrix Authentication Service (Docker, handles login/logout/OIDC for Synapse) |
|
||||
| Matrix MAS | edge2 (CT 106) | 100.64.0.37:8085 | Internal (via Caddy) | Matrix Authentication Service (Docker, handles login/logout/OIDC for Synapse) — **migrated from Contabo 2026-06-18** |
|
||||
| Termix | Contabo | 0.0.0.0:8083 | Internal (no Caddy block) | Terminal sharing tool (Docker, ghcr.io/lukegus/termix:latest) |
|
||||
| [[archivist]] | utility (CT 118) | 192.168.1.118 | Internal | Signal/Matrix room archive bot (systemd) — see archivist.md for details |
|
||||
| pt-transcoder | cortex (VM 150) | N/A | Internal | PeerTube H.265 NVENC transcoder (systemd, /opt/bulk-import/transcoder.py) |
|
||||
|
|
@ -343,6 +343,33 @@ updated: 2026-06-18
|
|||
- Reboot-survival fix: systemd unit on CT 105 gates `docker compose up` on `tailscale-online` (Docker was racing Tailscale on boot, failing the bind to the tailnet IP)
|
||||
- **Resources:** 2 cores / 4096 MB RAM / 512 MB swap / 20 GB rootfs on `local`; unprivileged; onboot; Docker
|
||||
|
||||
### edge2 - CT 106 (10.10.10.24 / Tailscale: 100.64.0.37, node `matrix`)
|
||||
- Matrix stack (https://matrix.echo6.co, https://element.echo6.co — **migrated from Contabo 2026-06-18**)
|
||||
- Headscale hostname `matrix`, tailnet IP 100.64.0.37
|
||||
- Compose path: `/opt/matrix/matrix-stack.service` (systemd service with tailscale-before-docker reboot guard)
|
||||
- Containers: `synapse` (:8008), `matrix-mas` (:8085), `element` (:8088), `matrix-postgres`, `mautrix-signal` (29328 internal) — 5 containers total
|
||||
- Databases: PostgreSQL DBs `synapse`, `mas`, `mautrix_signal` on `matrix-postgres` container — 3 DBs total
|
||||
- Binds to `100.64.0.37`; Contabo Caddy proxies:
|
||||
- `matrix.echo6.co` → Synapse (`100.64.0.37:8008`) + MAS sub-routes
|
||||
- `element.echo6.co` → Element (`100.64.0.37:8088`)
|
||||
- MAS (`100.64.0.37:8085`) handles login/logout/refresh/auth_metadata
|
||||
- Federation delegation from apex `echo6.co` (`.well-known`) unchanged
|
||||
- MAS→Authentik OIDC (`auth.echo6.co`) unchanged
|
||||
- Signal bridge: `@signalbot:echo6.co`, E2BE + MSC4190, double puppeting; bridge state preserved in `mautrix_signal` DB
|
||||
- Source on Contabo STOPPED but intact as cold rollback; `/etc/caddy/Caddyfile.bak-pre-matrix` exists
|
||||
- **Resources:** 2 cores / 4096 MB RAM / 1024 MB swap / 20 GB rootfs on `local`; unprivileged; onboot; Docker
|
||||
|
||||
### edge2 - CT 107 (10.10.10.25 / Tailscale: 100.64.0.38, node `headscale`)
|
||||
- Headscale + Headplane (https://vpn.echo6.co — **migrated from Contabo 2026-06-19**)
|
||||
- Headscale hostname `headscale`, tailnet IP 100.64.0.38
|
||||
- Compose path: `/opt/headscale/` (managed via `headscale-stack.service` systemd reboot guard)
|
||||
- Headscale 0.28.0 on `:8084`; Headplane on `:3100`
|
||||
- Contabo Caddy proxies `vpn.echo6.co` → `100.64.0.38:8084` (headscale) / `vpn.echo6.co/admin` → `100.64.0.38:3100` (headplane)
|
||||
- `noise_private.key` carried byte-for-byte (server identity preserved); 39 nodes
|
||||
- Public direct bind `5.189.158.149:8084` DROPPED — all clients use `vpn.echo6.co` via Caddy
|
||||
- Source on Contabo STOPPED but intact as cold rollback; `/etc/caddy/Caddyfile.bak-preheadscale` exists
|
||||
- **Resources:** 1 core / 512 MB RAM / 256 MB swap / 4 GB rootfs on `local`; unprivileged; onboot
|
||||
|
||||
### edge2 - CT 102 (10.10.10.20 / Tailscale: 100.64.0.33, node 45 `vaultwarden`)
|
||||
- Vaultwarden password manager (port 8086, https://vault.echo6.co, Docker)
|
||||
- Headscale node id 45, name `vaultwarden`, user `echo6`
|
||||
|
|
@ -355,33 +382,17 @@ updated: 2026-06-18
|
|||
### Contabo VPS (5.189.158.149 / Tailscale: 100.64.0.1)
|
||||
- ~~Authentik~~ — **migrated to edge2 CT 105 on 2026-06-18** (Caddy now proxies auth.echo6.co + notes.echo6.co outpost/forward_auth → 100.64.0.36:9000; source STOPPED at `/opt/authentik`, intact as cold rollback; `/etc/caddy/Caddyfile.bak-pre-authentik` exists)
|
||||
- Forge (Git) — **migrated to edge2 CT 103 on 2026-06-16** (Caddy now proxies to 100.64.0.34:3001; SSH DNAT via forgejo-ssh-dnat.service)
|
||||
- Headscale (mesh VPN)
|
||||
- ~~Headscale~~ — **migrated to edge2 CT 107 on 2026-06-19** (Caddy now proxies vpn.echo6.co → 100.64.0.38:8084 / headplane → 100.64.0.38:3100; noise_private.key carried byte-for-byte; direct public bind 5.189.158.149:8084 DROPPED; source STOPPED at `/opt/headscale`, intact as cold rollback; `/etc/caddy/Caddyfile.bak-preheadscale` exists)
|
||||
- Mailcow (email)
|
||||
- Vaultwarden — **migrated to edge2 CT 102 on 2026-06-16** (Caddy now proxies to 100.64.0.33:8086)
|
||||
- Syncthing (syncs with cortex)
|
||||
- ~~WATCHTOWER~~ — **decommissioned 2026-06-16** (was `/opt/watchtower`; archived to forge.echo6.co/matt/archive-watchtower; `wt.echo6.co` Caddy block + GoDaddy A record removed)
|
||||
- Matrix Synapse homeserver (port 8008, https://matrix.echo6.co, Docker, SSO via Authentik)
|
||||
- Element Web client (port 8088, https://element.echo6.co, Docker)
|
||||
- mautrix-signal bridge (port 29328 internal, Docker, E2BE with MSC4190)
|
||||
- Image: `dock.mau.dev/mautrix/signal:v0.2603.0`
|
||||
- Container: `mautrix-signal` on `matrix-net`
|
||||
- Config: `/opt/matrix/mautrix-signal/config.yaml`
|
||||
- Registration: `/opt/matrix/synapse/registration.yaml` + `/opt/matrix/synapse/doublepuppet.yaml`
|
||||
- Database: `mautrix_signal` on `matrix-postgres` (role: `mautrix_signal`, minimal grants)
|
||||
- Bot user: `@signalbot:echo6.co`, device: `UPX4KKLZVY`
|
||||
- Permissions: `@matt:echo6.co` = admin, `echo6.co` = user
|
||||
- Double puppeting: appservice-based (doublepuppet.yaml as_token)
|
||||
- Encryption: E2BE enabled (allow+default+require), MSC4190, self-signed cross-signing keys
|
||||
- Compose path: `/opt/matrix/docker-compose.yml`
|
||||
- Backup: daily at 3AM, 14-day retention (synapse + mas + mautrix_signal databases)
|
||||
- ~~Matrix Synapse~~ — **migrated to edge2 CT 106 on 2026-06-18** (Caddy now proxies matrix.echo6.co → 100.64.0.37:8008 / element.echo6.co → 100.64.0.37:8088 / MAS → 100.64.0.37:8085; source STOPPED at `/opt/matrix`, intact as cold rollback; `/etc/caddy/Caddyfile.bak-pre-matrix` exists)
|
||||
- ~~Element Web~~ — **migrated to edge2 CT 106 on 2026-06-18** (see Matrix entry above)
|
||||
- ~~mautrix-signal bridge~~ — **migrated to edge2 CT 106 on 2026-06-18** (see Matrix entry above; Signal bridge state preserved in mautrix_signal DB)
|
||||
- ~~LiveSync~~ — **migrated to edge2 CT 104 on 2026-06-16** (source `/opt/livesync` STOPPED, intact as cold rollback; Caddy now proxies `notes.echo6.co` → `100.64.0.35:5984/5985`; `/etc/caddy/Caddyfile.bak-prelivesync` exists)
|
||||
- ~~TAK Server~~ — **decommissioned 2026-06-16** (was `/opt/tak-server-deploy`; archived to forge.echo6.co/matt/archive-tak-server; `tak.echo6.co` Caddy block + GoDaddy A record removed)
|
||||
- ~~SIGIL~~ — **decommissioned 2026-06-16** (was `/opt/sigil`; archived to forge.echo6.co/matt/archive-tak-server)
|
||||
- Matrix Authentication Service (MAS) (port 8085, internal, Docker)
|
||||
- Container: `matrix-mas` on `matrix-net`
|
||||
- Handles login/logout/refresh/auth_metadata for Synapse
|
||||
- Caddy routes: `/_matrix/client/*/login`, `/_matrix/client/*/logout`, `/_matrix/client/*/refresh`, `/_matrix/client/*/auth_metadata` → MAS (8085); `/_matrix/*` and `/_synapse/*` → Synapse (8008); default → MAS (8085)
|
||||
- Compose: `/opt/matrix/docker-compose.yml` (shared with Synapse stack)
|
||||
- Termix (port 8083, internal, Docker)
|
||||
- Container: `termix` on `termix_default` network
|
||||
- Image: `ghcr.io/lukegus/termix:latest`
|
||||
|
|
@ -391,16 +402,6 @@ updated: 2026-06-18
|
|||
- Compose: `/opt/termix/` (inferred from Docker volume naming)
|
||||
- ~~echo6-agent~~ — **decommissioned 2026-06-16** (was systemd `echo6-agent.service` at `/opt/echo6-agent/`; archived to forge.echo6.co/matt/archive-echo6-agent; unit disabled)
|
||||
- ~~nexus-hub~~ + ~~nexus-agent~~ — **decommissioned 2026-06-16** (were systemd units at `/root/nexus-hub` and `/root/nexus-agent`; archived to forge.echo6.co/matt/archive-nexus-hub + archive-nexus-agent; units disabled)
|
||||
- mautrix-signal bridge (mautrix-signal container, port 29328 internal)
|
||||
- Image: dock.mau.dev/mautrix/signal:v0.2603.0
|
||||
- Config: `/opt/matrix/mautrix-signal/config.yaml`
|
||||
- Compose: `/opt/matrix/docker-compose.yml` (shared with Synapse stack)
|
||||
- DB: mautrix_signal on matrix-postgres
|
||||
- Bot: @signalbot:echo6.co, management room !fDjIRTMjxILVQoAcEN:echo6.co
|
||||
- E2BE enabled (MSC4190), double puppeting via doublepuppet.yaml
|
||||
- Signal account: +12083080811 (@matt:echo6.co)
|
||||
- Portals auto-create on incoming messages (no autocreate toggle available)
|
||||
- Ref: `/home/zvx/projects/.ref/docs/matrix/mautrix_signal.md`
|
||||
|
||||
## Decommissioned Services
|
||||
|
||||
|
|
|
|||
|
|
@ -10,11 +10,11 @@ related:
|
|||
- [[headscale-onboard-node]]
|
||||
- [[caddy]]
|
||||
- [[expose-service-contabo]]
|
||||
updated: 2026-06-18
|
||||
updated: 2026-06-19
|
||||
---
|
||||
# LXC Service Migration — Contabo → edge2
|
||||
|
||||
> Proven pilots: **Vaultwarden → edge2 CT 102** (SQLite, 2026-06-16), **Forgejo → edge2 CT 103** (PostgreSQL + non-Caddy SSH port, 2026-06-16), **LiveSync (CouchDB) → edge2 CT 104** (cold named-volume tar + bind-mounted config, 2026-06-16), and **[[authentik]] (PostgreSQL keystone) → edge2 CT 105** (SECRET_KEY-must-travel, multi-block [[caddy]] cutover across 2 site blocks, reboot tailscale-before-docker race, 2026-06-18). This runbook generalizes these patterns into a reusable template for evacuating any Contabo-Caddy-fronted service to an edge2 LXC.
|
||||
> Proven pilots: **Vaultwarden → edge2 CT 102** (SQLite, 2026-06-16), **Forgejo → edge2 CT 103** (PostgreSQL + non-Caddy SSH port, 2026-06-16), **LiveSync (CouchDB) → edge2 CT 104** (cold named-volume tar + bind-mounted config, 2026-06-16), **[[authentik]] (PostgreSQL keystone) → edge2 CT 105** (SECRET_KEY-must-travel, multi-block [[caddy]] cutover across 2 site blocks, reboot tailscale-before-docker race, 2026-06-18), **Matrix stack → edge2 CT 106** (multi-DB Postgres + stateful Signal bridge, 5 containers, 2026-06-18), and **Headscale → edge2 CT 107** (tailnet control plane, noise_private.key must travel, 2026-06-19). This runbook generalizes these patterns into a reusable template for evacuating any Contabo-Caddy-fronted service to an edge2 LXC.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -443,6 +443,7 @@ ssh root@100.64.0.1 'systemctl disable --now <service>-ssh-dnat.service && rm /e
|
|||
| G14 | **Reboot race — Docker binding to the tailnet IP can start before Tailscale is online, failing the bind and leaving the service unreachable after a reboot.** Fix: create a systemd unit on the CT that runs `docker compose up` and has `After=tailscale-online.target` + `Requires=tailscale-online.target` (or equivalent `tailscale status --wait` pre-check). Alternatively, `restart: unless-stopped` in the compose file will cause Docker to self-heal via restarts, but the service will be unreachable for the first ~10–30 s after reboot. Verify reboot survival explicitly (Phase 8). Proven required for Authentik (CT 105, 2026-06-18). |
|
||||
| G15 | **Do NOT change the dnsmasq split-DNS entry during cutover.** The dnsmasq entry for `<service>.echo6.co` points at the Caddy/TLS host (100.64.0.1 = Contabo), NOT the backend. Only the Caddy upstream changes. Repointing dnsmasq to the backend tailnet IP would break internal HTTPS (no cert, no TLS termination). The Caddy host is always the internal DNS target; the backend IP only appears in the Caddy `reverse_proxy` directive. |
|
||||
| G16 | **SECRET_KEY and session-signing material must travel byte-for-byte for keystone/session-bearing [[services]]** (e.g. Authentik `AUTHENTIK_SECRET_KEY`, Vaultwarden `rsa_key.pem`). Carrying them verbatim means existing browser sessions survive the cutover — users drop straight in with no forced re-login. If the key is regenerated on the target, all active sessions are invalidated immediately. Confirm from startup logs that no new key was generated. |
|
||||
| G17 | **Tailnet control-plane migration pattern (proven: Headscale → edge2 CT 107, 2026-06-19).** The data-plane (all 39 registered nodes' WireGuard tunnels) survives a brief control-plane outage — existing node connections stay up; only new key exchanges and re-auths are blocked during the gap. Key material: `noise_private.key` MUST travel byte-for-byte (this is the server's WireGuard identity; losing it forces every node to re-auth). Out-of-band re-entry path: Contabo public IP `5.189.158.149` and edge2 public IP `184.174.35.153` are reachable even when the tailnet is broken — always confirm these work before cutover. **Cutover MUST run inside a `tmux` session on a host reachable via public IP** (not only via tailnet), implementing a self-verifying auto-rollback: if the target fails its health gate within a deadline, the script automatically restores the source Caddy block and restarts the source — so a dropped SSH cannot leave the backbone broken mid-cutover. Drop the public direct-bind (`5.189.158.149:8084`) only after Caddy-proxy reachability is confirmed. |
|
||||
| N | The composed **Contabo-Caddy → edge2-LXC tailnet** path is unexercised for each new service. Keep the Phase 6 HTTP `/alive` 200 gate as a HARD pre-cutover requirement (use `curl`, not ICMP). |
|
||||
|
||||
---
|
||||
|
|
@ -460,4 +461,4 @@ ssh root@100.64.0.1 'systemctl disable --now <service>-ssh-dnat.service && rm /e
|
|||
|
||||
---
|
||||
|
||||
*Last updated: 2026-06-18 — Added Authentik pilot (PostgreSQL keystone, multi-block Caddy cutover across 2 site blocks, reboot tailscale-before-docker race); Phase 7 multi-block example; Gotchas G14 (reboot race), G15 (dnsmasq must not be repointed), G16 (SECRET_KEY must travel); previously added LiveSync/CouchDB pilot (Variant D, G11-G13)*
|
||||
*Last updated: 2026-06-19 — Added Matrix pilot (multi-DB Postgres + stateful Signal bridge, CT 106) and Headscale pilot (tailnet control plane, noise_private.key must travel, CT 107); Gotcha G17 (tailnet-control-plane migration pattern: data-plane survives outage, out-of-band re-entry via public IPs, self-verifying auto-rollback in tmux); previously added Authentik pilot (G14–G16)*
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue