Migration: consolidate Echo6 docs to cortex with full infrastructure cleanup sync
- Documents recent infrastructure cleanup (8 CTs destroyed, 35 DNS records removed, Headscale cleanup) - Adds 24 new runbooks covering Authentik, PeerTube, Meshtastic, RECON, Proxmox, Mailcow, Internet Archive, GPU routing - Adds project documentation for headscale, vaultwarden, peertube, matrix, mmud, advbbs, arr stack - Updates services.md, environment.md, caddy.md, authentik.md to match live infrastructure - Removes 4 deprecated runbook duplicates (canonical versions live in projects/) - Adds .gitignore for binary archives and editor temp files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
89834796ff
commit
e9231ac24a
93 changed files with 51223 additions and 254 deletions
469
projects/matrix-synapse-deployment.md
Normal file
469
projects/matrix-synapse-deployment.md
Normal file
|
|
@ -0,0 +1,469 @@
|
|||
# Matrix Synapse Deployment
|
||||
|
||||
**Status:** Deployed 2026-02-15, migrated to Contabo 2026-02-15
|
||||
**Target:** Contabo VPS (5.189.158.149 / 100.64.0.1)
|
||||
**URLs:** https://matrix.echo6.co (Synapse), https://element.echo6.co (Element Web)
|
||||
**Server Name:** echo6.co (federated identity: @user:echo6.co)
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
| Component | Detail |
|
||||
|-----------|--------|
|
||||
| Host | Contabo VPS (5.189.158.149 / 100.64.0.1) |
|
||||
| Docker services | Synapse (127.0.0.1:8008), Element Web (127.0.0.1:8088), PostgreSQL 16 |
|
||||
| Reverse proxy | Contabo Caddy (auto ACME certs) |
|
||||
| SSO | Authentik OIDC → communication-users group |
|
||||
| Federation | Well-known delegation on echo6.co base domain (served by utility Caddy) |
|
||||
| Compose path | `/opt/matrix/docker-compose.yml` |
|
||||
| Backup | Daily at 3AM, 14-day retention, `/opt/matrix/backups/` |
|
||||
|
||||
The server name is `echo6.co` (not `matrix.echo6.co`) so federated user IDs are `@user:echo6.co`. The Synapse instance lives at `matrix.echo6.co` and delegation is handled via `.well-known` endpoints on the base domain.
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Provision LXC Container
|
||||
|
||||
Run **ct-runbook.md** on the utility node with these parameters:
|
||||
|
||||
```
|
||||
CTID=108
|
||||
HOSTNAME=matrix
|
||||
STORAGE=local-lvm
|
||||
DISK_SIZE=16
|
||||
MEMORY=2048
|
||||
CORES=2
|
||||
BRIDGE=vmbr0
|
||||
```
|
||||
|
||||
After the runbook completes (user, SSH, Docker, Tailscale all verified), continue here.
|
||||
|
||||
---
|
||||
|
||||
## Step 2: Create Project Structure
|
||||
|
||||
SSH into CT 108:
|
||||
|
||||
```bash
|
||||
CT_IP=$(ssh root@192.168.1.241 "pct exec 108 -- hostname -I | awk '{print \$1}'")
|
||||
sshpass -p '7redditGold' ssh zvx@$CT_IP
|
||||
```
|
||||
|
||||
Create directories:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/matrix/{synapse,postgres,element,backups,scripts}
|
||||
sudo chown -R zvx:zvx /opt/matrix
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Create Docker Compose
|
||||
|
||||
Create `/opt/matrix/docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
container_name: matrix-postgres
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_DB: synapse
|
||||
POSTGRES_USER: synapse
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
||||
POSTGRES_INITDB_ARGS: "--encoding=UTF8 --lc-collate=C --lc-ctype=C"
|
||||
volumes:
|
||||
- ./postgres:/var/lib/postgresql/data
|
||||
networks:
|
||||
- matrix-net
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U synapse -d synapse"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
synapse:
|
||||
image: matrixdotorg/synapse:latest
|
||||
container_name: matrix-synapse
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
SYNAPSE_CONFIG_PATH: /data/homeserver.yaml
|
||||
volumes:
|
||||
- ./synapse:/data
|
||||
ports:
|
||||
- "8008:8008"
|
||||
networks:
|
||||
- matrix-net
|
||||
|
||||
element:
|
||||
image: vectorim/element-web:latest
|
||||
container_name: matrix-element
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ./element/config.json:/app/config.json:ro
|
||||
ports:
|
||||
- "8080:80"
|
||||
networks:
|
||||
- matrix-net
|
||||
|
||||
networks:
|
||||
matrix-net:
|
||||
driver: bridge
|
||||
```
|
||||
|
||||
Create `/opt/matrix/.env`:
|
||||
|
||||
```bash
|
||||
POSTGRES_PASSWORD=$(openssl rand -base64 24 | tr -d '/+=' | head -c 32)
|
||||
echo "POSTGRES_PASSWORD=$POSTGRES_PASSWORD" > /opt/matrix/.env
|
||||
chmod 600 /opt/matrix/.env
|
||||
echo "Save this password to /home/zvx/projects/.ref/credentials"
|
||||
cat /opt/matrix/.env
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Generate Synapse Config
|
||||
|
||||
```bash
|
||||
cd /opt/matrix
|
||||
docker run -it --rm \
|
||||
-v ./synapse:/data \
|
||||
-e SYNAPSE_SERVER_NAME=echo6.co \
|
||||
-e SYNAPSE_REPORT_STATS=no \
|
||||
matrixdotorg/synapse:latest generate
|
||||
```
|
||||
|
||||
Edit `synapse/homeserver.yaml` — replace the full `database` section and add OIDC config:
|
||||
|
||||
```yaml
|
||||
server_name: "echo6.co"
|
||||
public_baseurl: "https://matrix.echo6.co/"
|
||||
|
||||
listeners:
|
||||
- port: 8008
|
||||
type: http
|
||||
tls: false
|
||||
x_forwarded: true
|
||||
bind_addresses: ['0.0.0.0']
|
||||
resources:
|
||||
- names: [client, federation]
|
||||
compress: false
|
||||
|
||||
database:
|
||||
name: psycopg2
|
||||
args:
|
||||
user: synapse
|
||||
password: <POSTGRES_PASSWORD from .env>
|
||||
database: synapse
|
||||
host: matrix-postgres
|
||||
port: 5432
|
||||
cp_min: 5
|
||||
cp_max: 10
|
||||
|
||||
media_store_path: /data/media_store
|
||||
enable_registration: false
|
||||
url_preview_enabled: true
|
||||
|
||||
# Authentik OIDC — fill client_id and client_secret after running authentik-oidc-application.md
|
||||
oidc_providers:
|
||||
- idp_id: authentik
|
||||
idp_name: "Echo6 SSO"
|
||||
discover: true
|
||||
issuer: "https://auth.echo6.co/application/o/matrix/"
|
||||
client_id: "<from authentik-oidc-application.md>"
|
||||
client_secret: "<from authentik-oidc-application.md>"
|
||||
scopes: ["openid", "profile", "email"]
|
||||
user_mapping_provider:
|
||||
config:
|
||||
localpart_template: "{{ user.preferred_username }}"
|
||||
display_name_template: "{{ user.name }}"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Configure Element Web
|
||||
|
||||
Create `/opt/matrix/element/config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"default_server_config": {
|
||||
"m.homeserver": {
|
||||
"base_url": "https://matrix.echo6.co",
|
||||
"server_name": "echo6.co"
|
||||
}
|
||||
},
|
||||
"brand": "Echo6 Chat",
|
||||
"disable_guests": true,
|
||||
"disable_3pid_login": false
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 6: Start Services
|
||||
|
||||
```bash
|
||||
cd /opt/matrix
|
||||
docker compose up -d
|
||||
docker compose ps
|
||||
```
|
||||
|
||||
Wait for Synapse to initialize the database (watch logs):
|
||||
|
||||
```bash
|
||||
docker compose logs -f synapse
|
||||
# Wait for "Synapse now listening on TCP port 8008"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 7: Expose via Caddy and DNS
|
||||
|
||||
Run **expose-service-home.md** twice — once for `matrix.echo6.co` and once for `element.echo6.co`.
|
||||
|
||||
This service has OIDC, so use local IP per the runbook's decision table.
|
||||
|
||||
### matrix.echo6.co
|
||||
|
||||
- Backend: `192.168.1.108:8008` (local IP, has OIDC)
|
||||
- Issue cert, install cert, add Caddy site block, add GoDaddy DNS
|
||||
|
||||
Caddy site block (note the path-based routing for Matrix):
|
||||
|
||||
```caddyfile
|
||||
matrix.echo6.co {
|
||||
tls /etc/caddy/certs/matrix.echo6.co.fullchain.crt /etc/caddy/certs/matrix.echo6.co.key
|
||||
reverse_proxy /_matrix/* 192.168.1.108:8008
|
||||
reverse_proxy /_synapse/* 192.168.1.108:8008
|
||||
}
|
||||
```
|
||||
|
||||
### element.echo6.co
|
||||
|
||||
- Backend: `192.168.1.108:8080` (local IP)
|
||||
- Issue cert, install cert, add Caddy site block, add GoDaddy DNS
|
||||
|
||||
```caddyfile
|
||||
element.echo6.co {
|
||||
tls /etc/caddy/certs/element.echo6.co.fullchain.crt /etc/caddy/certs/element.echo6.co.key
|
||||
reverse_proxy 192.168.1.108:8080
|
||||
}
|
||||
```
|
||||
|
||||
### Well-known delegation (federation)
|
||||
|
||||
This must go on the `echo6.co` base domain. Check if there's already an `echo6.co` block in the Utility Caddy Caddyfile — if so, merge these `handle` directives into it. If not, add a new block:
|
||||
|
||||
```caddyfile
|
||||
echo6.co {
|
||||
tls /etc/caddy/certs/echo6.co.fullchain.crt /etc/caddy/certs/echo6.co.key
|
||||
|
||||
handle /.well-known/matrix/server {
|
||||
header Content-Type application/json
|
||||
respond `{"m.server": "matrix.echo6.co:443"}`
|
||||
}
|
||||
handle /.well-known/matrix/client {
|
||||
header Content-Type application/json
|
||||
header Access-Control-Allow-Origin *
|
||||
respond `{"m.homeserver": {"base_url": "https://matrix.echo6.co"}}`
|
||||
}
|
||||
|
||||
# ... any existing handlers for echo6.co ...
|
||||
}
|
||||
```
|
||||
|
||||
If `echo6.co` doesn't have a cert yet, issue one via acme.sh following the same pattern in expose-service-home.md.
|
||||
|
||||
### dnsmasq split DNS
|
||||
|
||||
Add to `/etc/dnsmasq.d/tailscale-dns.conf` on Contabo:
|
||||
|
||||
```
|
||||
address=/matrix.echo6.co/100.64.0.8
|
||||
address=/element.echo6.co/100.64.0.8
|
||||
```
|
||||
|
||||
Both point to the Utility Caddy Tailscale IP (100.64.0.8), which proxies to CT 108.
|
||||
|
||||
Restart dnsmasq:
|
||||
|
||||
```bash
|
||||
ssh root@100.64.0.1 "systemctl restart dnsmasq"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 8: Configure Authentik SSO
|
||||
|
||||
Run **authentik-oidc-application.md** with these inputs:
|
||||
|
||||
```
|
||||
SERVICE_NAME=Matrix
|
||||
SERVICE_SLUG=matrix
|
||||
SERVICE_URL=https://matrix.echo6.co
|
||||
OIDC_CALLBACK_PATH=/_synapse/client/oidc/callback
|
||||
NEEDS_OFFLINE_ACCESS=no
|
||||
CLIENT_TYPE=confidential
|
||||
```
|
||||
|
||||
After completing the runbook, take the Client ID and Client Secret and update `synapse/homeserver.yaml` (Step 4) with the real values. Then restart Synapse:
|
||||
|
||||
```bash
|
||||
cd /opt/matrix && docker compose restart synapse
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 9: Bind Access Group
|
||||
|
||||
Run **authentik-access-groups.md** Procedure B to bind the `matrix` application to the `communication-users` group.
|
||||
|
||||
```
|
||||
APP_SLUG=matrix
|
||||
GROUP_PK=31bce176-cd86-4aea-8db3-a57e03d5c2d1 # communication-users
|
||||
```
|
||||
|
||||
This shares the same access group as Mailcow.
|
||||
|
||||
---
|
||||
|
||||
## Step 10: Create Admin User
|
||||
|
||||
```bash
|
||||
docker exec -it matrix-synapse register_new_matrix_user \
|
||||
-u matt \
|
||||
-p <secure-password> \
|
||||
-a \
|
||||
-c /data/homeserver.yaml \
|
||||
http://localhost:8008
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 11: Schedule PostgreSQL Backups
|
||||
|
||||
Run **pg-backup.md** with these inputs:
|
||||
|
||||
```
|
||||
CONTAINER_NAME=matrix-postgres
|
||||
DB_NAME=synapse
|
||||
DB_USER=synapse
|
||||
BACKUP_DIR=/opt/matrix/backups
|
||||
RETENTION_DAYS=14
|
||||
CRON_SCHEDULE="0 3 * * *"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
### Internal (from CT 108)
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:8008/_matrix/client/versions | jq .
|
||||
curl -s http://localhost:8008/_matrix/federation/v1/version | jq .
|
||||
curl -s http://localhost:8080 | head -5
|
||||
```
|
||||
|
||||
### External (from cortex or any tailnet device)
|
||||
|
||||
```bash
|
||||
curl -s https://matrix.echo6.co/_matrix/client/versions | jq .
|
||||
curl -s https://matrix.echo6.co/_matrix/federation/v1/version | jq .
|
||||
curl -sI https://element.echo6.co | head -5
|
||||
curl -s https://echo6.co/.well-known/matrix/server | jq .
|
||||
curl -s https://echo6.co/.well-known/matrix/client | jq .
|
||||
```
|
||||
|
||||
### Federation
|
||||
|
||||
```bash
|
||||
curl -s "https://federationtester.matrix.org/api/report?server_name=echo6.co" | jq '.FederationOK'
|
||||
```
|
||||
|
||||
Must return `true`.
|
||||
|
||||
### SSO
|
||||
|
||||
1. Open https://element.echo6.co
|
||||
2. Click SSO login
|
||||
3. Should redirect to auth.echo6.co → authenticate → redirect back to Element
|
||||
4. Verify user identity matches Authentik profile
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Synapse won't start
|
||||
|
||||
```bash
|
||||
docker compose logs synapse 2>&1 | tail -50
|
||||
```
|
||||
|
||||
Common causes: bad YAML indentation in homeserver.yaml, wrong PostgreSQL password, database not ready.
|
||||
|
||||
### Federation test fails
|
||||
|
||||
Check in order:
|
||||
1. `.well-known/matrix/server` returns `{"m.server": "matrix.echo6.co:443"}`
|
||||
2. `/_matrix/federation/v1/version` is accessible from the public internet
|
||||
3. Caddy is routing `/_matrix/*` paths correctly (not just root)
|
||||
4. GoDaddy DNS for `echo6.co` points to 199.6.36.163
|
||||
|
||||
### SSO login loop
|
||||
|
||||
See troubleshooting in authentik-oidc-application.md. Most common cause: missing signing key on the Authentik provider, or wrong callback path.
|
||||
|
||||
### Element can't connect
|
||||
|
||||
Verify Element's `config.json` has `base_url` set to `https://matrix.echo6.co` (not `http://`, not `localhost`).
|
||||
|
||||
---
|
||||
|
||||
## Runbook References
|
||||
|
||||
| Step | Runbook | Purpose |
|
||||
|------|---------|---------|
|
||||
| 1 | ct-runbook.md | LXC provisioning, Docker, user, SSH, Tailscale |
|
||||
| 7 | expose-service-home.md | SSL cert, Caddy site block, GoDaddy DNS |
|
||||
| 8 | authentik-oidc-application.md | Create OIDC provider + application |
|
||||
| 9 | authentik-access-groups.md | Bind communication-users group |
|
||||
| 11 | pg-backup.md | Scheduled PostgreSQL backup with retention |
|
||||
|
||||
---
|
||||
|
||||
## Credentials Reference
|
||||
|
||||
Store in `/home/zvx/projects/.ref/credentials`:
|
||||
|
||||
```
|
||||
# Matrix Synapse
|
||||
MATRIX_POSTGRES_PASSWORD=<from .env>
|
||||
MATRIX_OIDC_CLIENT_ID=<from authentik-oidc-application.md>
|
||||
MATRIX_OIDC_CLIENT_SECRET=<from authentik-oidc-application.md>
|
||||
MATRIX_OIDC_ISSUER=https://auth.echo6.co/application/o/matrix/
|
||||
MATRIX_ADMIN_USER=matt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Post-Deploy Updates
|
||||
|
||||
After deployment, update these docs:
|
||||
|
||||
- `docs/services/services.md` — add Matrix entry
|
||||
- `docs/software/caddy.md` — add matrix.echo6.co and element.echo6.co site blocks
|
||||
- `docs/software/dns.md` — note well-known delegation on echo6.co
|
||||
- `docs/hardware/environment.md` — add CT 108 to LXC table and Headscale node list
|
||||
- `runbooks/authentik-access-groups.md` — add Matrix to application bindings table
|
||||
|
||||
---
|
||||
|
||||
*Created: 2026-02-15*
|
||||
Loading…
Add table
Add a link
Reference in a new issue