docs: migrate Authentik (SSO keystone) to edge2 CT 105
- Authentik -> edge2 CT 105 (Postgres pg_dump/restore; SECRET_KEY carried verbatim; zero-downtime until ~2s cutover) - Multi-block Caddy cutover: auth.echo6.co + notes.echo6.co outpost/forward_auth -> 100.64.0.36:9000 - runbook: add reboot tailscale-before-docker gotcha; clarify dnsmasq must NOT be repointed (points at Caddy host) - source left stopped + intact on Contabo as cold rollback Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
30f70793f8
commit
44f0257376
140 changed files with 4013 additions and 24 deletions
235
vault/docs/matrix/archivist.md
Normal file
235
vault/docs/matrix/archivist.md
Normal file
|
|
@ -0,0 +1,235 @@
|
|||
# Signal Archive Bot — Deployment Reference
|
||||
# Created: 2026-04-12 (Phase 3)
|
||||
# Status: Phase 5 COMPLETE — bot deployed, transcripts writing, sync token dedup verified
|
||||
|
||||
## CT 118 — archivist
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| VMID | 118 |
|
||||
| Hostname | archivist |
|
||||
| Host node | utility (192.168.1.241) |
|
||||
| IP | 192.168.1.118/24 |
|
||||
| Gateway | 192.168.1.1 |
|
||||
| OS | Debian 12 (bookworm) |
|
||||
| Disk | 8GB (local-lvm:vm-118-disk-0) |
|
||||
| RAM | 1024 MB |
|
||||
| Swap | 512 MB |
|
||||
| Cores | 1 |
|
||||
| Unprivileged | Yes |
|
||||
| Features | keyctl=1 |
|
||||
| Onboot | Yes |
|
||||
| MAC | BC:24:11:74:E9:DC |
|
||||
|
||||
## NFS Mount
|
||||
|
||||
### In-container NFS mount: FAILED
|
||||
- Attempt 1 (no mount=nfs feature): `access denied by server`
|
||||
- Attempt 2 (with mount=nfs feature): `Operation not permitted`
|
||||
- Root cause: unprivileged LXC containers cannot mount NFS directly
|
||||
|
||||
### Final approach: host-side NFS + Proxmox bind mount
|
||||
- **Utility host fstab:** `192.168.1.245:/export/library /mnt/library nfs defaults,soft,timeo=150 0 0`
|
||||
- **CT 118 mp0:** `/mnt/library,mp=/mnt/library,ro=0`
|
||||
- **Pattern source:** recon-vm (VM 1130, RECON) on data node uses identical approach
|
||||
- Utility host did NOT have /mnt/library mounted before this deployment
|
||||
|
||||
### Write access
|
||||
- `/mnt/library/` permissions: 2777 (drwxrwsrwx), owner nobody:nogroup
|
||||
- `/mnt/library/signal-archive/` permissions: 777 (drwxrwxrwx), created by CT root
|
||||
- Archivist user UID 999 (container) maps to UID 100999 (host/NFS)
|
||||
- Write access verified: archivist can create dirs, write files, delete within signal-archive/
|
||||
- No NFS export changes needed — world-writable parent dir permits all UIDs
|
||||
|
||||
## Service User
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| Username | archivist |
|
||||
| UID | 999 |
|
||||
| GID | 996 |
|
||||
| Shell | /usr/sbin/nologin |
|
||||
| Home | /opt/archivist |
|
||||
| Host-mapped UID | 100999 |
|
||||
| Host-mapped GID | 100996 |
|
||||
|
||||
## Directory Layout
|
||||
|
||||
```
|
||||
/opt/archivist/ # Home dir (owned by archivist:archivist)
|
||||
archivist.py # Main bot script (Phase 5)
|
||||
.env # Environment variables (Phase 5)
|
||||
/store/ # E2EE key store + sync token (SqliteStore)
|
||||
/logs/ # Bot logs (archivist.log)
|
||||
/venv/ # Python virtual environment
|
||||
|
||||
/mnt/library/signal-archive/ # Transcript output (NFS bind mount)
|
||||
<room-slug>/
|
||||
transcript.log # Append-only human-readable transcript
|
||||
media/ # Downloaded media files (images, audio, video, etc.)
|
||||
```
|
||||
|
||||
## Python Environment
|
||||
|
||||
| Component | Version |
|
||||
|-----------|---------|
|
||||
| Python | 3.11.2 |
|
||||
| pip | 26.0.1 |
|
||||
| matrix-nio | 0.25.2 (with e2e extras) |
|
||||
| python-olm | 3.2.16 |
|
||||
| libolm-dev | 3.2.13~dfsg-1 |
|
||||
| aiohttp | 3.13.5 |
|
||||
|
||||
Venv path: `/opt/archivist/venv/`
|
||||
|
||||
### Verified imports
|
||||
- `nio.AsyncClient` — Matrix client
|
||||
- `nio.crypto.OlmDevice` — E2EE device management
|
||||
- `olm.Account` — libolm C binding
|
||||
- `nio.store.SqliteStore` — crypto key persistence
|
||||
|
||||
## System Packages
|
||||
|
||||
Installed via apt:
|
||||
- python3, python3-venv, python3-pip, python3-dev
|
||||
- libolm-dev
|
||||
- gcc, g++, make
|
||||
- nfs-common
|
||||
- curl, ca-certificates
|
||||
|
||||
## Access
|
||||
|
||||
```bash
|
||||
# From utility host
|
||||
pct exec 118 -- bash
|
||||
|
||||
# Direct SSH (not configured yet — no SSH keys installed)
|
||||
# ssh root@192.168.1.118
|
||||
```
|
||||
|
||||
## Matrix User — @archivist:echo6.co
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| User ID | @archivist:echo6.co |
|
||||
| Display name | Archivist Bot |
|
||||
| Created via | mas-cli (MAS user existed from 2026-04-10, password set via `manage set-password`) |
|
||||
| Password | `<REDACTED — see credentials file: MATRIX_ARCHIVIST_BOT_PASSWORD>` |
|
||||
| Device ID | ARCHIVIST |
|
||||
| Access token | Stable compat token via `mas-cli manage issue-compatibility-token archivist ARCHIVIST` |
|
||||
| Admin | No (not needed — uses room invitation, not admin force-join) |
|
||||
|
||||
### E2EE / Cross-Signing
|
||||
|
||||
| Component | Status |
|
||||
|-----------|--------|
|
||||
| Device keys (curve25519 + ed25519) | Uploaded |
|
||||
| One-time keys (signed_curve25519) | 50 uploaded |
|
||||
| Master key | Published (nKFt5nA+TvUo0AvY1gsk1QL8OK1t9z/ChON30Kdvlek) |
|
||||
| Self-signing key | Published (9gOB+AHgyBzLP5/xerYon04NLZuIh+o5OHAybmetK2A) |
|
||||
| User-signing key | Published (1EYngPiwpjOy2aQWY02g5SZaFLM5kZgsHLhFXgwHh0Q) |
|
||||
| Device self-signed | Yes (ARCHIVIST signed by self-signing key) |
|
||||
| Cross-signing seeds | /opt/archivist/store/cross_signing_seeds.json (chmod 600) |
|
||||
| nio store | /opt/archivist/store/@archivist:echo6.co_ARCHIVIST.db |
|
||||
|
||||
### Key Sharing — How It Works
|
||||
|
||||
- Bridge key sharing policy: `cross-signed-tofu`
|
||||
- ARCHIVIST device is cross-signed → bridge shares Megolm session keys automatically
|
||||
- **Interactive verification (SAS emoji) is NOT required** — cross-signing alone is sufficient
|
||||
- Old messages (before archivist joined) remain undecryptable (Megolm keys not retroactively shared)
|
||||
- New messages are decryptable immediately
|
||||
|
||||
### E2BE Decryption Test — PASSED
|
||||
|
||||
- Date: 2026-04-12 14:35 UTC
|
||||
- Room: COMMS LP group (!XUeWZuPdWQQnUYLJBJ:echo6.co)
|
||||
- Message: "You'll know that radio has come of age when the median cellphone incorporates a LoRa radio stack."
|
||||
- Sender: @signal_cdf98bca-c4b7-4fda-8ceb-03db5eb4e7e2:echo6.co (Signal puppet via bridge)
|
||||
- Result: Successfully decrypted by ARCHIVIST device
|
||||
|
||||
### User Creation Notes
|
||||
|
||||
- Shared-secret registration (`/_synapse/admin/v1/register`) returns 404 under MAS — endpoint disabled
|
||||
- Must use `mas-cli manage register-user` or `manage set-password` for existing users
|
||||
- MAS creates user in both MAS DB and Synapse DB
|
||||
- Orphaned Synapse `profiles` row caused provisioning failure — fixed by DELETE
|
||||
- Each `client.login()` creates a NEW MAS compat session with random device ID — use `restore_login()` with stable compat token instead
|
||||
- matrix-nio v0.25.2 does NOT implement `bootstrap_cross_signing()` — manual implementation required via python-olm PkSigning + raw HTTP API
|
||||
|
||||
## Joined Rooms
|
||||
|
||||
| Room | Room ID | Type | Archive Slug |
|
||||
|------|---------|------|-------------|
|
||||
| COMMS LP group | !XUeWZuPdWQQnUYLJBJ:echo6.co | Bridged Signal group | comms-lp-group |
|
||||
| DM with Matt | !wgbnqhnYKTHzzJMjDu:echo6.co | Direct message | — |
|
||||
| Liberal_Preppers_OG | !RvWNPmcKtPImhKPYcA:echo6.co | Bridged Signal group | liberal-preppers-og |
|
||||
| (3 additional rooms) | !vBXtbgfYcptEuimrmn, !SnGDZgBtYOQuTWeYXp, !aQWFQMrzbkwjyjCPte | Bridged Signal groups | (initialized on first message) |
|
||||
|
||||
## Scripts on CT 118
|
||||
|
||||
| Script | Purpose | Status |
|
||||
|--------|---------|--------|
|
||||
| /opt/archivist/archivist.py | Main bot — transcript writer | **Running in tmux** |
|
||||
| /opt/archivist/login_once.py | One-shot login + key upload | Completed (superseded) |
|
||||
| /opt/archivist/bootstrap_crosssigning.py | Cross-signing key bootstrap | Completed (one-time) |
|
||||
| /opt/archivist/setup_and_verify.py | Device setup + verification listener | Completed |
|
||||
| /opt/archivist/test_decrypt.py | E2BE decryption test listener | Completed (superseded by archivist.py) |
|
||||
|
||||
## Bot Architecture (Phase 5)
|
||||
|
||||
### archivist.py — Event-Driven Transcript Bot (~260 lines)
|
||||
|
||||
**Core design:** Single-file async Python bot using matrix-nio `sync_forever` with `ClientConfig(store_sync_tokens=True)` for restart deduplication.
|
||||
|
||||
**Callbacks:**
|
||||
- `on_invite` (InviteMemberEvent) → auto-join
|
||||
- `on_text` (RoomMessageText) → write transcript line, detect edits via `m.replace`
|
||||
- `on_image/audio/video/file` (RoomMessage* + RoomEncrypted*) → download + decrypt + save to media/
|
||||
- `on_sticker` (StickerEvent) → same as media
|
||||
- `on_redaction` (RedactionEvent) → log deletion with original content if cached
|
||||
- `on_megolm` (MegolmEvent) → log decryption failure (counter per room)
|
||||
|
||||
**In-memory caches (NOT persisted):**
|
||||
- `room_slugs: dict[str, str]` — room_id → slug (rebuilt from transcript headers on startup)
|
||||
- `event_cache: dict[str, dict]` — event_id → {body, sender, ts} (for edit/redact tracking)
|
||||
- `name_cache: dict[str, str]` — mxid → display name (Signal ghosts get "(Signal)" suffix)
|
||||
|
||||
**Sync token persistence:** matrix-nio SqliteStore handles save/load automatically when `store_sync_tokens=True`. On restart, `loaded_sync_token` resumes from last position — no event replay.
|
||||
|
||||
**Encrypted media handling:** RoomEncryptedImage/Audio/Video/File carry `key`, `hashes`, `iv` attributes. Bot downloads ciphertext via `client.download(mxc=url)`, then decrypts with `nio.crypto.decrypt_attachment()`.
|
||||
|
||||
### Transcript Format
|
||||
|
||||
```
|
||||
# Transcript: Room Display Name
|
||||
# Room ID: !xxxxx:echo6.co
|
||||
# Archive started: 2026-04-12 20:47:01 UTC
|
||||
# ---
|
||||
|
||||
[2026-04-12 18:55:34 UTC] Sender Name (Signal): Message text
|
||||
[2026-04-12 18:56:00 UTC] Sender Name (Signal): [EDITED] New text
|
||||
(was: Original text)
|
||||
[2026-04-12 18:57:00 UTC] Sender Name (Signal): [DELETED] (was: Original text)
|
||||
[2026-04-12 18:58:00 UTC] Sender Name (Signal): [image: media/1234567890_filename.jpg] caption
|
||||
```
|
||||
|
||||
### Running the Bot
|
||||
|
||||
```bash
|
||||
# Start (tmux, as archivist user)
|
||||
pct exec 118 -- su -s /bin/bash archivist -c "tmux new-session -d -s archivist /opt/archivist/venv/bin/python3 -u /opt/archivist/archivist.py"
|
||||
|
||||
# Check logs
|
||||
pct exec 118 -- tail -f /opt/archivist/logs/archivist.log
|
||||
|
||||
# Stop
|
||||
pct exec 118 -- su -s /bin/bash archivist -c "tmux send-keys -t archivist C-c"
|
||||
```
|
||||
|
||||
## What's NOT done yet (Phase 6+)
|
||||
|
||||
- No systemd service (running in tmux)
|
||||
- No Tailscale registration
|
||||
- No SSH key auth configured
|
||||
- Bot needs invitations to additional bridged rooms as they appear
|
||||
17
vault/docs/matrix/matrix_host.md
Normal file
17
vault/docs/matrix/matrix_host.md
Normal file
|
|
@ -0,0 +1,17 @@
|
|||
# Matrix Host Reference — Contabo VPS
|
||||
# Generated: 2026-04-09 (Phase 0)
|
||||
|
||||
## SSH Access
|
||||
- Host: 100.64.0.1 (Tailscale) / 5.189.158.149 (public)
|
||||
- Auth: SSH key as root — CONFIRMED WORKING
|
||||
- sudo: root user, no sudo needed
|
||||
|
||||
## System Info
|
||||
- OS: Ubuntu 24.04.3 LTS (Noble Numbat)
|
||||
- Kernel: 6.8.0-90-generic
|
||||
- CPU: AMD EPYC (12 vCPUs)
|
||||
- RAM: 47 GB total, ~35 GB available
|
||||
- Disk: /dev/sda1 968 GB, 76 GB used, 893 GB available (8%)
|
||||
- Docker Root: /var/lib/docker
|
||||
- Docker: 29.0.2
|
||||
- Docker Compose: v2.40.3
|
||||
103
vault/docs/matrix/mautrix_signal.md
Normal file
103
vault/docs/matrix/mautrix_signal.md
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
# mautrix-signal Bridge Reference
|
||||
# Deployed: 2026-04-09 (Phase 3)
|
||||
# Updated: 2026-04-09 (Phase 5)
|
||||
|
||||
## Deployment
|
||||
|
||||
- **Image:** dock.mau.dev/mautrix/signal:v0.2603.0
|
||||
- **Container:** mautrix-signal
|
||||
- **Compose:** /opt/matrix/docker-compose.yml (same stack as Synapse)
|
||||
- **Config dir:** /opt/matrix/mautrix-signal/
|
||||
- **Network:** matrix-net (internal only, no host port mapping)
|
||||
- **Appservice port:** 29328 (container-internal)
|
||||
|
||||
## Database
|
||||
|
||||
- **DB name:** mautrix_signal
|
||||
- **DB role:** mautrix_signal (NOSUPERUSER, NOCREATEDB, NOCREATEROLE)
|
||||
- **Host:** matrix-postgres:5432 (same container as Synapse/MAS)
|
||||
- **Collation:** C/C (matches Synapse)
|
||||
|
||||
## Signal Account
|
||||
|
||||
- **Phone:** +12083080811
|
||||
- **Login ID:** 58f99d83-f3a8-487f-a2b7-3d118e236d23
|
||||
- **Matrix user:** @matt:echo6.co
|
||||
|
||||
## Bot
|
||||
|
||||
- **MXID:** @signalbot:echo6.co
|
||||
- **Device ID:** UPX4KKLZVY
|
||||
- **Management room:** !fDjIRTMjxILVQoAcEN:echo6.co
|
||||
|
||||
## Encryption
|
||||
|
||||
- **E2BE:** enabled (allow: true, default: true, require: true)
|
||||
- **MSC4190:** enabled (required for MAS compatibility)
|
||||
- **Pickle key:** auto-generated, stored in config
|
||||
- **Self-sign:** true
|
||||
|
||||
## Synapse Integration
|
||||
|
||||
- **Registration files:** /opt/matrix/synapse/registration.yaml, /opt/matrix/synapse/doublepuppet.yaml
|
||||
- **homeserver.yaml additions:**
|
||||
- `app_service_config_files: [/data/registration.yaml, /data/doublepuppet.yaml]`
|
||||
- `experimental_features.msc3202_transaction_extensions: true`
|
||||
- `experimental_features.msc2409_to_device_messages_enabled: true`
|
||||
- **Registration file permissions:** 644 (world-readable, Synapse UID 991)
|
||||
|
||||
## Double Puppeting
|
||||
|
||||
- **Method:** appservice-based (doublepuppet.yaml)
|
||||
- **as_token:** configured in bridge config `double_puppet.secrets`
|
||||
- **Effect:** Messages matt sends from Signal appear as @matt:echo6.co in Matrix
|
||||
|
||||
## MAS Compatibility
|
||||
|
||||
- **Confirmed working:** as_token auth bypasses MAS entirely
|
||||
- **MSC4190:** Parsed from registration.yaml `io.element.msc4190: true`, NOT a homeserver experimental_features flag
|
||||
- **Synapse handles appservice auth independently of MAS token introspection**
|
||||
|
||||
## Autocreate Behavior
|
||||
|
||||
- **No autocreate toggle exists** in bridgev2 signal connector
|
||||
- `portal_create_filter` config key is silently stripped on restart
|
||||
- Old keys (`autocreate_group_portal`, `autocreate_contact_portal`) don't exist
|
||||
- Portals auto-create when incoming Signal messages arrive for untracked groups/DMs
|
||||
- This is the default and only behavior — cannot be disabled
|
||||
|
||||
## Permissions
|
||||
|
||||
```yaml
|
||||
bridge:
|
||||
permissions:
|
||||
"*": relay
|
||||
"echo6.co": user
|
||||
"@matt:echo6.co": admin
|
||||
```
|
||||
|
||||
## Backup
|
||||
|
||||
- DB dumped by /opt/matrix/scripts/pg_backup.sh (added alongside synapse + mas dumps)
|
||||
- 14-day retention, daily at 3 AM
|
||||
|
||||
## Management Commands
|
||||
|
||||
Available in management room (!fDjIRTMjxILVQoAcEN:echo6.co):
|
||||
- `!signal help` — list commands
|
||||
- `!signal ping-matrix` — verify Matrix connectivity
|
||||
- `!signal list-logins` — show linked Signal accounts
|
||||
- `!signal sync-portal` — force-sync current portal
|
||||
- `!signal id` — show bridge IDs for current room
|
||||
- `!signal create-group` — create a new Signal group
|
||||
- `!signal start-chat` — start a DM with a Signal user
|
||||
- `!signal resolve-identifier` — look up a Signal user
|
||||
|
||||
NOT available (bridgev2 signal connector doesn't expose):
|
||||
- `list-chats`, `bridge`, `create-portal`, `filter`
|
||||
|
||||
## Provisioning API
|
||||
|
||||
- **Shared secret:** in config.yaml (do NOT log or put in ref files)
|
||||
- **Allow Matrix auth:** true
|
||||
- **Debug endpoints:** false
|
||||
79
vault/docs/matrix/synapse.md
Normal file
79
vault/docs/matrix/synapse.md
Normal file
|
|
@ -0,0 +1,79 @@
|
|||
# Synapse Deployment Reference
|
||||
# Generated: 2026-04-09 (Phase 1)
|
||||
|
||||
## Install Method
|
||||
- Docker Compose at /opt/matrix/docker-compose.yml
|
||||
- Four containers on `matrix-net` bridge network:
|
||||
- matrix-synapse (matrixdotorg/synapse:latest) → 127.0.0.1:8008
|
||||
- matrix-mas (ghcr.io/element-hq/matrix-authentication-service:latest) → 127.0.0.1:8085
|
||||
- matrix-postgres (postgres:16-alpine) → internal 5432
|
||||
- matrix-element (vectorim/element-web:latest) → 127.0.0.1:8088
|
||||
|
||||
## Synapse Version
|
||||
- 1.147.1 (confirmed via admin API and docker exec)
|
||||
|
||||
## Key Config Values (from homeserver.yaml)
|
||||
- server_name: echo6.co
|
||||
- public_baseurl: https://matrix.echo6.co/
|
||||
- listeners: port 8008, HTTP, x_forwarded=true, bind 0.0.0.0, resources=[client, federation]
|
||||
- database: psycopg2 → matrix-postgres:5432, db=synapse, user=synapse
|
||||
- media_store_path: /data/media_store (95 MB used)
|
||||
- registration_shared_secret: PRESENT (not redacted here — see synapse_homeserver.yaml.sanitized)
|
||||
- signing_key_path: /data/echo6.co.signing.key
|
||||
- enable_registration: false
|
||||
- url_preview_enabled: true
|
||||
- report_stats: false
|
||||
- app_service_config_files: NOT PRESENT (no appservices currently registered)
|
||||
- encryption_enabled_by_default_for_room_type: NOT SET (default=off)
|
||||
- MAS delegation: enabled, endpoint=http://matrix-mas:8080/, shared secret present
|
||||
|
||||
## MAS (Matrix Authentication Service)
|
||||
- Version: v1.12.0
|
||||
- Listen: 8080 (web), 8081 (internal/health)
|
||||
- Database: postgresql://mas:***@matrix-postgres:5432/mas
|
||||
- Upstream OAuth2: Authentik (auth.echo6.co) via OIDC
|
||||
- Client ID: 93kCoZkBlnJyD9EcAm7E4btKflecOcBm9DGONB5T
|
||||
- Issuer: https://auth.echo6.co/application/o/matrix/
|
||||
- Matrix integration: kind=synapse, homeserver=echo6.co, endpoint=http://matrix-synapse:8008/
|
||||
- Passwords: enabled (bcrypt v1 + argon2id v2)
|
||||
- Email transport: blackhole (not sending)
|
||||
|
||||
## Database (PostgreSQL)
|
||||
- Version: 16.12 (Alpine)
|
||||
- Host: matrix-postgres container on matrix-net
|
||||
- Databases: synapse (owner: synapse), mas (owner: mas), postgres, template0, template1
|
||||
- Users: synapse (Superuser, Create role, Create DB), mas (regular)
|
||||
- Auth: password from .env file (POSTGRES_PASSWORD)
|
||||
- New DB/user can be created without collision — synapse user has Superuser/Create role privileges
|
||||
|
||||
## Reverse Proxy (Caddy on Contabo)
|
||||
- matrix.echo6.co routes:
|
||||
- /_matrix/client/*/login|logout|refresh|auth_metadata → MAS (127.0.0.1:8085)
|
||||
- /_matrix/* → Synapse (127.0.0.1:8008)
|
||||
- /_synapse/* → Synapse (127.0.0.1:8008)
|
||||
- Everything else → MAS (127.0.0.1:8085)
|
||||
- element.echo6.co → 127.0.0.1:8088
|
||||
|
||||
## Federation
|
||||
- Well-known served from echo6.co (utility Caddy, NOT matrix.echo6.co)
|
||||
- /.well-known/matrix/server: {"m.server": "matrix.echo6.co:443"}
|
||||
- /.well-known/matrix/client: base_url=https://matrix.echo6.co, issuer=https://matrix.echo6.co/
|
||||
- Federation tester: AllChecksOK=true, TLS 1.3, valid Ed25519 key, valid certificates
|
||||
- Port 8448 exposed in container but NOT used externally (federation via 443 + well-known)
|
||||
|
||||
## Existing Appservices
|
||||
- NONE — no app_service_config_files in homeserver.yaml, no registration files in /opt/matrix/synapse/
|
||||
|
||||
## Backup
|
||||
- Cron: daily at 3 AM via /opt/matrix/scripts/pg_backup.sh
|
||||
- Backs up synapse DB only (NOT mas DB)
|
||||
- Retention: 14 days
|
||||
- Location: /opt/matrix/backups/
|
||||
|
||||
## Ports in 29xxx Range
|
||||
- NONE in use — confirmed via ss -tlnp
|
||||
|
||||
## MSC4190 Support
|
||||
- Synapse 1.147.1 supports MSC4190 (merged in 1.121.0)
|
||||
- Requires io.element.msc4190: true in appservice registration file
|
||||
- Also requires experimental_features.msc3202_transaction_extensions: true in homeserver.yaml
|
||||
54
vault/docs/matrix/synapse_homeserver.yaml.sanitized
Normal file
54
vault/docs/matrix/synapse_homeserver.yaml.sanitized
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
server_name: "echo6.co"
|
||||
public_baseurl: "https://matrix.echo6.co/"
|
||||
pid_file: /data/homeserver.pid
|
||||
|
||||
listeners:
|
||||
- port: 8008
|
||||
type: http
|
||||
tls: false
|
||||
x_forwarded: true
|
||||
bind_addresses: ["0.0.0.0"]
|
||||
resources:
|
||||
- names: [client, federation]
|
||||
compress: false
|
||||
|
||||
database:
|
||||
name: psycopg2
|
||||
args:
|
||||
user: synapse
|
||||
password: <REDACTED>
|
||||
database: synapse
|
||||
host: matrix-postgres
|
||||
port: 5432
|
||||
cp_min: 5
|
||||
cp_max: 10
|
||||
|
||||
log_config: "/data/echo6.co.log.config"
|
||||
media_store_path: /data/media_store
|
||||
enable_registration: false
|
||||
url_preview_enabled: true
|
||||
url_preview_ip_range_blacklist:
|
||||
- "127.0.0.0/8"
|
||||
- "10.0.0.0/8"
|
||||
- "172.16.0.0/12"
|
||||
- "192.168.0.0/16"
|
||||
- "100.64.0.0/10"
|
||||
- "169.254.0.0/16"
|
||||
- "::1/128"
|
||||
- "fe80::/10"
|
||||
- "fc00::/7"
|
||||
|
||||
registration_shared_secret: <REDACTED>
|
||||
macaroon_secret_key: <REDACTED>
|
||||
form_secret: <REDACTED>
|
||||
signing_key_path: "/data/echo6.co.signing.key"
|
||||
report_stats: false
|
||||
|
||||
trusted_key_servers:
|
||||
- server_name: "matrix.org"
|
||||
|
||||
# MAS delegation - replaces oidc_providers
|
||||
matrix_authentication_service:
|
||||
enabled: true
|
||||
endpoint: http://matrix-mas:8080/
|
||||
secret: <REDACTED>
|
||||
147
vault/docs/matrix/synapse_retention_discovery.md
Normal file
147
vault/docs/matrix/synapse_retention_discovery.md
Normal file
|
|
@ -0,0 +1,147 @@
|
|||
# Synapse Retention Discovery
|
||||
# Generated: 2026-04-09 (Phase 6.0, Question 1)
|
||||
|
||||
## Current homeserver.yaml Values
|
||||
|
||||
### APPLIED 2026-04-12 (Archivist Phase 2)
|
||||
|
||||
Two settings added to homeserver.yaml:
|
||||
|
||||
```diff
|
||||
report_stats: false
|
||||
|
||||
+redaction_retention_period: null
|
||||
+
|
||||
trusted_key_servers:
|
||||
- server_name: "matrix.org"
|
||||
```
|
||||
|
||||
```diff
|
||||
experimental_features:
|
||||
msc3202_transaction_extensions: true
|
||||
msc2409_to_device_messages_enabled: true
|
||||
+ msc2815_enabled: true
|
||||
```
|
||||
|
||||
- Backup: `/opt/matrix/synapse/homeserver.yaml.bak-20260412`
|
||||
- Applied: 2026-04-12 03:06 UTC
|
||||
- Synapse restarted, health verified, bridge reconnected, Element login confirmed
|
||||
- Rollback: restore backup and `docker compose restart synapse`
|
||||
|
||||
### Previous state (before 2026-04-12)
|
||||
|
||||
| Setting | Previous Value | Current Value | Source |
|
||||
|---------|---------------|---------------|--------|
|
||||
| redaction_retention_period | NOT SET (default 7d) | null (disabled) | synapse/config/server.py |
|
||||
| msc2815_enabled | NOT SET (default false) | true | synapse/config/experimental.py |
|
||||
| forgotten_room_retention_period | NOT SET | NOT SET (unchanged) | synapse/config/server.py |
|
||||
| media_retention.local_media_lifetime | NOT SET | NOT SET (unchanged) | synapse/config/repository.py |
|
||||
| media_retention.remote_media_lifetime | NOT SET | NOT SET (unchanged) | synapse/config/repository.py |
|
||||
|
||||
Full experimental_features block (current):
|
||||
```yaml
|
||||
experimental_features:
|
||||
msc3202_transaction_extensions: true
|
||||
msc2409_to_device_messages_enabled: true
|
||||
msc2815_enabled: true
|
||||
```
|
||||
|
||||
## What `redaction_retention_period: null` Changes
|
||||
|
||||
Source: Synapse v1.147.1 synapse/config/server.py, synapse/storage/databases/main/events.py
|
||||
|
||||
When set to `null`:
|
||||
- Synapse STOPS censoring (overwriting) redacted event content in the DB
|
||||
- The original unredacted content stays in event_json table forever
|
||||
- The 5-minute censoring job still runs but skips all events
|
||||
- Redactions still WORK from the client perspective — clients see events as redacted
|
||||
- The only difference is the server retains the pre-redaction content internally
|
||||
|
||||
When at default `7d`:
|
||||
- After 7 days post-redaction, Synapse replaces stored event content with the redacted form
|
||||
- Original content is permanently lost from the DB
|
||||
- Any tool querying the DB after 7 days gets only the stripped event
|
||||
|
||||
## MSC2815 Support — CONFIRMED in Synapse 1.147.1
|
||||
|
||||
Config key: `experimental_features.msc2815_enabled` (default: false)
|
||||
Source: synapse/config/experimental.py line 372
|
||||
Feature flag advertised as: `fi.mau.msc2815: true` in /_matrix/client/versions
|
||||
|
||||
What MSC2815 does:
|
||||
- Adds `include_unredacted_content=true` query param to GET /rooms/{roomId}/event/{eventId}
|
||||
- Requires requester to have power level >= room's redact PL (default 50)
|
||||
- Returns original unredacted content if still in DB
|
||||
- Returns FI.MAU.MSC2815_UNREDACTED_CONTENT_DELETED if content was already censored
|
||||
|
||||
Critical interaction: MSC2815 + redaction_retention_period
|
||||
- With default 7d: MSC2815 only works for 7 days after redaction
|
||||
- With null: MSC2815 works forever (content never censored)
|
||||
|
||||
## Current DB Sizes
|
||||
|
||||
| Database | Size |
|
||||
|----------|------|
|
||||
| synapse | 259 MB |
|
||||
| mas | 17 MB |
|
||||
| mautrix_signal | 14 MB |
|
||||
|
||||
Top 5 tables in synapse:
|
||||
| Table | Size |
|
||||
|-------|------|
|
||||
| state_groups_state | 91 MB |
|
||||
| event_json | 20 MB |
|
||||
| events | 15 MB |
|
||||
| event_auth | 14 MB |
|
||||
| device_lists_changes_in_room | 12 MB |
|
||||
|
||||
## Growth Rate Analysis
|
||||
|
||||
Backup sizes over 14 days (Mar 26 - Apr 9, gzipped):
|
||||
- Mar 26: 5.28 MB → Apr 9: 5.41 MB
|
||||
- Delta: ~130 KB compressed over 14 days = ~9 KB/day compressed
|
||||
- Uncompressed estimate: ~50-100 KB/day growth
|
||||
|
||||
Event volume:
|
||||
- Total events: 2,617 across 18 rooms
|
||||
- Last 7 days: 2,356 events (bridge just deployed, bulk of these are portal creation)
|
||||
- Steady-state will be much lower — Signal groups average 10-50 messages/day
|
||||
- Only 3 redaction events in entire history
|
||||
|
||||
## Storage Impact Estimate (disabling redaction purge)
|
||||
|
||||
Current scale:
|
||||
- 2,617 events = 259 MB total DB (but most of that is state, not event content)
|
||||
- event_json table: 20 MB for 2,617 events = ~7.6 KB avg per event
|
||||
- Redactions are 3 out of 2,617 (0.1%)
|
||||
|
||||
Even at 10x Signal bridge traffic (100 messages/day across all groups):
|
||||
- 100 events/day × 7.6 KB = 760 KB/day event_json growth
|
||||
- Redactions typically <5% of messages = <5 events/day × 7.6 KB = 38 KB/day saved by NOT censoring
|
||||
- Over 1 year: ~14 MB retained from not censoring
|
||||
|
||||
Verdict: Storage impact of `redaction_retention_period: null` is NEGLIGIBLE.
|
||||
The entire Synapse DB after months of use is 259 MB. Even with aggressive Signal
|
||||
bridge traffic, retaining redacted content adds <15 MB/year.
|
||||
|
||||
## MAS Interaction with Redacted Content
|
||||
|
||||
MAS does NOT interact with redacted content. MAS handles:
|
||||
- Authentication (login/logout/refresh)
|
||||
- Token management
|
||||
- Upstream OIDC delegation
|
||||
|
||||
MAS has no audit policy, no event content access, and no retention policy of its own.
|
||||
Redaction handling is entirely within Synapse's event store. No conflict.
|
||||
|
||||
## Rollback Procedure
|
||||
|
||||
If `redaction_retention_period: null` and `msc2815_enabled: true` are added and need reverting:
|
||||
|
||||
1. Remove both settings from homeserver.yaml
|
||||
2. Restart Synapse: `docker compose restart synapse`
|
||||
3. Synapse returns to default 7d censoring
|
||||
4. Already-retained content will be censored within 7 days + 5 minutes
|
||||
5. No DB migration needed — Synapse's censoring job handles cleanup automatically
|
||||
|
||||
The rollback is clean and non-destructive.
|
||||
Loading…
Add table
Add a link
Reference in a new issue