Migration: consolidate Echo6 docs to cortex with full infrastructure cleanup sync
- Documents recent infrastructure cleanup (8 CTs destroyed, 35 DNS records removed, Headscale cleanup) - Adds 24 new runbooks covering Authentik, PeerTube, Meshtastic, RECON, Proxmox, Mailcow, Internet Archive, GPU routing - Adds project documentation for headscale, vaultwarden, peertube, matrix, mmud, advbbs, arr stack - Updates services.md, environment.md, caddy.md, authentik.md to match live infrastructure - Removes 4 deprecated runbook duplicates (canonical versions live in projects/) - Adds .gitignore for binary archives and editor temp files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
89834796ff
commit
e9231ac24a
93 changed files with 51223 additions and 254 deletions
147
synapse_retention_discovery.ref
Normal file
147
synapse_retention_discovery.ref
Normal file
|
|
@ -0,0 +1,147 @@
|
|||
# Synapse Retention Discovery
|
||||
# Generated: 2026-04-09 (Phase 6.0, Question 1)
|
||||
|
||||
## Current homeserver.yaml Values
|
||||
|
||||
### APPLIED 2026-04-12 (Archivist Phase 2)
|
||||
|
||||
Two settings added to homeserver.yaml:
|
||||
|
||||
```diff
|
||||
report_stats: false
|
||||
|
||||
+redaction_retention_period: null
|
||||
+
|
||||
trusted_key_servers:
|
||||
- server_name: "matrix.org"
|
||||
```
|
||||
|
||||
```diff
|
||||
experimental_features:
|
||||
msc3202_transaction_extensions: true
|
||||
msc2409_to_device_messages_enabled: true
|
||||
+ msc2815_enabled: true
|
||||
```
|
||||
|
||||
- Backup: `/opt/matrix/synapse/homeserver.yaml.bak-20260412`
|
||||
- Applied: 2026-04-12 03:06 UTC
|
||||
- Synapse restarted, health verified, bridge reconnected, Element login confirmed
|
||||
- Rollback: restore backup and `docker compose restart synapse`
|
||||
|
||||
### Previous state (before 2026-04-12)
|
||||
|
||||
| Setting | Previous Value | Current Value | Source |
|
||||
|---------|---------------|---------------|--------|
|
||||
| redaction_retention_period | NOT SET (default 7d) | null (disabled) | synapse/config/server.py |
|
||||
| msc2815_enabled | NOT SET (default false) | true | synapse/config/experimental.py |
|
||||
| forgotten_room_retention_period | NOT SET | NOT SET (unchanged) | synapse/config/server.py |
|
||||
| media_retention.local_media_lifetime | NOT SET | NOT SET (unchanged) | synapse/config/repository.py |
|
||||
| media_retention.remote_media_lifetime | NOT SET | NOT SET (unchanged) | synapse/config/repository.py |
|
||||
|
||||
Full experimental_features block (current):
|
||||
```yaml
|
||||
experimental_features:
|
||||
msc3202_transaction_extensions: true
|
||||
msc2409_to_device_messages_enabled: true
|
||||
msc2815_enabled: true
|
||||
```
|
||||
|
||||
## What `redaction_retention_period: null` Changes
|
||||
|
||||
Source: Synapse v1.147.1 synapse/config/server.py, synapse/storage/databases/main/events.py
|
||||
|
||||
When set to `null`:
|
||||
- Synapse STOPS censoring (overwriting) redacted event content in the DB
|
||||
- The original unredacted content stays in event_json table forever
|
||||
- The 5-minute censoring job still runs but skips all events
|
||||
- Redactions still WORK from the client perspective — clients see events as redacted
|
||||
- The only difference is the server retains the pre-redaction content internally
|
||||
|
||||
When at default `7d`:
|
||||
- After 7 days post-redaction, Synapse replaces stored event content with the redacted form
|
||||
- Original content is permanently lost from the DB
|
||||
- Any tool querying the DB after 7 days gets only the stripped event
|
||||
|
||||
## MSC2815 Support — CONFIRMED in Synapse 1.147.1
|
||||
|
||||
Config key: `experimental_features.msc2815_enabled` (default: false)
|
||||
Source: synapse/config/experimental.py line 372
|
||||
Feature flag advertised as: `fi.mau.msc2815: true` in /_matrix/client/versions
|
||||
|
||||
What MSC2815 does:
|
||||
- Adds `include_unredacted_content=true` query param to GET /rooms/{roomId}/event/{eventId}
|
||||
- Requires requester to have power level >= room's redact PL (default 50)
|
||||
- Returns original unredacted content if still in DB
|
||||
- Returns FI.MAU.MSC2815_UNREDACTED_CONTENT_DELETED if content was already censored
|
||||
|
||||
Critical interaction: MSC2815 + redaction_retention_period
|
||||
- With default 7d: MSC2815 only works for 7 days after redaction
|
||||
- With null: MSC2815 works forever (content never censored)
|
||||
|
||||
## Current DB Sizes
|
||||
|
||||
| Database | Size |
|
||||
|----------|------|
|
||||
| synapse | 259 MB |
|
||||
| mas | 17 MB |
|
||||
| mautrix_signal | 14 MB |
|
||||
|
||||
Top 5 tables in synapse:
|
||||
| Table | Size |
|
||||
|-------|------|
|
||||
| state_groups_state | 91 MB |
|
||||
| event_json | 20 MB |
|
||||
| events | 15 MB |
|
||||
| event_auth | 14 MB |
|
||||
| device_lists_changes_in_room | 12 MB |
|
||||
|
||||
## Growth Rate Analysis
|
||||
|
||||
Backup sizes over 14 days (Mar 26 - Apr 9, gzipped):
|
||||
- Mar 26: 5.28 MB → Apr 9: 5.41 MB
|
||||
- Delta: ~130 KB compressed over 14 days = ~9 KB/day compressed
|
||||
- Uncompressed estimate: ~50-100 KB/day growth
|
||||
|
||||
Event volume:
|
||||
- Total events: 2,617 across 18 rooms
|
||||
- Last 7 days: 2,356 events (bridge just deployed, bulk of these are portal creation)
|
||||
- Steady-state will be much lower — Signal groups average 10-50 messages/day
|
||||
- Only 3 redaction events in entire history
|
||||
|
||||
## Storage Impact Estimate (disabling redaction purge)
|
||||
|
||||
Current scale:
|
||||
- 2,617 events = 259 MB total DB (but most of that is state, not event content)
|
||||
- event_json table: 20 MB for 2,617 events = ~7.6 KB avg per event
|
||||
- Redactions are 3 out of 2,617 (0.1%)
|
||||
|
||||
Even at 10x Signal bridge traffic (100 messages/day across all groups):
|
||||
- 100 events/day × 7.6 KB = 760 KB/day event_json growth
|
||||
- Redactions typically <5% of messages = <5 events/day × 7.6 KB = 38 KB/day saved by NOT censoring
|
||||
- Over 1 year: ~14 MB retained from not censoring
|
||||
|
||||
Verdict: Storage impact of `redaction_retention_period: null` is NEGLIGIBLE.
|
||||
The entire Synapse DB after months of use is 259 MB. Even with aggressive Signal
|
||||
bridge traffic, retaining redacted content adds <15 MB/year.
|
||||
|
||||
## MAS Interaction with Redacted Content
|
||||
|
||||
MAS does NOT interact with redacted content. MAS handles:
|
||||
- Authentication (login/logout/refresh)
|
||||
- Token management
|
||||
- Upstream OIDC delegation
|
||||
|
||||
MAS has no audit policy, no event content access, and no retention policy of its own.
|
||||
Redaction handling is entirely within Synapse's event store. No conflict.
|
||||
|
||||
## Rollback Procedure
|
||||
|
||||
If `redaction_retention_period: null` and `msc2815_enabled: true` are added and need reverting:
|
||||
|
||||
1. Remove both settings from homeserver.yaml
|
||||
2. Restart Synapse: `docker compose restart synapse`
|
||||
3. Synapse returns to default 7d censoring
|
||||
4. Already-retained content will be censored within 7 days + 5 minutes
|
||||
5. No DB migration needed — Synapse's censoring job handles cleanup automatically
|
||||
|
||||
The rollback is clean and non-destructive.
|
||||
Loading…
Add table
Add a link
Reference in a new issue