echo6-docs/synapse_retention_discovery.ref
Matt Johnson e9231ac24a Migration: consolidate Echo6 docs to cortex with full infrastructure cleanup sync
- Documents recent infrastructure cleanup (8 CTs destroyed, 35 DNS records removed, Headscale cleanup)
- Adds 24 new runbooks covering Authentik, PeerTube, Meshtastic, RECON, Proxmox, Mailcow, Internet Archive, GPU routing
- Adds project documentation for headscale, vaultwarden, peertube, matrix, mmud, advbbs, arr stack
- Updates services.md, environment.md, caddy.md, authentik.md to match live infrastructure
- Removes 4 deprecated runbook duplicates (canonical versions live in projects/)
- Adds .gitignore for binary archives and editor temp files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 06:02:16 +00:00

147 lines
5.3 KiB
Text
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Synapse Retention Discovery
# Generated: 2026-04-09 (Phase 6.0, Question 1)
## Current homeserver.yaml Values
### APPLIED 2026-04-12 (Archivist Phase 2)
Two settings added to homeserver.yaml:
```diff
report_stats: false
+redaction_retention_period: null
+
trusted_key_servers:
- server_name: "matrix.org"
```
```diff
experimental_features:
msc3202_transaction_extensions: true
msc2409_to_device_messages_enabled: true
+ msc2815_enabled: true
```
- Backup: `/opt/matrix/synapse/homeserver.yaml.bak-20260412`
- Applied: 2026-04-12 03:06 UTC
- Synapse restarted, health verified, bridge reconnected, Element login confirmed
- Rollback: restore backup and `docker compose restart synapse`
### Previous state (before 2026-04-12)
| Setting | Previous Value | Current Value | Source |
|---------|---------------|---------------|--------|
| redaction_retention_period | NOT SET (default 7d) | null (disabled) | synapse/config/server.py |
| msc2815_enabled | NOT SET (default false) | true | synapse/config/experimental.py |
| forgotten_room_retention_period | NOT SET | NOT SET (unchanged) | synapse/config/server.py |
| media_retention.local_media_lifetime | NOT SET | NOT SET (unchanged) | synapse/config/repository.py |
| media_retention.remote_media_lifetime | NOT SET | NOT SET (unchanged) | synapse/config/repository.py |
Full experimental_features block (current):
```yaml
experimental_features:
msc3202_transaction_extensions: true
msc2409_to_device_messages_enabled: true
msc2815_enabled: true
```
## What `redaction_retention_period: null` Changes
Source: Synapse v1.147.1 synapse/config/server.py, synapse/storage/databases/main/events.py
When set to `null`:
- Synapse STOPS censoring (overwriting) redacted event content in the DB
- The original unredacted content stays in event_json table forever
- The 5-minute censoring job still runs but skips all events
- Redactions still WORK from the client perspective — clients see events as redacted
- The only difference is the server retains the pre-redaction content internally
When at default `7d`:
- After 7 days post-redaction, Synapse replaces stored event content with the redacted form
- Original content is permanently lost from the DB
- Any tool querying the DB after 7 days gets only the stripped event
## MSC2815 Support — CONFIRMED in Synapse 1.147.1
Config key: `experimental_features.msc2815_enabled` (default: false)
Source: synapse/config/experimental.py line 372
Feature flag advertised as: `fi.mau.msc2815: true` in /_matrix/client/versions
What MSC2815 does:
- Adds `include_unredacted_content=true` query param to GET /rooms/{roomId}/event/{eventId}
- Requires requester to have power level >= room's redact PL (default 50)
- Returns original unredacted content if still in DB
- Returns FI.MAU.MSC2815_UNREDACTED_CONTENT_DELETED if content was already censored
Critical interaction: MSC2815 + redaction_retention_period
- With default 7d: MSC2815 only works for 7 days after redaction
- With null: MSC2815 works forever (content never censored)
## Current DB Sizes
| Database | Size |
|----------|------|
| synapse | 259 MB |
| mas | 17 MB |
| mautrix_signal | 14 MB |
Top 5 tables in synapse:
| Table | Size |
|-------|------|
| state_groups_state | 91 MB |
| event_json | 20 MB |
| events | 15 MB |
| event_auth | 14 MB |
| device_lists_changes_in_room | 12 MB |
## Growth Rate Analysis
Backup sizes over 14 days (Mar 26 - Apr 9, gzipped):
- Mar 26: 5.28 MB → Apr 9: 5.41 MB
- Delta: ~130 KB compressed over 14 days = ~9 KB/day compressed
- Uncompressed estimate: ~50-100 KB/day growth
Event volume:
- Total events: 2,617 across 18 rooms
- Last 7 days: 2,356 events (bridge just deployed, bulk of these are portal creation)
- Steady-state will be much lower — Signal groups average 10-50 messages/day
- Only 3 redaction events in entire history
## Storage Impact Estimate (disabling redaction purge)
Current scale:
- 2,617 events = 259 MB total DB (but most of that is state, not event content)
- event_json table: 20 MB for 2,617 events = ~7.6 KB avg per event
- Redactions are 3 out of 2,617 (0.1%)
Even at 10x Signal bridge traffic (100 messages/day across all groups):
- 100 events/day × 7.6 KB = 760 KB/day event_json growth
- Redactions typically <5% of messages = <5 events/day × 7.6 KB = 38 KB/day saved by NOT censoring
- Over 1 year: ~14 MB retained from not censoring
Verdict: Storage impact of `redaction_retention_period: null` is NEGLIGIBLE.
The entire Synapse DB after months of use is 259 MB. Even with aggressive Signal
bridge traffic, retaining redacted content adds <15 MB/year.
## MAS Interaction with Redacted Content
MAS does NOT interact with redacted content. MAS handles:
- Authentication (login/logout/refresh)
- Token management
- Upstream OIDC delegation
MAS has no audit policy, no event content access, and no retention policy of its own.
Redaction handling is entirely within Synapse's event store. No conflict.
## Rollback Procedure
If `redaction_retention_period: null` and `msc2815_enabled: true` are added and need reverting:
1. Remove both settings from homeserver.yaml
2. Restart Synapse: `docker compose restart synapse`
3. Synapse returns to default 7d censoring
4. Already-retained content will be censored within 7 days + 5 minutes
5. No DB migration needed — Synapse's censoring job handles cleanup automatically
The rollback is clean and non-destructive.