echo6-docs/projects/peertube-rebuild.md

Project: PeerTube YouTube Archive Rebuild

Goal: Rebuild PeerTube at stream.echo6.co with Authentik SSO, 18TB NFS storage, and a bulk import pipeline for 250 YouTube channels (~136K videos).

Status: Phase 1 — Complete (2026-02-13). CT 110 on media, 192.168.1.170, TS 100.64.0.23, PeerTube v8.0.2

---

Architecture

                         ┌─────────────────────────────────┐
                         │         utility node             │
Internet ──── DNS ──────▶│  Caddy LXC (CT 101)             │
                         │  stream.echo6.co → PT LXC:80    │
                         └──────────────┬──────────────────┘
                                        │
                         ┌──────────────▼──────────────────┐
                         │          media node              │
                         │  PeerTube LXC (CT 100)            │
                         │    ├── nginx (port 80)           │
                         │    ├── PeerTube (port 9000)      │
                         │    ├── PostgreSQL 16              │
                         │    ├── Redis                      │
                         │    └── /var/www/peertube/storage  │
                         │           └── NFS mount (18TB)    │
                         └──────────────────────────────────┘

Authentik ◄──── OIDC ────► PeerTube

Phase 2: cortex (VM 150 on TOC, has GPU) = remote transcoding runner

Key decisions:

• LXC on media node (not VM, not Docker) — CT 100
• Privileged container (NFS bind-mount uid mapping is hell otherwise)
• Native PeerTube install (Node.js + PostgreSQL + Redis + nginx, no Docker)
• PeerTube v8.0.2 — config and nginx template may differ from v6.x docs; verify during install
• Node.js 20 (v8 requirement, not 18)
• Caddy on utility handles TLS, nginx inside LXC handles WebSocket/static files
• Caddy proxies to local IP (192.168.1.x) since PeerTube has OIDC
• Transcoding: 480p + 720p only (storage budget)
• Built-in channel sync DISABLED — bulk pipeline handles imports
• Signup disabled — Authentik SSO only
• NFS storage from pi-nas /export/peertube (separate from arr)

---

Phase 1: PeerTube Up and Secure

1.1 Provision LXC on media

Run: runbooks/ct-runbook.md with these inputs:

Variable	Value
Host	media (192.168.1.243)
CTID	100
Hostname	peertube
Template	Debian 12 (not Ubuntu — PeerTube docs target Debian)
Memory	4096 MB
Cores	4
Disk	50 GB root
Privileged	YES (override ct-runbook default)
Network	DHCP initially

Deviations from ct-runbook:

• Use Debian 12 template instead of Ubuntu 24.04
• Use privileged container (--unprivileged 0) for NFS compatibility
• Skip Docker install — PeerTube runs native
• Still do: base packages, zvx user, SSH, Tailscale

1.2 Mount NFS storage

On the media host (not inside LXC):

# Mount NFS on host
mkdir -p /mnt/peertube-storage
mount -t nfs 192.168.1.245:/export/peertube /mnt/peertube-storage

# Persist
echo "192.168.1.245:/export/peertube /mnt/peertube-storage nfs defaults,_netdev 0 0" >> /etc/fstab

# Bind-mount into LXC
echo "mp0: /mnt/peertube-storage,mp=/var/www/peertube/storage" >> /etc/pve/lxc/<CTID>.conf

# Restart LXC to pick up mount
pct stop <CTID> && pct start <CTID>

Verify inside LXC:

df -h /var/www/peertube/storage    # Should show ~18TB
touch /var/www/peertube/storage/test && rm /var/www/peertube/storage/test

NFS details:

• Server: pi-nas (192.168.1.245 / 100.64.0.21)
• Export: /export/peertube
• Access: Already configured in OMV for 100.64.0.0/10 and 192.168.1.0/24

1.3 Install PeerTube dependencies

Inside the LXC:

# PostgreSQL 16
sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | gpg --dearmor -o /etc/apt/trusted.gpg.d/postgresql.gpg
apt update && apt install -y postgresql-16 postgresql-contrib-16

sudo -u postgres psql << 'SQL'
CREATE USER peertube WITH PASSWORD '<PG_PASSWORD>';
CREATE DATABASE peertube_prod OWNER peertube;
\c peertube_prod
CREATE EXTENSION IF NOT EXISTS pg_trgm;
CREATE EXTENSION IF NOT EXISTS unaccent;
SQL

# Redis
apt install -y redis-server
sed -i 's/^# requirepass .*/requirepass <REDIS_PASSWORD>/' /etc/redis/redis.conf
sed -i 's/^bind .*/bind 127.0.0.1 -::1/' /etc/redis/redis.conf
systemctl restart redis-server && systemctl enable redis-server

# Node.js 20 (PeerTube v8 requires Node.js 20+)
curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
apt install -y nodejs
npm install -g yarn

# ffmpeg (for transcoding)
apt install -y ffmpeg

1.4 Install PeerTube

# Create peertube user
adduser --system --group --home /var/www/peertube --shell /bin/bash peertube
chown -R peertube:peertube /var/www/peertube/storage

# Get latest version (joinpeertube.org API is dead, use GitHub)
PEERTUBE_VERSION=$(curl -s https://api.github.com/repos/Chocobozzz/PeerTube/releases/latest | grep -oP '"tag_name": "v\K[^"]+' || echo "8.0.2")

# Download and install
cd /var/www/peertube
sudo -u peertube mkdir -p config
sudo -u peertube mkdir -p storage/{avatars,caches,captions,logs,plugins,previews,redundancy,streaming-playlists,thumbnails,tmp,torrents,videos,bin,storyboards,web-videos,original-video-files}
sudo -u peertube wget -q "https://github.com/Chocobozzz/PeerTube/releases/download/v${PEERTUBE_VERSION}/peertube-v${PEERTUBE_VERSION}.tar.xz"
sudo -u peertube tar xf peertube-v${PEERTUBE_VERSION}.tar.xz
sudo -u peertube ln -s peertube-v${PEERTUBE_VERSION} peertube-latest
cd peertube-latest
sudo -u peertube yarn install --production --pure-lockfile

1.5 Configure PeerTube

Create /var/www/peertube/config/local-production.json:

{
  "listen": { "hostname": "0.0.0.0", "port": 9000 },
  "webserver": { "https": true, "hostname": "stream.echo6.co", "port": 443 },
  "database": {
    "hostname": "localhost", "port": 5432,
    "name": "peertube_prod", "username": "peertube", "password": "<PG_PASSWORD>"
  },
  "redis": { "hostname": "localhost", "port": 6379, "auth": "<REDIS_PASSWORD>" },
  "storage": {
    "avatars": "/var/www/peertube/storage/avatars/",
    "caches": "/var/www/peertube/storage/caches/",
    "captions": "/var/www/peertube/storage/captions/",
    "logs": "/var/www/peertube/storage/logs/",
    "plugins": "/var/www/peertube/storage/plugins/",
    "previews": "/var/www/peertube/storage/previews/",
    "redundancy": "/var/www/peertube/storage/redundancy/",
    "streaming_playlists": "/var/www/peertube/storage/streaming-playlists/",
    "thumbnails": "/var/www/peertube/storage/thumbnails/",
    "tmp": "/var/www/peertube/storage/tmp/",
    "torrents": "/var/www/peertube/storage/torrents/",
    "videos": "/var/www/peertube/storage/videos/",
    "bin": "/var/www/peertube/storage/bin/",
    "storyboards": "/var/www/peertube/storage/storyboards/",
    "web_videos": "/var/www/peertube/storage/web-videos/",
    "original_video_files": "/var/www/peertube/storage/original-video-files/"
  },
  "admin": { "email": "admin@echo6.co" },
  "signup": { "enabled": false },
  "import": {
    "videos": { "concurrency": 10, "http": { "enabled": true }, "torrent": { "enabled": false } },
    "video_channel_synchronization": { "enabled": false }
  },
  "transcoding": {
    "enabled": true, "threads": 2, "concurrency": 2,
    "allow_additional_extensions": true, "allow_audio_files": true,
    "resolutions": {
      "0p": false, "144p": false, "240p": false, "360p": false,
      "480p": true, "720p": true,
      "1080p": false, "1440p": false, "2160p": false
    },
    "hls": { "enabled": true },
    "web_videos": { "enabled": true }
  }
}

chown peertube:peertube /var/www/peertube/config/local-production.json
chmod 600 /var/www/peertube/config/local-production.json

1.6 nginx (inside LXC)

apt install -y nginx
rm -f /etc/nginx/sites-enabled/default

cat > /etc/nginx/sites-available/peertube << 'NGINXCONF'
server {
    listen 80;
    server_name stream.echo6.co;

    add_header X-Frame-Options "SAMEORIGIN";
    add_header X-Content-Type-Options "nosniff";

    client_max_body_size 20G;
    proxy_connect_timeout 600;
    proxy_send_timeout 600;
    proxy_read_timeout 600;
    send_timeout 600;

    location ~ ^/client/(.*\.(js|css|woff2|otf|ttf|woff|eot|svg|png|jpg|gif|ico|webp))$ {
        add_header Cache-Control "public, max-age=31536000, immutable";
        alias /var/www/peertube/peertube-latest/client/dist/$1;
    }

    location ~ ^(/static/(webseed|web-videos|streaming-playlists|redundancy)/.+)$ {
        set $upstream_peertube http://127.0.0.1:9000;
        try_files /var/www/peertube/storage$1 @api;
        root /;
        add_header Cache-Control "public, max-age=7200";
    }

    location @api {
        proxy_pass http://127.0.0.1:9000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }

    location / {
        proxy_pass http://127.0.0.1:9000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        client_max_body_size 20G;
    }
}
NGINXCONF

ln -s /etc/nginx/sites-available/peertube /etc/nginx/sites-enabled/peertube
nginx -t && systemctl restart nginx && systemctl enable nginx

1.7 systemd service

cat > /etc/systemd/system/peertube.service << 'EOF'
[Unit]
Description=PeerTube daemon
After=network.target postgresql.service redis-server.service

[Service]
Type=simple
User=peertube
Group=peertube
Environment=NODE_ENV=production
Environment=NODE_CONFIG_DIR=/var/www/peertube/config
WorkingDirectory=/var/www/peertube/peertube-latest
ExecStart=/usr/bin/node dist/server
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=peertube
TimeoutStartSec=60

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable peertube
systemctl start peertube

# Grab auto-generated root password
journalctl -u peertube | grep -i "password"

1.8 Expose via Caddy

Run: runbooks/expose-service-home.md with these inputs:

Variable	Value
Service	stream
Domain	stream.echo6.co
Backend IP	PeerTube LXC local IP (192.168.1.x — has OIDC, use local IP pattern)
Backend port	80
Has OIDC	YES (use local IP, not Tailscale)

DNS record for stream.echo6.co does NOT exist in GoDaddy yet — create it pointing to 199.6.36.163.

Also update dnsmasq split DNS on Contabo — entry exists pointing to old IP 100.64.0.7, update to 100.64.0.8 (utility Caddy, same pattern as jellyfin/requests).

1.9 Authentik OIDC

Run: runbooks/authentik-oidc-application.md with these inputs:

Variable	Value
SERVICE_NAME	PeerTube
SERVICE_SLUG	peertube
SERVICE_URL	https://stream.echo6.co
OIDC_CALLBACK_PATH	Check PeerTube OIDC plugin docs — likely /plugins/auth-openid-connect/router/code-cb
NEEDS_OFFLINE_ACCESS	yes
CLIENT_TYPE	confidential

Note: No existing PeerTube provider in Authentik — create from scratch using the runbook.

Then install the plugin inside PeerTube:

cd /var/www/peertube/peertube-latest
sudo -u peertube NODE_ENV=production NODE_CONFIG_DIR=/var/www/peertube/config \
  node dist/server/tools/peertube-plugins.js install \
  --npm-name peertube-plugin-auth-openid-connect
systemctl restart peertube

Configure via Admin UI → Plugins → OpenID Connect:

• Discover URL: https://auth.echo6.co/application/o/peertube/.well-known/openid-configuration
• Client ID/Secret from Authentik
• Scope: openid email profile
• Username property: preferred_username
• Display name property: name

1.10 First login and lockdown

1. Log in as root with the auto-generated password
2. Change root password immediately
3. Test Authentik SSO login
4. Promote your Authentik user to admin
5. Admin → Configuration: instance name "Echo6 Archive", signup disabled, HTTP import enabled

Phase 1 checklist

[x] LXC on media — CT 110, privileged, Debian 12, 4C/4GB/50GB
[x] NFS 22TB mounted and writable (/export/peertube from 192.168.1.245)
[x] PostgreSQL 16 + Redis installed
[x] Node.js 22 + pnpm + ffmpeg installed (v8.0.2 requires Node 22, pnpm not yarn)
[x] PeerTube v8.0.2 installed and configured
[x] nginx configured (port 80, WebSocket, static files)
[x] systemd service running
[x] Tailscale registered (100.64.0.23)
[x] Caddy on utility proxying stream.echo6.co → 192.168.1.170:80
[x] DNS verified (GoDaddy + dnsmasq split DNS → 100.64.0.8)
[x] Authentik OIDC working (provider pk:12, app slug: peertube)
[ ] Root password changed, your user promoted to admin (manual step)

Update after Phase 1:

• docs/hardware/environment.md — add PeerTube LXC
• docs/services/services.md — add PeerTube entry
• docs/software/caddy.md — add stream.echo6.co site block

---

Phase 2: Import Pipeline

2.1 Create PeerTube channels

Script to create all 250 channels from the master spreadsheet via PeerTube API. One channel per YouTube channel, matching names.

2.2 Bulk downloader

• yt-dlp with cookies + PO tokens
• --match-filter "duration > 61" to exclude Shorts
• Round-robin across channels (5-10 videos per channel, rotate)
• Download to NFS staging area
• Track downloaded video IDs in archive file (prevent re-downloads)

2.3 Import pipeline

• Watch staging area for new downloads
• Import to correct PeerTube channel via API
• Move source file after successful import (or delete if transcoded)
• Rate limit to avoid overwhelming PeerTube

2.4 GPU transcoding on cortex

• PeerTube remote runner protocol
• cortex already has RTX A4000 + nvidia-container-toolkit
• NVENC encoding for 480p + 720p HLS
• PeerTube delegates transcoding jobs to cortex runner

Phase 2 checklist

[x] 100 channels created in PeerTube (99 planned + extras, channel-map.json at /opt/bulk-import/config/)
[x] yt-dlp configured (cookies, Shorts filter)
[x] Bulk downloader script with round-robin (pt-downloader service on CT 110)
[x] Import pipeline (pt-importer service on CT 110, resumable chunked upload)
[x] Archive tracking (downloaded.txt, downloader-state.json)
[x] GPU transcoding runner on cortex (pt-transcoder service, H.265 NVENC)
[x] PeerTube remote runner on cortex (Whisper auto-captioning, medium model, smart GPU/CPU routing)
[x] Test: full cycle — download → transcode → import → playable
[ ] VPN/IP rotation (NordVPN token pending from Matt)

---

Phase 3: Monitoring

3.1 WATCHTOWER dashboard

• Import queue depth and throughput
• Per-channel video counts vs YouTube totals
• Storage usage and growth rate
• Transcoding queue status

3.2 Alerts

• Storage threshold warnings (80%, 90%, 95%)
• Stalled imports (no progress for N hours)
• Failed downloads (rate limiting, auth issues)

Phase 3 checklist

[ ] Dashboard showing import progress
[ ] Per-channel completion tracking
[ ] Storage alerts configured
[ ] Stall detection working

---

Reference

• Master channel list: youtube_archive_master.xlsx (250 channels, 19 categories)
• PeerTube LXC: CT 110 on media (192.168.1.243)
• NFS: pi-nas (192.168.1.245) export /export/peertube
• Previous PeerTube Tailscale IP: 100.64.0.7 (do not reuse — assign fresh)
• Previous bulk import map: final-channel-map.json (lost with crash)
• Previous download archive: downloaded.txt (21,714 video IDs, lost with crash)
• Runbooks used: ct-runbook.md, expose-service-home.md, authentik-oidc-application.md
• Docs to update after Phase 1: environment.md, services.md, caddy.md, dns.md (dnsmasq entry)