Migration: consolidate Echo6 docs to cortex with full infrastructure cleanup sync
- Documents recent infrastructure cleanup (8 CTs destroyed, 35 DNS records removed, Headscale cleanup) - Adds 24 new runbooks covering Authentik, PeerTube, Meshtastic, RECON, Proxmox, Mailcow, Internet Archive, GPU routing - Adds project documentation for headscale, vaultwarden, peertube, matrix, mmud, advbbs, arr stack - Updates services.md, environment.md, caddy.md, authentik.md to match live infrastructure - Removes 4 deprecated runbook duplicates (canonical versions live in projects/) - Adds .gitignore for binary archives and editor temp files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
89834796ff
commit
e9231ac24a
93 changed files with 51223 additions and 254 deletions
173
runbooks/add-peertube-channel.md
Normal file
173
runbooks/add-peertube-channel.md
Normal file
|
|
@ -0,0 +1,173 @@
|
|||
# Add PeerTube Channel
|
||||
|
||||
## Overview
|
||||
|
||||
Add a YouTube channel to the PeerTube bulk import pipeline. Creates the PeerTube channel, adds to channel-map.json, and the downloader will begin syncing videos automatically.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- SSH access from CT 130 (RECON) → CT 110 (PeerTube): working
|
||||
- Sudoers: `/etc/sudoers.d/recon-mgmt` on CT 110 (allows zvx to run yt-dlp, psql, tee as peertube)
|
||||
- YouTube cookies at `/opt/bulk-import/config/cookies.txt` on CT 110 (not stale)
|
||||
|
||||
## Method 1: Web UI (Preferred)
|
||||
|
||||
1. Open **RECON Dashboard** → Upload tab: `http://192.168.1.130:8420/upload`
|
||||
2. Scroll to **PeerTube Channels** section
|
||||
3. Enter YouTube URL, category, priority
|
||||
4. Click **Add Channel**
|
||||
5. Wait for "Added: ChannelName" confirmation
|
||||
|
||||
**Note:** If the channel has members-only content, the API will automatically retry with `--ignore-errors` on the `/videos` tab.
|
||||
|
||||
## Method 2: CLI (For Troubleshooting)
|
||||
|
||||
Use when the web UI fails or you need manual control.
|
||||
|
||||
### Variables
|
||||
|
||||
```bash
|
||||
YT_URL="https://www.youtube.com/@ChannelName"
|
||||
CATEGORY="CategoryName"
|
||||
PRIORITY="M" # H, M, or L
|
||||
```
|
||||
|
||||
### Step 1: Resolve Channel Info
|
||||
|
||||
```bash
|
||||
# From CT 130 or cortex:
|
||||
ssh zvx@192.168.1.170 "sudo -u peertube /usr/local/bin/yt-dlp \
|
||||
--cookies /opt/bulk-import/config/cookies.txt \
|
||||
--print channel --print channel_url --print channel_id \
|
||||
--playlist-items 1 --skip-download '$YT_URL'"
|
||||
```
|
||||
|
||||
**If members-only error:** Append `/videos` to URL and add `--ignore-errors --playlist-items 1:5`:
|
||||
|
||||
```bash
|
||||
ssh zvx@192.168.1.170 "sudo -u peertube /usr/local/bin/yt-dlp \
|
||||
--cookies /opt/bulk-import/config/cookies.txt \
|
||||
--print channel --print channel_url --print channel_id \
|
||||
--ignore-errors --playlist-items 1:5 --skip-download '${YT_URL}/videos' 2>/dev/null" | head -3
|
||||
```
|
||||
|
||||
Record the output:
|
||||
```
|
||||
CHANNEL_NAME="Civilian Rifleman"
|
||||
CHANNEL_URL="https://www.youtube.com/channel/UC..."
|
||||
CHANNEL_ID="UC..."
|
||||
```
|
||||
|
||||
### Step 2: Slugify Actor Name
|
||||
|
||||
```bash
|
||||
ACTOR_NAME=$(echo "$CHANNEL_NAME" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/-/g; s/--*/-/g; s/^-//; s/-$//' | cut -c1-50)
|
||||
echo "$ACTOR_NAME"
|
||||
```
|
||||
|
||||
### Step 3: Check for Duplicates
|
||||
|
||||
```bash
|
||||
ssh zvx@192.168.1.170 "cat /opt/bulk-import/config/channel-map.json" \
|
||||
| python3 -c "import sys,json; d=json.load(sys.stdin); \
|
||||
matches=[c for c in d if c.get('actor_name')=='$ACTOR_NAME' or c.get('youtube_channel_id')=='$CHANNEL_ID']; \
|
||||
print('DUPLICATE:', matches[0]['channel_name']) if matches else print('OK - no conflicts')"
|
||||
```
|
||||
|
||||
### Step 4: Create PeerTube Channel
|
||||
|
||||
```bash
|
||||
ssh zvx@192.168.1.170 bash << 'REMOTE'
|
||||
CLIENT=$(curl -s http://localhost:9000/api/v1/oauth-clients/local -H "Host: stream.echo6.co")
|
||||
CID=$(echo "$CLIENT" | python3 -c "import sys,json; print(json.load(sys.stdin)['client_id'])")
|
||||
CSEC=$(echo "$CLIENT" | python3 -c "import sys,json; print(json.load(sys.stdin)['client_secret'])")
|
||||
|
||||
TOKEN=$(curl -s http://localhost:9000/api/v1/users/token -H "Host: stream.echo6.co" \
|
||||
--data "client_id=$CID&client_secret=$CSEC&grant_type=password&username=root&password=7redditGold" \
|
||||
| python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])")
|
||||
|
||||
curl -s -X POST http://localhost:9000/api/v1/video-channels \
|
||||
-H "Host: stream.echo6.co" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"name\":\"ACTOR_NAME\",\"displayName\":\"(YT)CHANNEL_NAME\"}"
|
||||
REMOTE
|
||||
```
|
||||
|
||||
Replace `ACTOR_NAME` and `CHANNEL_NAME` in the `-d` payload. Record the returned `videoChannel.id`.
|
||||
|
||||
### Step 5: Update channel-map.json
|
||||
|
||||
**IMPORTANT:** Write to temp file first, then tee into place. Never pipe directly into tee on the same file being read — it causes a race condition that empties the file.
|
||||
|
||||
```bash
|
||||
ssh zvx@192.168.1.170 bash << 'REMOTE'
|
||||
python3 -c "
|
||||
import json
|
||||
with open('/opt/bulk-import/config/channel-map.json') as f:
|
||||
channels = json.load(f)
|
||||
channels.append({
|
||||
'category': 'CATEGORY',
|
||||
'channel_name': '(YT)CHANNEL_NAME',
|
||||
'actor_name': 'ACTOR_NAME',
|
||||
'youtube_url': 'CHANNEL_URL',
|
||||
'youtube_channel_id': 'CHANNEL_ID',
|
||||
'peertube_channel_id': PT_CHANNEL_ID,
|
||||
'video_count': 0,
|
||||
'priority': 'PRIORITY',
|
||||
'est_videos': 0,
|
||||
'est_gb': 0
|
||||
})
|
||||
print(json.dumps(channels, indent=2))
|
||||
" > /tmp/channel-map-new.json \
|
||||
&& sudo -u peertube tee /opt/bulk-import/config/channel-map.json < /tmp/channel-map-new.json > /dev/null \
|
||||
&& rm -f /tmp/channel-map-new.json \
|
||||
&& echo "OK"
|
||||
REMOTE
|
||||
```
|
||||
|
||||
Replace all placeholder values (CATEGORY, CHANNEL_NAME, ACTOR_NAME, CHANNEL_URL, CHANNEL_ID, PT_CHANNEL_ID, PRIORITY).
|
||||
|
||||
### Step 6: Verify
|
||||
|
||||
```bash
|
||||
# Check channel count
|
||||
curl -s http://192.168.1.130:8420/api/peertube/channels/stats | python3 -m json.tool
|
||||
|
||||
# Verify new channel in list
|
||||
curl -s http://192.168.1.130:8420/api/peertube/channels \
|
||||
| python3 -c "import sys,json; d=json.load(sys.stdin); print(d[-1]['actor_name'], d[-1]['category'])"
|
||||
```
|
||||
|
||||
## Recovery: Empty channel-map.json
|
||||
|
||||
If `tee` race condition empties the file:
|
||||
|
||||
1. Check Contabo backup: `ssh root@100.64.0.1 ls -la /opt/backups/recon/`
|
||||
2. Or rebuild from PeerTube DB:
|
||||
```bash
|
||||
ssh zvx@192.168.1.170 "sudo -u peertube psql peertube_prod -t -A -c \
|
||||
\"SELECT name, \\\"displayName\\\" FROM \\\"videoChannel\\\" WHERE name != 'root_channel' AND name != 'default' ORDER BY id;\""
|
||||
```
|
||||
|
||||
## API Endpoints (RECON Dashboard)
|
||||
|
||||
| Endpoint | Method | Purpose |
|
||||
|----------|--------|---------|
|
||||
| `/api/peertube/channels` | GET | List all channels with video counts |
|
||||
| `/api/peertube/channels/stats` | GET | Total channels, videos, downloader status |
|
||||
| `/api/peertube/channels/add` | POST | Add channel (JSON: youtube_url, category, priority) |
|
||||
| `/api/peertube/channels/<actor_name>` | DELETE | Remove channel from JSON and PeerTube |
|
||||
|
||||
## Common Issues
|
||||
|
||||
| Issue | Cause | Fix |
|
||||
|-------|-------|-----|
|
||||
| yt-dlp "Join this channel" error | Members-only first video | API auto-retries with `/videos` tab. CLI: add `--ignore-errors --playlist-items 1:5` and use `/videos` URL |
|
||||
| channel-map.json empty (0 bytes) | tee race condition | Always write to temp file first, then tee. Restore from backup or Contabo |
|
||||
| sudo: password required | Sudoers not set up | Create `/etc/sudoers.d/recon-mgmt` via `pct exec 110` from root@192.168.1.243 |
|
||||
| PeerTube "actor name already exists" | Channel exists in PeerTube but not in JSON | Add entry to JSON manually with correct `peertube_channel_id` |
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2026-02-18 — Initial creation*
|
||||
Loading…
Add table
Add a link
Reference in a new issue