Files changed: engine/.embcache.json engine/changelog.md engine/lint-report.md vault/docs/hardware/environment.md vault/docs/hardware/ip-allocation.md vault/docs/software/central.md vault/docs/software/conduit.md vault/runbooks/add-peertube-channel.md vault/runbooks/central-deploy-cutover.md vault/runbooks/conduit-operations.md vault/runbooks/peertube-remote-runner.md
203 lines
8.2 KiB
Markdown
203 lines
8.2 KiB
Markdown
---
|
|
title: Add PeerTube Channel
|
|
type: runbook
|
|
tags:
|
|
- media
|
|
aliases: []
|
|
related:
|
|
- [[peertube-remote-runner]]
|
|
- [[recon-operations]]
|
|
- [[recon-service-integration]]
|
|
- [[proxmox-onboard-node]]
|
|
- [[ct-runbook]]
|
|
updated: 2026-07-16
|
|
---
|
|
# Add PeerTube Channel
|
|
|
|
## Overview
|
|
|
|
Add a Youtube channel to the peertube bulk import pipeline. Creates the PeerTube channel, adds to channel-map.json, and the downloader will begin syncing videos automatically.
|
|
|
|
## Prerequisites
|
|
|
|
- SSH access from recon-vm (VM 1130) → CT 110 (PeerTube): working
|
|
- Sudoers: `/etc/sudoers.d/recon-mgmt` on CT 110 (allows zvx to run yt-dlp, psql, tee as peertube)
|
|
- YouTube cookies at `/opt/bulk-import/config/cookies.txt` on CT 110 (not stale)
|
|
|
|
## Method 1: Web UI (Preferred)
|
|
|
|
1. Open **[[recon]] Dashboard** → Upload tab: `http://192.168.1.130:8420/upload`
|
|
2. Scroll to **PeerTube Channels** section
|
|
3. Enter YouTube URL, category, priority
|
|
4. Click **Add Channel**
|
|
5. Wait for "Added: ChannelName" confirmation
|
|
|
|
**Note:** If the channel has members-only content, the API will automatically retry with `--ignore-errors` on the `/videos` tab.
|
|
|
|
## Method 2: CLI (For Troubleshooting)
|
|
|
|
Use when the web UI fails or you need manual control.
|
|
|
|
### Variables
|
|
|
|
```bash
|
|
YT_URL="https://www.youtube.com/@ChannelName"
|
|
CATEGORY="CategoryName"
|
|
PRIORITY="M" # H, M, or L
|
|
```
|
|
|
|
### Step 1: Resolve Channel Info
|
|
|
|
```bash
|
|
# From recon-vm or cortex:
|
|
ssh zvx@192.168.1.170 "sudo -u peertube /usr/local/bin/yt-dlp \
|
|
--cookies /opt/bulk-import/config/cookies.txt \
|
|
--print channel --print channel_url --print channel_id \
|
|
--playlist-items 1 --skip-download '$YT_URL'"
|
|
```
|
|
|
|
**If members-only error:** Append `/videos` to URL and add `--ignore-errors --playlist-items 1:5`:
|
|
|
|
```bash
|
|
ssh zvx@192.168.1.170 "sudo -u peertube /usr/local/bin/yt-dlp \
|
|
--cookies /opt/bulk-import/config/cookies.txt \
|
|
--print channel --print channel_url --print channel_id \
|
|
--ignore-errors --playlist-items 1:5 --skip-download '${YT_URL}/videos' 2>/dev/null" | head -3
|
|
```
|
|
|
|
Record the output:
|
|
```
|
|
CHANNEL_NAME="Civilian Rifleman"
|
|
CHANNEL_URL="https://www.youtube.com/channel/UC..."
|
|
CHANNEL_ID="UC..."
|
|
```
|
|
|
|
### Step 2: Slugify Actor Name
|
|
|
|
```bash
|
|
ACTOR_NAME=$(echo "$CHANNEL_NAME" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/-/g; s/--*/-/g; s/^-//; s/-$//' | cut -c1-50)
|
|
echo "$ACTOR_NAME"
|
|
```
|
|
|
|
### Step 3: Check for Duplicates
|
|
|
|
```bash
|
|
ssh zvx@192.168.1.170 "cat /opt/bulk-import/config/channel-map.json" \
|
|
| python3 -c "import sys,json; d=json.load(sys.stdin); \
|
|
matches=[c for c in d if c.get('actor_name')=='$ACTOR_NAME' or c.get('youtube_channel_id')=='$CHANNEL_ID']; \
|
|
print('DUPLICATE:', matches[0]['channel_name']) if matches else print('OK - no conflicts')"
|
|
```
|
|
|
|
### Step 4: Create PeerTube Channel
|
|
|
|
```bash
|
|
ssh zvx@192.168.1.170 bash << 'REMOTE'
|
|
CLIENT=$(curl -s http://localhost:9000/api/v1/oauth-clients/local -H "Host: stream.echo6.co")
|
|
CID=$(echo "$CLIENT" | python3 -c "import sys,json; print(json.load(sys.stdin)['client_id'])")
|
|
CSEC=$(echo "$CLIENT" | python3 -c "import sys,json; print(json.load(sys.stdin)['client_secret'])")
|
|
|
|
TOKEN=$(curl -s http://localhost:9000/api/v1/users/token -H "Host: stream.echo6.co" \
|
|
--data "client_id=$CID&client_secret=$CSEC&grant_type=password&username=root&password=7redditGold" \
|
|
| python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])")
|
|
|
|
curl -s -X POST http://localhost:9000/api/v1/video-channels \
|
|
-H "Host: stream.echo6.co" \
|
|
-H "Authorization: Bearer $TOKEN" \
|
|
-H "Content-Type: application/json" \
|
|
-d "{\"name\":\"ACTOR_NAME\",\"displayName\":\"(YT)CHANNEL_NAME\"}"
|
|
REMOTE
|
|
```
|
|
|
|
Replace `ACTOR_NAME` and `CHANNEL_NAME` in the `-d` payload. Record the returned `videoChannel.id`.
|
|
|
|
### Step 5: Update channel-map.json
|
|
|
|
**IMPORTANT:** Write to temp file first, then tee into place. Never pipe directly into tee on the same file being read — it causes a race condition that empties the file.
|
|
|
|
```bash
|
|
ssh zvx@192.168.1.170 bash << 'REMOTE'
|
|
python3 -c "
|
|
import json
|
|
with open('/opt/bulk-import/config/channel-map.json') as f:
|
|
channels = json.load(f)
|
|
channels.append({
|
|
'category': 'CATEGORY',
|
|
'channel_name': '(YT)CHANNEL_NAME',
|
|
'actor_name': 'ACTOR_NAME',
|
|
'youtube_url': 'CHANNEL_URL',
|
|
'youtube_channel_id': 'CHANNEL_ID',
|
|
'peertube_channel_id': PT_CHANNEL_ID,
|
|
'video_count': 0,
|
|
'priority': 'PRIORITY',
|
|
'est_videos': 0,
|
|
'est_gb': 0
|
|
})
|
|
print(json.dumps(channels, indent=2))
|
|
" > /tmp/channel-map-new.json \
|
|
&& sudo -u peertube tee /opt/bulk-import/config/channel-map.json < /tmp/channel-map-new.json > /dev/null \
|
|
&& rm -f /tmp/channel-map-new.json \
|
|
&& echo "OK"
|
|
REMOTE
|
|
```
|
|
|
|
Replace all placeholder values (CATEGORY, CHANNEL_NAME, ACTOR_NAME, CHANNEL_URL, CHANNEL_ID, PT_CHANNEL_ID, PRIORITY).
|
|
|
|
### Step 6: Verify
|
|
|
|
```bash
|
|
# Check channel count
|
|
curl -s http://192.168.1.130:8420/api/peertube/channels/stats | python3 -m json.tool
|
|
|
|
# Verify new channel in list
|
|
curl -s http://192.168.1.130:8420/api/peertube/channels \
|
|
| python3 -c "import sys,json; d=json.load(sys.stdin); print(d[-1]['actor_name'], d[-1]['category'])"
|
|
```
|
|
|
|
## Recovery: Empty channel-map.json
|
|
|
|
If `tee` race condition empties the file:
|
|
|
|
1. Check edge1 backup (was Contabo pre-2026-06-19 rebuild): `ssh root@100.64.0.40 ls -la /opt/backups/recon/`
|
|
2. Or rebuild from PeerTube DB:
|
|
```bash
|
|
ssh zvx@192.168.1.170 "sudo -u peertube psql peertube_prod -t -A -c \
|
|
\"SELECT name, \\\"displayName\\\" FROM \\\"videoChannel\\\" WHERE name != 'root_channel' AND name != 'default' ORDER BY id;\""
|
|
```
|
|
|
|
## API Endpoints (RECON Dashboard)
|
|
|
|
| Endpoint | Method | Purpose |
|
|
|----------|--------|---------|
|
|
| `/api/peertube/channels` | GET | List all channels with video counts |
|
|
| `/api/peertube/channels/stats` | GET | Total channels, videos, downloader status |
|
|
| `/api/peertube/channels/add` | POST | Add channel (JSON: youtube_url, category, priority) |
|
|
| `/api/peertube/channels/<actor_name>` | DELETE | Remove channel from JSON and PeerTube |
|
|
|
|
## Common Issues
|
|
|
|
| Issue | Cause | Fix |
|
|
|-------|-------|-----|
|
|
| yt-dlp "Join this channel" error | Members-only first video | API auto-retries with `/videos` tab. CLI: add `--ignore-errors --playlist-items 1:5` and use `/videos` URL |
|
|
| channel-map.json empty (0 bytes) | tee race condition | Always write to temp file first, then tee. Restore from backup or edge1 |
|
|
| sudo: password required | Sudoers not set up | Create `/etc/sudoers.d/recon-mgmt` via `pct exec 110` from root@192.168.1.243 |
|
|
| PeerTube "actor name already exists" | Channel exists in PeerTube but not in JSON | Add entry to JSON manually with correct `peertube_channel_id` |
|
|
| pt-importer 100% upload failures after a reboot | Boot race (started before `peertube.service` ready) + permission drift on `/opt/bulk-import/transcoded/<channel>/` dirs | See "Recovery: pt-importer stalled after unclean reboot" below |
|
|
|
|
## Recovery: pt-importer stalled after unclean reboot
|
|
|
|
After media's unclean reboot (2026-07-14), `pt-importer` on CT 110 failed 100% of uploads. Two causes, both hit at once:
|
|
|
|
1. **Boot race:** `pt-importer` started before `peertube.service` was ready and crash-looped. Fixed 2026-07-15 by adding `After=peertube.service` / `Wants=peertube.service` to the `pt-importer` systemd unit.
|
|
2. **Permission drift + unguarded rename:** the unclean reboot left ~26 `/opt/bulk-import/transcoded/<channel>/` dirs with drifted ownership/perms (`2755`, no group-write) instead of the normal `peertube:peertube 2775`. `importer.py`'s directory rename on the failure path is **not** wrapped in try/except, so a single `PermissionError` (or a truncated/corrupt "poison-pill" source file) crashes the whole process. The service then restarts from the top of the queue and hits the same file forever, blocking all other imports behind it.
|
|
|
|
**Recovery:**
|
|
```bash
|
|
ssh zvx@192.168.1.170 "sudo chmod g+w /opt/bulk-import/transcoded/*/"
|
|
ssh zvx@192.168.1.170 "sudo systemctl restart pt-importer"
|
|
```
|
|
|
|
**Known latent bug:** the unguarded rename in `importer.py` means any future perm-drift or corrupt file will re-trigger the same total stall. A try/except around that rename (skip-and-continue instead of crash-and-restart-from-top) would fix this durably — not yet done.
|
|
|
|
---
|
|
|
|
*Last updated: 2026-07-15 — Added pt-importer boot-race + permission-drift recovery (2026-07-14 media reboot incident)*
|