echo6-docs/docs/hardware/environment.md

# Echo6 Environment Reference

## Proxmox Cluster (echo6-cluster)

Five nodes running Proxmox VE:

| Node | Local IP | Tailscale | Hardware | RAM | Purpose |
|------|----------|-----------|----------|-----|---------|
| data | 192.168.1.240 | 100.64.0.6 | AMD Ryzen 7 PRO 5750GE, 1TB NVMe + 1TB SATA SSD | 32GB DDR4-3200 | Database services |
| utility | 192.168.1.241 | 100.64.0.5 | AMD Ryzen 7 PRO 5750GE, 512GB NVMe | 32GB DDR4-3200 | Utility services, monitoring |
| cloud | 192.168.1.242 | 100.64.0.4 | Intel i7-12700T, 512GB NVMe | 32GB DDR4-3200 | Cloud storage, personal services |
| media | 192.168.1.243 | 100.64.0.3 | Intel i7-14700T, 2x 512GB NVMe | 32GB DDR5-5600 | Media server, *arr stack |
| toc | 192.168.1.244 | 100.64.0.13 | Workstation (i9-10900X) | 64GB DDR4 | GPU compute, AI/ML workloads |

### Node Storage Details

| Node | Primary Disk | Secondary Disk |
|------|-------------|----------------|
| data | Samsung SSD 980 1TB (NVMe) | SanDisk SDSSDH3 1TB (SATA SSD) |
| utility | WD PC SN740 512GB (NVMe) | — |
| cloud | SK Hynix HFS512GEJ9X164N 512GB (NVMe) | — |
| media | 2x Intel SSDPEKNU512GZH 512GB (NVMe) | — |
| toc | 512GB NVMe | — |

### Network Notes

- **media NIC:** Original Intel e1000e NIC crashes under sustained NFS load — replaced with USB Realtek RTL8153 GbE adapter on vmbr0
- **Tailscale DNS bootstrap:** All LXC containers with Tailscale have a systemd drop-in (`/etc/systemd/system/tailscaled.service.d/dns-bootstrap.conf`) that ensures fallback DNS exists before tailscaled starts, preventing chicken-and-egg DNS resolution failures on reboot

### TOC Node Details

- **Hardware:** Intel i9-10900X (20 threads), 64GB RAM (4×8GB + 2×16GB DDR4), 512GB NVMe, RTX A4000
- **GPU:** Passed through via VFIO to VM 150 (cortex), not used on host
- **VMID ranges:** 100-149 (LXC), 150-199 (VMs)
- **Presave backup:** `/home/zvx/toc-presave/` on Contabo (1.8G) — contains old Ubuntu config

## Virtual Machines

| VM | Host | VMID | Local IP | Tailscale | Purpose |
|----|------|------|----------|-----------|---------|
| cortex | toc | 150 | 192.168.1.150 | 100.64.0.14 | GPU compute — Open WebUI, Ollama, Qdrant, TEI, Claude Code |
| recon-vm | data | 1130 | 192.168.1.130 | 100.64.0.24 | RECON knowledge extraction pipeline, Files, Kiwix |
| arr | media | 105 | 192.168.1.160 | 100.64.0.18 | ARR media automation stack (Jellyfin, Sonarr, Radarr, etc.) |

### cortex VM Details

- **OS:** Ubuntu 24.04 (cloud-init), kernel 6.8.0-100-generic
- **Resources:** 16 threads, 32GB RAM, 300GB disk
- **Swap:** 32GB swapfile (`/swapfile`), `vm.swappiness=10` (prefer RAM, swap under pressure only). Provisioned 2026-04-23 to support bursty jobs (gdal_contour, tippecanoe) alongside Docker ML containers.
- **GPU:** RTX A4000 (passthrough), NVIDIA driver 580.126.09, CUDA 13.0
- **Software:** Docker 29.2.1 + nvidia-container-toolkit 1.18.2, Node.js 22.22.0, Python 3.12.3
- **Docker containers:** open-webui (8080), ollama (11434 w/ GPU), qdrant (6333), tei (8090)
- **User:** zvx (sudo, SSH keys from cluster)
- **Claude Code:** installed

### recon-vm Details

- **OS:** Ubuntu 24.04.4 LTS (cloud-init), kernel 6.8.0-110-generic
- **Resources:** 4 cores, 16GB RAM, 100GB disk
- **Software:** Docker 29.4.0, Python 3.12.3, nginx, sqlite3, Tailscale
- **Systemd services:** recon (8420), recon-watchdog, kiwix (8430), nginx (8888)
- **NFS mounts:** pi-nas:/export/library → /mnt/library, /mnt/nav, /mnt/kiwix
- **User:** zvx (sudo, SSH key auth)
- **Migrated from:** CT 130 (LXC) on 2026-04-19. Tailscale identity preserved (100.64.0.24).
- **Note:** Old CT 130 is stopped on the data host, not yet destroyed.

### arr VM Details

- **OS:** Ubuntu 24.04 (cloud-init)
- **Resources:** 4 cores, 8GB RAM, 30GB disk on local-lvm
- **Software:** Docker 29.2.1, Tailscale, NFS client, sshpass, qemu-guest-agent
- **User:** zvx (sudo, SSH key from cortex)
- **NFS:** pi-nas:/export/arr → /mnt/arr (22TB, movies/tv/downloads)
- **Docker containers:** jellyfin (8096), jellyseer (5055), sonarr (8989), radarr (7878), prowlarr (9696), sabnzbd (8080), lidarr (8686/6595), navidrome (4533)
- **Docker network:** arr-net (bridge)
- **Config dirs:** /opt/arr/{jellyfin,jellyseer,sonarr,radarr,prowlarr,sabnzbd,lidarr}

## Key Servers

| Server | Local IP | Tailscale | Purpose |
|--------|----------|-----------|---------|
| aida-nebra | 192.168.1.253 | 100.64.0.9 | AIDA-N2(RPT,LLM) — meshtasticd node !27780c47, Nebra 2W hat, port 4403. MeshAI (CT 108) connects here via TCP |
| mt-isr | 192.168.1.141 | 100.100.0.5 (IdahoMesh) | Meshtastic sidecar Pi (G2 WiFi bridge, meshtasticd, CLI) |
| mt-burleybutte | 192.168.1.185 | — | Meshtastic node (meshtasticd, Nebra 2W hat, IdahoMesh VPN) |
| pi-nas | 192.168.1.245 | 100.64.0.21 | Raspberry Pi NAS |
| matt-desktop | 192.168.1.111 | 100.64.0.10 | Personal workstation (Windows, your PC) |
| Contabo Server | 5.189.158.149 | 100.64.0.1 | External VPS: Mail, Authentik, Headscale, Forge, Matrix |

*Last updated: 2026-04-20 — CT 130 (RECON LXC) migrated to VM 1130 (recon-vm)*

## LXC Containers

| Container | Host | Local IP | Tailscale | Purpose |
|-----------|------|----------|-----------|---------|
| meshmonitor | utility (CT 100) | 192.168.1.100 | 100.64.0.7 | Meshtastic mesh monitoring (zvx-echo6/meshmonitor fork, multi-channel) |
| caddy | utility (CT 101) | 192.168.1.101 | 100.64.0.8 | Home reverse proxy |
| searxng | utility (CT 102) | 192.168.1.102 | 100.64.0.15 | Echo6 Search homepage (SearXNG, echo6.co) |
| advbbs | utility (CT 103) | 192.168.1.103 | 100.64.0.31 | Meshtastic sim node (ADVBBS) |
| immich | cloud (CT 120) | 192.168.1.182 | 100.64.0.2 | Immich photo management |
| nextcloud | cloud (CT 121) | 192.168.1.183 | 100.64.0.11 | Nextcloud AIO |
| meshtastic-hs | utility (CT 106) | 192.168.1.106 | — | IdahoMesh Headscale VPN coordination |
| mesh-bridge | utility (CT 107) | 192.168.1.107 | 100.64.0.22 | Dual-tailscaled bridge (echo6 ↔ idahomesh) |
| meshai | utility (CT 108) | 192.168.1.144 | 100.64.0.32 | MeshAI - LLM-powered Meshtastic assistant |
| archivist | utility (CT 118) | 192.168.1.118 | — | Archivist knowledge pipeline |
| peertube | media (CT 110) | 192.168.1.170 | 100.64.0.23 | PeerTube video streaming |

## IP Allocation Scheme

| Range | Purpose |
|-------|---------|
| .1-.10 | Network infrastructure |
| .11-.99 | DHCP clients |
| .100-.149 | LXC containers |
| .150-.199 | VMs |
| .240-.250 | Proxmox hosts + bare metal |
| .251-.254 | Meshtastic nodes |

Full details: `/home/zvx/projects/utility/ip-allocation.md`

## Headscale Node List

Current registered nodes (25 total):

| Node | Tailscale IP | Type |
|------|-------------|------|
| contabo | 100.64.0.1 | VPS |
| immich | 100.64.0.2 | LXC |
| media | 100.64.0.3 | Proxmox |
| cloud | 100.64.0.4 | Proxmox |
| utility | 100.64.0.5 | Proxmox |
| data | 100.64.0.6 | Proxmox |
| meshmonitor | 100.64.0.7 | LXC |
| caddy | 100.64.0.8 | LXC |
| aida-nebra | 100.64.0.9 | Pi |
| matt-desktop | 100.64.0.10 | Desktop |
| nextcloud | 100.64.0.11 | LXC |
| toc | 100.64.0.13 | Proxmox |
| cortex | 100.64.0.14 | VM |
| searxng | 100.64.0.15 | LXC |
| iphone-eud | 100.64.0.16 | Mobile |
| arr | 100.64.0.18 | VM |
| pi-nas | 100.64.0.21 | Pi |
| mesh-bridge | 100.64.0.22 | LXC |
| peertube | 100.64.0.23 | LXC |
| recon | 100.64.0.24 | VM |
| meshmonitor-dev | 100.64.0.27 | LXC |
| gl-a1300 | 100.64.0.29 | Router |
| bluefin | 100.64.0.30 | Desktop |
| advbbs | 100.64.0.31 | LXC |
| meshai | 100.64.0.32 | LXC |

## IdahoMesh Headscale Node List

Separate Headscale instance on CT 106 (192.168.1.106), prefix 100.100.0.0/16.
Reachable from echo6 tailnet via mesh-bridge (CT 107).

| Node | Tailscale IP | User | Type |
|------|-------------|------|------|
| mesh-bridge | 100.100.0.3 | malice | LXC (bridge) |
| burley-butte | 100.100.0.1 | nebra | Pi (offline) |
| mt-isr | 100.100.0.5 | nebra | Pi Zero 2 W |

## SSH Access

**Standard user:** `zvx`
**Credentials:** Source from `/home/zvx/projects/.ref/credentials`

Most servers use SSH key auth. Exceptions noted below.

```bash
# SSH to any server (key auth)
ssh zvx@<ip-address>

# Examples
ssh zvx@192.168.1.244  # TOC (Proxmox host)
ssh zvx@192.168.1.150  # cortex VM
ssh zvx@192.168.1.241  # utility Proxmox
ssh root@100.64.0.1    # Contabo (via Tailscale)
ssh zvx@cortex         # cortex via Tailscale hostname
```

### Password-auth hosts

These require password authentication (no SSH keys installed):

| Host | User | Password | Access |
|------|------|----------|--------|
| aida-nebra | zvx | 7redditGold | `sshpass -p '7redditGold' ssh zvx@aida-nebra` |
| mt-isr | isr | UfPsfwyMIUIKb1 | `sshpass -p 'UfPsfwyMIUIKb1' ssh isr@192.168.1.141` |
| mt-burleybutte | bb | (see credentials) | `sshpass -p '<pw>' ssh bb@192.168.1.185` |
| matt-desktop | administrator | Qw1290opzx | `ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no administrator@192.168.1.111` |
| toc | root | 7redditGold | `sshpass -p '7redditGold' ssh -o PubkeyAuthentication=no root@100.64.0.13` |

Use the Tailscale hostname (`aida-nebra`) or local IP (`192.168.1.253`) — both work for aida-nebra.
mt-isr is on IdahoMesh tailnet (100.100.0.5) — reachable from echo6 via bridge.
matt-desktop is accessible via local IP (192.168.1.111) or Tailscale (100.64.0.10) — requires explicit password auth flags.

## Key External IPs

| Purpose | IP |
|---------|-----|
| Home external (public services) | 199.6.36.163 |
| Contabo VPS | 5.189.158.149 |