matt/echo6-docs
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
echo6-docs/vault/docs/hardware/environment.md
echo6-autocommit 77b4715384 auto: docs sync 2026-07-16T12:00:15+00:00 
Files changed: engine/.embcache.json engine/changelog.md engine/lint-report.md vault/docs/hardware/environment.md vault/docs/hardware/ip-allocation.md vault/docs/software/central.md vault/docs/software/conduit.md vault/runbooks/add-peertube-channel.md vault/runbooks/central-deploy-cutover.md vault/runbooks/conduit-operations.md vault/runbooks/peertube-remote-runner.md
2026-07-16 12:00:15 +00:00

15 KiB
Raw Blame History

title type tags aliases related updated
Echo6 Environment Reference reference
proxmox
fleet-platform-baseline
ip-allocation
headscale-onboard-node
proxmox-create-ubuntu-vm
toc-cortex-pve9.2-update
2026-07-16

Echo6 Environment Reference

Proxmox Cluster (echo6-cluster)

Five nodes running Proxmox VE:

Node Local IP Tailscale Hardware RAM Purpose
data 192.168.1.240 100.64.0.6 AMD Ryzen 7 PRO 5750GE, 1TB NVMe + 1TB SATA SSD 32GB DDR4-3200 Database services
utility 192.168.1.241 100.64.0.5 AMD Ryzen 7 PRO 5750GE, 512GB NVMe 32GB DDR4-3200 Utility services, monitoring
cloud 192.168.1.242 100.64.0.4 Intel i7-12700T, 512GB NVMe 32GB DDR4-3200 Cloud storage, personal services
media 192.168.1.243 100.64.0.3 Intel i7-14700T, 2x 512GB NVMe 32GB DDR5-5600 Media server, *arr stack
toc 192.168.1.244 100.64.0.13 Workstation (i9-10900X) 64GB DDR4 GPU compute, AI/ML workloads

Node Storage Details

Node Primary Disk Secondary Disk
data Samsung SSD 980 1TB (NVMe) SanDisk SDSSDH3 1TB (SATA SSD)
utility WD PC SN740 512GB (NVMe) —
cloud SK Hynix HFS512GEJ9X164N 512GB (NVMe) —
media 2x Intel SSDPEKNU512GZH 512GB (NVMe) —
toc 512GB NVMe —

Node Hardware Identifiers

Node Make/Model Serial/Service Tag
data Lenovo ThinkCentre M75q Gen 2 (11JN002RUS) MZ010LPV
utility Lenovo ThinkCentre M75q Gen 2 (11JN002RUS) MJ0LZNYT
cloud Lenovo ThinkCentre M70q Gen 3 (11T3000RUS) MJ0LQCGJ
media Dell OptiPlex Micro 7020 58SM6X3 (BIOS 1.20.0, 2025-09-04)
toc HP Z4 G4 Workstation MXL2383MVK

No BMC/IPMI/iDRAC on any node (verified dmidecode -t 38 empty on all five, 2026-07-14) — there is no remote power-cycle path for any Proxmox host. Recovery from a hung or powered-off node requires physical access.

Network Notes

  • media NIC: Original Intel e1000e NIC (nic0, MAC e8:cf:83:20:8b:cb) crashes under sustained NFS load — present but DOWN/unused. Sole uplink is a USB Realtek RTL8153 GbE dongle (MAC 0c:37:96:0e:e8:53) on vmbr0.
  • Tailscale dns bootstrap: All LXC containers with Tailscale have a systemd drop-in (/etc/systemd/system/tailscaled.service.d/dns-bootstrap.conf) that ensures fallback dns exists before tailscaled starts, preventing chicken-and-egg dns resolution failures on reboot

TOC Node Details

  • Hardware: Intel i9-10900X (20 threads), 64GB RAM (4×8GB + 2×16GB DDR4), 512GB NVMe, RTX A4000
  • GPU: Passed through via VFIO to VM 150 (cortex), not used on host
  • VMID ranges: 100-149 (LXC), 150-199 (VMs)
  • Presave backup: /home/zvx/toc-presave/ on Contabo (1.8G) — contains old Ubuntu config

Virtual Machines

VM Host VMID Local IP Tailscale Purpose
cortex toc 150 192.168.1.150 100.64.0.14 GPU compute — Open WebUI, Ollama, Qdrant, TEI, Claude Code
recon-vm data 1130 192.168.1.130 100.64.0.24 recon + navi platforms, Files, Kiwix, geo backends
arr media 105 192.168.1.160 100.64.0.18 ARR media automation stack (Jellyfin, Sonarr, Radarr, etc.)

cortex VM Details

  • OS: Ubuntu 24.04 (cloud-init), kernel 6.8.0-100-generic
  • Resources: 16 threads, 32GB RAM, 300GB disk
  • Swap: 32GB swapfile (/swapfile), vm.swappiness=10 (prefer RAM, swap under pressure only). Provisioned 2026-04-23 to support bursty jobs (gdal_contour, tippecanoe) alongside Docker ML containers.
  • GPU: RTX A4000 (passthrough), NVIDIA driver 580.126.09, CUDA 13.0
  • Software: Docker 29.2.1 + nvidia-container-toolkit 1.18.2, Node.js 22.22.0, Python 3.12.3
  • Docker containers: open-webui (8080), ollama (11434 w/ GPU), qdrant (6333), tei (8090)
  • User: zvx (sudo, SSH keys from cluster)
  • Claude Code: installed

recon-vm Details

  • OS: Ubuntu 24.04.4 LTS (cloud-init), kernel 6.8.0-110-generic
  • Resources: 4 cores, 24GB RAM, 180GB disk
  • Software: Docker 29.4.0, Python 3.12.3, nginx, sqlite3, Tailscale
  • Platforms: recon (knowledge extraction pipeline, :8420) and navi (offline navigation, navi.echo6.co, :8440) with geo backends (Valhalla :8002, Nominatim :8010, Photon :2322, PostgreSQL/PostGIS :5432)
  • Systemd services: recon (8420), recon-watchdog, kiwix (8430), nginx (8888)
  • NFS mounts: pi-nas:/export/library → /mnt/library, /mnt/nav, /mnt/kiwix
  • User: zvx (sudo, SSH key auth)
  • Migrated from: CT 130 (LXC) on 2026-04-19. Tailscale identity preserved (100.64.0.24).
  • Note: Old CT 130 (LXC) has since been destroyed (verified 2026-06-15) — only VM 1130 remains.

arr VM Details

  • OS: Ubuntu 24.04 (cloud-init)
  • Resources: 4 cores, 8GB RAM, 30GB disk on local-lvm
  • Software: Docker 29.2.1, Tailscale, NFS client, sshpass, qemu-guest-agent
  • User: zvx (sudo, SSH key from cortex)
  • NFS: pi-nas:/export/arr → /mnt/arr (22TB, movies/tv/downloads)
  • Docker containers: jellyfin (8096), jellyseer (5055), sonarr (8989), radarr (7878), prowlarr (9696), sabnzbd (8080), lidarr (8686/6595), navidrome (4533)
  • Docker network: arr-net (bridge)
  • Config dirs: /opt/arr/{jellyfin,jellyseer,sonarr,radarr,prowlarr,sabnzbd,lidarr}

Key Servers

Server Local IP Tailscale Purpose
aida-nebra 192.168.1.253 100.64.0.9 AIDA-N2(RPT,LLM) — meshtasticd node !27780c47, Nebra 2W hat, port 4403. meshai (CT 108) connects here via TCP
mt-isr 192.168.1.141 100.100.0.5 (IdahoMesh) Meshtastic sidecar Pi (G2 WiFi bridge, meshtasticd, CLI)
mt-burleybutte 192.168.1.185 — Meshtastic node (meshtasticd, Nebra 2W hat, IdahoMesh VPN)
pi-nas 192.168.1.245 100.64.0.21 Raspberry Pi NAS
matt-desktop 192.168.1.254 100.64.0.10 Personal workstation (Windows, your PC)
ha 192.168.1.151 100.64.0.16 Home Assistant (VM 151 on cloud, Docker, home automation)

| edge1 (rebuilt Contabo VPS) | 5.189.158.149 | 100.64.0.40 | Debian 12 + Proxmox 8.4.19, mail-only — Mailcow in CT 101; host caddy + mailcow-dnat.service; rebuilt 2026-06-19 | | edge2 | 184.174.35.153 | 100.64.0.26 | Contabo Cloud VPS 30 NVMe — Proxmox VE 8.4.19 (LXC-only), 8c/24GB/400GB — permanent front door for vault/forge/notes/auth/matrix/element/vpn/proxmox.echo6.co + idahomesh/intermountainmesh |

Last updated: 2026-06-19 — Contabo VPS rebuilt as edge1 (mail-only, Debian 12 + Proxmox 8.4.19, 5.189.158.149 / tailnet 100.64.0.40); Mailcow CT 101 (10.10.10.2) on edge1; edge2 is now the permanent front door for all other services; Headscale node contabo moved to 100.64.0.40; previously added edge2 CT 107 (headscale), CT 106 (matrix), CT 105 (authentik), CT 104 (livesync), CT 103 (forgejo), CT 102 (vaultwarden)

LXC Containers

Container Host Local IP Tailscale Purpose
meshmonitor utility (CT 100) 192.168.1.100 100.64.0.7 Meshtastic mesh monitoring (zvx-echo6/meshmonitor fork, multi-channel)
caddy utility (CT 101) 192.168.1.101 100.64.0.8 Home reverse proxy
searxng utility (CT 102) 192.168.1.102 100.64.0.15 Echo6 Search homepage (searxng, echo6.co)
immich cloud (CT 120) 192.168.1.182 100.64.0.2 Immich photo management
nextcloud cloud (CT 121) 192.168.1.183 100.64.0.11 Nextcloud AIO
meshtastic-hs utility (CT 106) 192.168.1.106 — IdahoMesh Headscale VPN coordination
mesh-bridge utility (CT 107) 192.168.1.107 100.64.0.22 Dual-tailscaled bridge (echo6 ↔ idahomesh)
meshai utility (CT 108) 192.168.1.144 100.64.0.32 MeshAI - LLM-powered Meshtastic assistant
archivist utility (CT 118) 192.168.1.118 — archivist knowledge pipeline
argus utility (CT 103) 192.168.1.103 100.64.0.25 argus - OSINT intelligence gathering platform
central utility (CT 104) 192.168.1.104 100.64.0.12 Data-hub spine (central.echo6.mesh) — ~25 adapters, NATS/JetStream, TimescaleDB/PostGIS — see central
peertube media (CT 110) 192.168.1.170 100.64.0.17 PeerTube video streaming — Tailscale identity is peertube-4hve9pdr (collision-suffixed) since ~2026-06-22; .23 is stale/gone
mcc media (CT 111) 192.168.1.111 100.64.0.19 pymc console web app (caddy + Postfix, /api+/auth+/ws → aida-nebra :8000)
mailcow edge1 (CT 101) 10.10.10.2 — Mailcow email server (privileged LXC, mail-only host; reached via host DNAT + Caddy)
pdm edge2 (CT 100) 10.10.10.10 100.64.0.28 Proxmox Datacenter Manager
wordpress edge2 (CT 101) 10.10.10.11 100.64.0.31 WordPress for intermountainmesh.com
vaultwarden edge2 (CT 102) 10.10.10.20 100.64.0.33 Vaultwarden password manager (migrated from Contabo 2026-06-16)
forgejo edge2 (CT 103) 10.10.10.21 100.64.0.34 Forgejo git server (migrated from Contabo 2026-06-16)
livesync edge2 (CT 104) 10.10.10.22 100.64.0.35 LiveSync Obsidian sync (CouchDB + provisioner; migrated from Contabo 2026-06-16)
authentik edge2 (CT 105) 10.10.10.23 100.64.0.36 Authentik SSO platform (migrated from Contabo 2026-06-18)
matrix edge2 (CT 106) 10.10.10.24 100.64.0.37 Matrix stack (synapse + MAS + Element + mautrix_signal; migrated from Contabo 2026-06-18)
headscale edge2 (CT 107) 10.10.10.25 100.64.0.38 Headscale + Headplane tailnet control plane (migrated from Contabo 2026-06-19)

Note (2026-06-19): edge2 CT placements CT 102–107 confirmed; Forge git-SSH DNAT (forgejo-ssh-dnat.service) is a permanent systemd unit on edge2 host.

IP Allocation Scheme

Range Purpose
.1-.10 Network infrastructure
.11-.99 DHCP clients
.100-.149 LXC containers
.150-.199 VMs
.240-.250 Proxmox hosts + bare metal
.251-.254 Meshtastic nodes

Full details: ip-allocation

Headscale Node List

Current registered nodes (25 total):

Node Tailscale IP Type
contabo 100.64.0.40 VPS (edge1, mail-only, rebuilt 2026-06-19; formerly 100.64.0.1)
immich 100.64.0.2 LXC
media 100.64.0.3 Proxmox
cloud 100.64.0.4 Proxmox
utility 100.64.0.5 Proxmox
data 100.64.0.6 Proxmox
meshmonitor 100.64.0.7 LXC
caddy 100.64.0.8 LXC
aida-nebra 100.64.0.9 Pi
matt-desktop 100.64.0.10 Desktop
nextcloud 100.64.0.11 LXC
toc 100.64.0.13 Proxmox
cortex 100.64.0.14 VM
searxng 100.64.0.15 LXC
ha 100.64.0.16 VM (cloud VM 151 — node id 54, user echo6; home automation)
arr 100.64.0.18 VM
pi-nas 100.64.0.21 Pi
mesh-bridge 100.64.0.22 LXC
peertube-4hve9pdr 100.64.0.17 LXC (collision-suffixed identity since ~2026-06-22; formerly peertube at 100.64.0.23, now stale/gone)
recon 100.64.0.24 VM
argus 100.64.0.25 LXC
central 100.64.0.12 LXC (utility CT 104 — central.echo6.mesh)
edge2 100.64.0.26 Proxmox/Contabo VPS
gl-a1300 100.64.0.29 Router
bluefin 100.64.0.30 Desktop
wordpress 100.64.0.31 LXC
meshai 100.64.0.32 LXC
vaultwarden 100.64.0.33 LXC (edge2 CT 102)
forgejo 100.64.0.34 LXC (edge2 CT 103) — node id 46
livesync 100.64.0.35 LXC (edge2 CT 104) — migrated 2026-06-16
authentik 100.64.0.36 LXC (edge2 CT 105) — node id 48, migrated 2026-06-18
matrix 100.64.0.37 LXC (edge2 CT 106) — migrated 2026-06-18
headscale 100.64.0.38 LXC (edge2 CT 107) — migrated 2026-06-19

IdahoMesh Headscale Node List

Separate Headscale instance on CT 106 (192.168.1.106), prefix 100.100.0.0/16. Reachable from echo6 tailnet via mesh-bridge (CT 107).

Node Tailscale IP User Type
mesh-bridge 100.100.0.3 malice LXC (bridge)
burley-butte 100.100.0.1 nebra Pi (offline)
mt-isr 100.100.0.5 nebra Pi Zero 2 W

SSH Access

Standard user: zvx Credentials: Source from /home/zvx/projects/.ref/credentials

Most servers use SSH key auth. Exceptions noted below.

# SSH to any server (key auth)
ssh zvx@<ip-address>

# Examples
ssh zvx@192.168.1.244  # TOC (Proxmox host)
ssh zvx@192.168.1.150  # cortex VM
ssh zvx@192.168.1.241  # utility Proxmox
ssh -i ~/.ssh/contabo2_ed25519 root@5.189.158.149  # edge1 (mail-only; formerly Contabo)
ssh zvx@cortex         # cortex via Tailscale hostname
ssh edge2              # edge2 via SSH config alias (admin@100.64.0.26)

edge2 SSH access

edge2 is hardened: PasswordAuthentication no, root login disabled. SSH user is admin with passwordless sudo. The SSH config alias edge2 maps to admin@100.64.0.26.

Authorized keys on edge2 (/home/admin/.ssh/authorized_keys):

  • echo6-contabo2-184.174.35.153 — the original provisioning key (cortex ~/.ssh/contabo2_ed25519)
  • cortex — cortex default id_ed25519
  • matt-desktop-wsl2 — WSL2 default key
  • cookie-sync — Windows native SSH key

To access CTs on edge2: ssh edge2 then sudo pct exec <vmid> -- bash.

Password-auth hosts

These require password authentication (no SSH keys installed):

Host User Password Access
aida-nebra zvx 7redditGold sshpass -p '7redditGold' ssh zvx@aida-nebra
mt-isr isr UfPsfwyMIUIKb1 sshpass -p 'UfPsfwyMIUIKb1' ssh isr@192.168.1.141
mt-burleybutte bb (see credentials) sshpass -p '<pw>' ssh bb@192.168.1.185
matt-desktop administrator Qw1290opzx ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no administrator@192.168.1.254
toc root 7redditGold sshpass -p '7redditGold' ssh -o PubkeyAuthentication=no root@100.64.0.13

Use the Tailscale hostname (aida-nebra) or local IP (192.168.1.253) — both work for aida-nebra. mt-isr is on IdahoMesh tailnet (100.100.0.5) — reachable from echo6 via bridge. matt-desktop is accessible via local IP (192.168.1.254) or Tailscale (100.64.0.10) — requires explicit password auth flags.

Key External IPs

Purpose IP
Home external (public services) 199.6.36.163
edge1 (mail-only, rebuilt Contabo VPS) 5.189.158.149
edge2 (front door: vault/forge/notes/auth/matrix/element/vpn/proxmox + idahomesh/intermountainmesh) 184.174.35.153