echo6-docs/vault/docs/hardware/environment.md

title	type	tags	aliases	related	updated
Echo6 Environment Reference	reference	proxmox		fleet-platform-baseline fleet-storage-memory-upgrade ip-allocation toc-cortex-pve9.2-update pve-guest-park-and-adopt	2026-08-15

Echo6 Environment Reference

Proxmox Cluster (echo6-cluster)

Five nodes running Proxmox VE:

Node	Local IP	Tailscale	Hardware	RAM	Purpose
data	192.168.1.240	100.64.0.6	AMD Ryzen 7 PRO 5750GE, 1TB NVMe + 1TB SATA SSD	32GB DDR4-3200	Database services
utility	192.168.1.241	100.64.0.5	AMD Ryzen 7 PRO 5750GE, 512GB NVMe	32GB DDR4-3200	Utility services, monitoring
cloud	192.168.1.242	100.64.0.4	Intel i7-12700T, 512GB NVMe	32GB DDR4-3200	Cloud storage, personal services
media	192.168.1.243	100.64.0.3	Intel i7-14700T, 512GB + 2TB NVMe	64GB DDR5-4800	Media server, *arr stack
toc	192.168.1.244	100.64.0.13	Workstation (i9-10900X)	64GB DDR4	GPU compute, AI/ML workloads

Node Storage Details

Node	Primary Disk	Secondary Disk
data	Samsung SSD 980 1TB (NVMe)	SanDisk SDSSDH3 1TB (SATA SSD)
utility	WD PC SN740 512GB (NVMe)	—
cloud	SK Hynix HFS512GEJ9X164N 512GB (NVMe)	—
media	Intel SSDPEKNU512GZH 512GB (NVMe, boot/pve)	WD Green SN350 2TB (NVMe, VG tank)
toc	512GB NVMe	—

media upgraded 2026-08-15: 32 → 64 GB DDR5-4800 (2 × 32 GB, board maximum reached), and the second M.2 — which held a leftover BitLocker Windows install — replaced with a 2 TB WD Green SN350. That drive is VG tank as a single 1.82 TB volume — tank/data, ext4, mounted /mnt/nvme2tb, registered as Proxmox dir storage media-nvme (nodes media-scoped). Deliberately not split into separate pools: navi's geo data is the intended consumer and it needs one large contiguous filesystem it can grow into. The displaced Windows data was archived to pi-nas first — see fleet-storage-memory-upgrade.

Free positions (verified live 2026-08-14): data has none — only three external PCIe root ports exist and all are populated (NVMe, NIC, Wi-Fi), and the Wi-Fi M.2 is E-keyed so it cannot take a storage drive. utility and cloud each have an empty 2.5" SATA bay. media's second M.2 holds a leftover BitLocker Windows install (serial PHKA142402U8512A; the Proxmox drive is PHKA142504HP512A). toc has 8 unpopulated SATA ports and 4 free PCIe slots. pi-nas has one free SATA port (ata5). Placement plan is fleet-storage-memory-upgrade.

Memory ceilings: data and utility are hard-capped at 32 GB and already there. cloud and media max at 64 GB. toc has 2 of 8 DIMM slots free but takes full-size DDR4 DIMMs, not SODIMM, and currently runs all modules at 2133 MT/s because rated speeds are mixed (3200/2666/2133).

Cluster Backup Storage

pinas-backup — NFS 192.168.1.245:/export/pvebackup, vers=3, content backup, mounted at /mnt/pve/pinas-backup on all five nodes. Backed by pi-nas sdc1 (~19 TB free), deliberately not sdd1 which carries PeerTube. Added 2026-08-14 for park-and-adopt guest relocation — see pve-guest-park-and-adopt and omv-add-nfs-share.

Non-Cluster Physical Hosts

Host	Local IP	Tailscale	Hardware
pi-nas	192.168.1.245	100.64.0.21	Raspberry Pi 5, 8GB soldered, JMicron JMB58x 5-port SATA HBA, 2x 3TB btrfs RAID1 + 2x 24TB single ext4
aida-nebra	192.168.1.253	100.64.0.9	Raspberry Pi Compute Module 3, Cortex-A53, 906MiB soldered, eMMC boot, RAK4631 on ttyACM1

Neither is upgradable — soldered RAM, no DIMM sockets.

Node Hardware Identifiers

Node	Make/Model	Serial/Service Tag
data	Lenovo ThinkCentre M75q Gen 2 (11JN002RUS)	MZ010LPV
utility	Lenovo ThinkCentre M75q Gen 2 (11JN002RUS)	MJ0LZNYT
cloud	Lenovo ThinkCentre M70q Gen 3 (11T3000RUS)	MJ0LQCGJ
media	Dell OptiPlex Micro 7020	58SM6X3 (BIOS 1.20.0, 2025-09-04)
toc	HP Z4 G4 Workstation	MXL2383MVK

No BMC/IPMI/iDRAC on any node (verified dmidecode -t 38 empty on all five, 2026-07-14) — there is no remote power-cycle path for any Proxmox host. Recovery from a hung or powered-off node requires physical access.

Network Notes

• media NIC: Original Intel e1000e NIC (nic0, MAC e8:cf:83:20:8b:cb) crashes under sustained NFS load — present but DOWN/unused. Sole uplink is a USB Realtek RTL8153 GbE dongle (MAC 0c:37:96:0e:e8:53) on vmbr0.
• Tailscale dns bootstrap: All LXC containers with Tailscale have a systemd drop-in (/etc/systemd/system/tailscaled.service.d/dns-bootstrap.conf) that ensures fallback dns exists before tailscaled starts, preventing chicken-and-egg dns resolution failures on reboot

TOC Node Details

• Hardware: Intel i9-10900X (20 threads), 64GB RAM (4×8GB + 2×16GB DDR4), 512GB NVMe, RTX A4000
• GPU: Passed through via VFIO to VM 150 (cortex), not used on host
• VMID ranges: 100-149 (LXC), 150-199 (VMs)
• Presave backup: /home/zvx/toc-presave/ on Contabo (1.8G) — contains old Ubuntu config

Virtual Machines

VM	Host	VMID	Local IP	Tailscale	Purpose
cortex	toc	150	192.168.1.150	100.64.0.14	GPU compute — Open WebUI, Ollama, Qdrant, TEI, Claude Code
recon-vm	data	1130	192.168.1.130	100.64.0.24	recon + navi platforms, Files, Kiwix, geo backends
arr	media	105	192.168.1.160	100.64.0.18	ARR media automation stack (Jellyfin, Sonarr, Radarr, etc.)

cortex VM Details

• OS: Ubuntu 24.04 (cloud-init), kernel 6.8.0-100-generic
• Resources: 16 threads, 32GB RAM, 300GB disk
• Swap: 32GB swapfile (/swapfile), vm.swappiness=10 (prefer RAM, swap under pressure only). Provisioned 2026-04-23 to support bursty jobs (gdal_contour, tippecanoe) alongside Docker ML containers.
• GPU: RTX A4000 (passthrough), NVIDIA driver 580.126.09, CUDA 13.0
• Software: Docker 29.2.1 + nvidia-container-toolkit 1.18.2, Node.js 22.22.0, Python 3.12.3
• Docker containers: open-webui (8080), ollama (11434 w/ GPU), qdrant (6333), tei (8090)
• User: zvx (sudo, SSH keys from cluster)
• Claude Code: installed

recon-vm Details

• OS: Ubuntu 24.04.4 LTS (cloud-init), kernel 6.8.0-110-generic
• Resources: 4 cores, 24GB RAM, 180GB disk
• Software: Docker 29.4.0, Python 3.12.3, nginx, sqlite3, Tailscale
• Platforms: recon (knowledge extraction pipeline, :8420) and navi (offline navigation, navi.echo6.co, :8440) with geo backends (Valhalla :8002, Nominatim :8010, Photon :2322, PostgreSQL/PostGIS :5432)
• Systemd services: recon (8420), recon-watchdog, kiwix (8430), nginx (8888)
• NFS mounts: pi-nas:/export/library → /mnt/library, /mnt/nav, /mnt/kiwix
• User: zvx (sudo, SSH key auth)
• Migrated from: CT 130 (LXC) on 2026-04-19. Tailscale identity preserved (100.64.0.24).
• Note: Old CT 130 (LXC) has since been destroyed (verified 2026-06-15) — only VM 1130 remains.

arr VM Details

• OS: Ubuntu 24.04 (cloud-init)
• Resources: 4 cores, 8GB RAM, 30GB disk on local-lvm
• Software: Docker 29.2.1, Tailscale, NFS client, sshpass, qemu-guest-agent
• User: zvx (sudo, SSH key from cortex)
• NFS: pi-nas:/export/arr → /mnt/arr (22TB, movies/tv/downloads)
• Docker containers: jellyfin (8096), jellyseer (5055), sonarr (8989), radarr (7878), prowlarr (9696), sabnzbd (8080), lidarr (8686/6595), navidrome (4533)
• Docker network: arr-net (bridge)
• Config dirs: /opt/arr/{jellyfin,jellyseer,sonarr,radarr,prowlarr,sabnzbd,lidarr}

Key Servers

Server	Local IP	Tailscale	Purpose
aida-nebra	192.168.1.253	100.64.0.9	AIDA-N2(RPT,LLM) — meshtasticd node !27780c47, Nebra 2W hat, port 4403. meshai (CT 108) connects here via TCP
mt-isr	192.168.1.141	100.100.0.5 (IdahoMesh)	Meshtastic sidecar Pi (G2 WiFi bridge, meshtasticd, CLI)
mt-burleybutte	192.168.1.185	—	Meshtastic node (meshtasticd, Nebra 2W hat, IdahoMesh VPN)
pi-nas	192.168.1.245	100.64.0.21	Raspberry Pi NAS
matt-desktop	192.168.1.254	100.64.0.10	Personal workstation (Windows, your PC)
ha	192.168.1.151	100.64.0.16	Home Assistant (VM 151 on cloud, Docker, home automation)

| edge1 (rebuilt Contabo VPS) | 5.189.158.149 | 100.64.0.40 | Debian 12 + Proxmox 8.4.19, mail-only — Mailcow in CT 101; host caddy + mailcow-dnat.service; rebuilt 2026-06-19 | | edge2 | 184.174.35.153 | 100.64.0.26 | Contabo Cloud VPS 30 NVMe — Proxmox VE 8.4.19 (LXC-only), 8c/24GB/400GB — permanent front door for vault/forge/notes/auth/matrix/element/vpn/proxmox.echo6.co + idahomesh.com |

Last updated: 2026-06-19 — Contabo VPS rebuilt as edge1 (mail-only, Debian 12 + Proxmox 8.4.19, 5.189.158.149 / tailnet 100.64.0.40); Mailcow CT 101 (10.10.10.2) on edge1; edge2 is now the permanent front door for all other services; Headscale node contabo moved to 100.64.0.40; previously added edge2 CT 107 (headscale), CT 106 (matrix), CT 105 (authentik), CT 104 (livesync), CT 103 (forgejo), CT 102 (vaultwarden)

LXC Containers

Container	Host	Local IP	Tailscale	Purpose
meshmonitor	utility (CT 100)	192.168.1.100	100.64.0.7	Meshtastic mesh monitoring (zvx-echo6/meshmonitor fork, multi-channel)
caddy	utility (CT 101)	192.168.1.101	100.64.0.8	Home reverse proxy
searxng	utility (CT 102)	192.168.1.102	100.64.0.15	Echo6 Search homepage (searxng, echo6.co)
immich	cloud (CT 120)	192.168.1.182	100.64.0.2	Immich photo management
nextcloud	cloud (CT 121)	192.168.1.183	100.64.0.11	Nextcloud AIO
meshtastic-hs	utility (CT 106)	192.168.1.106	—	IdahoMesh Headscale VPN coordination
mesh-bridge	utility (CT 107)	192.168.1.107	100.64.0.22	Dual-tailscaled bridge (echo6 ↔ idahomesh)
meshai	utility (CT 108)	192.168.1.144	100.64.0.32	meshai - LLM-powered Meshtastic assistant
archivist	utility (CT 118)	192.168.1.118	—	archivist knowledge pipeline
argus	utility (CT 103)	192.168.1.103	100.64.0.25	argus - OSINT intelligence gathering platform
central	utility (CT 104)	192.168.1.104	100.64.0.12	Data-hub spine (central.echo6.mesh) — ~25 adapters, NATS/JetStream, TimescaleDB/PostGIS — see central
peertube	media (CT 110)	192.168.1.170	100.64.0.17	PeerTube video streaming — Tailscale identity is peertube-4hve9pdr (collision-suffixed) since ~2026-06-22; .23 is stale/gone
mcc	media (CT 111)	192.168.1.111	100.64.0.19	pymc console web app (caddy + Postfix, /api+/auth+/ws → aida-nebra :8000)
mailcow	edge1 (CT 101)	10.10.10.2	—	Mailcow email server (privileged LXC, mail-only host; reached via host DNAT + caddy)
pdm	edge2 (CT 100)	10.10.10.10	100.64.0.28	Proxmox Datacenter Manager
wordpress	edge2 (CT 101)	10.10.10.11	100.64.0.31	Grav CMS 2.0.11 for idahomesh.com (migrated from WordPress 2026-07-17; hostname wordpress unchanged; MariaDB purged — flat-file, no DB)
vaultwarden	edge2 (CT 102)	10.10.10.20	100.64.0.33	Vaultwarden password manager (migrated from Contabo 2026-06-16)
forgejo	edge2 (CT 103)	10.10.10.21	100.64.0.34	Forgejo git server (migrated from Contabo 2026-06-16)
livesync	edge2 (CT 104)	10.10.10.22	100.64.0.35	LiveSync Obsidian sync (CouchDB + provisioner; migrated from Contabo 2026-06-16)
authentik	edge2 (CT 105)	10.10.10.23	100.64.0.36	authentik SSO platform (migrated from Contabo 2026-06-18)
matrix	edge2 (CT 106)	10.10.10.24	100.64.0.37	Matrix stack (synapse + MAS + Element + mautrix_signal; migrated from Contabo 2026-06-18)
headscale	edge2 (CT 107)	10.10.10.25	100.64.0.38	Headscale + Headplane tailnet control plane (migrated from Contabo 2026-06-19)

Note (2026-06-19): edge2 CT placements CT 102–107 confirmed; Forge git-SSH DNAT (forgejo-ssh-dnat.service) is a permanent systemd unit on edge2 host.

IP Allocation Scheme

Range	Purpose
.1-.10	Network infrastructure
.11-.99	DHCP clients
.100-.149	LXC containers
.150-.199	VMs
.240-.250	Proxmox hosts + bare metal
.251-.254	Meshtastic nodes

Full details: ip-allocation

Headscale Node List

Current registered nodes (25 total):

Node	Tailscale IP	Type
contabo	100.64.0.40	VPS (edge1, mail-only, rebuilt 2026-06-19; formerly 100.64.0.1)
immich	100.64.0.2	LXC
media	100.64.0.3	Proxmox
cloud	100.64.0.4	Proxmox
utility	100.64.0.5	Proxmox
data	100.64.0.6	Proxmox
meshmonitor	100.64.0.7	LXC
caddy	100.64.0.8	LXC
aida-nebra	100.64.0.9	Pi
matt-desktop	100.64.0.10	Desktop
nextcloud	100.64.0.11	LXC
toc	100.64.0.13	Proxmox
cortex	100.64.0.14	VM
searxng	100.64.0.15	LXC
ha	100.64.0.16	VM (cloud VM 151 — node id 54, user echo6; home automation)
arr	100.64.0.18	VM
pi-nas	100.64.0.21	Pi
mesh-bridge	100.64.0.22	LXC
peertube-4hve9pdr	100.64.0.17	LXC (collision-suffixed identity since ~2026-06-22; formerly peertube at 100.64.0.23, now stale/gone)
recon	100.64.0.24	VM
argus	100.64.0.25	LXC
central	100.64.0.12	LXC (utility CT 104 — central.echo6.mesh)
edge2	100.64.0.26	Proxmox/Contabo VPS
gl-a1300	100.64.0.29	Router
bluefin	100.64.0.30	Desktop
wordpress	100.64.0.31	LXC (edge2 CT 101 — now runs Grav CMS; hostname unchanged)
meshai	100.64.0.32	LXC
vaultwarden	100.64.0.33	LXC (edge2 CT 102)
forgejo	100.64.0.34	LXC (edge2 CT 103) — node id 46
livesync	100.64.0.35	LXC (edge2 CT 104) — migrated 2026-06-16
authentik	100.64.0.36	LXC (edge2 CT 105) — node id 48, migrated 2026-06-18
matrix	100.64.0.37	LXC (edge2 CT 106) — migrated 2026-06-18
headscale	100.64.0.38	LXC (edge2 CT 107) — migrated 2026-06-19

IdahoMesh Headscale Node List

Separate Headscale instance on CT 106 (192.168.1.106), prefix 100.100.0.0/16. Reachable from echo6 tailnet via mesh-bridge (CT 107).

Node	Tailscale IP	User	Type
mesh-bridge	100.100.0.3	malice	LXC (bridge)
burley-butte	100.100.0.1	nebra	Pi (offline)
mt-isr	100.100.0.5	nebra	Pi Zero 2 W

SSH Access

Standard user: zvx Credentials: Source from /home/zvx/projects/.ref/credentials

Most servers use SSH key auth. Exceptions noted below.

# SSH to any server (key auth)
ssh zvx@<ip-address>

# Examples
ssh zvx@192.168.1.244  # TOC (Proxmox host)
ssh zvx@192.168.1.150  # cortex VM
ssh zvx@192.168.1.241  # utility Proxmox
ssh -i ~/.ssh/contabo2_ed25519 root@5.189.158.149  # edge1 (mail-only; formerly Contabo)
ssh zvx@cortex         # cortex via Tailscale hostname
ssh edge2              # edge2 via SSH config alias (admin@100.64.0.26)

edge2 SSH access

edge2 is hardened: PasswordAuthentication no, root login disabled. SSH user is admin with passwordless sudo. The SSH config alias edge2 maps to admin@100.64.0.26.

Authorized keys on edge2 (/home/admin/.ssh/authorized_keys):

• echo6-contabo2-184.174.35.153 — the original provisioning key (cortex ~/.ssh/contabo2_ed25519)
• cortex — cortex default id_ed25519
• matt-desktop-wsl2 — WSL2 default key
• cookie-sync — Windows native SSH key

To access CTs on edge2: ssh edge2 then sudo pct exec <vmid> -- bash.

Password-auth hosts

These require password authentication (no SSH keys installed):

Host	User	Password	Access
aida-nebra	zvx	7redditGold	sshpass -p '7redditGold' ssh zvx@aida-nebra
mt-isr	isr	UfPsfwyMIUIKb1	sshpass -p 'UfPsfwyMIUIKb1' ssh isr@192.168.1.141
mt-burleybutte	bb	(see credentials)	sshpass -p '<pw>' ssh bb@192.168.1.185
matt-desktop	administrator	Qw1290opzx	ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no administrator@192.168.1.254
toc	root	7redditGold	sshpass -p '7redditGold' ssh -o PubkeyAuthentication=no root@100.64.0.13

Use the Tailscale hostname (aida-nebra) or local IP (192.168.1.253) — both work for aida-nebra. mt-isr is on IdahoMesh tailnet (100.100.0.5) — reachable from echo6 via bridge. matt-desktop is accessible via local IP (192.168.1.254) or Tailscale (100.64.0.10) — requires explicit password auth flags.

Key External IPs

Purpose	IP
Home external (public services)	199.6.36.163
edge1 (mail-only, rebuilt Contabo VPS)	5.189.158.149
edge2 (front door: vault/forge/notes/auth/matrix/element/vpn/proxmox + idahomesh.com)	184.174.35.153