echo6-docs/vault/docs/hardware/environment.md

---
title: Echo6 Environment Reference
type: reference
tags:
  - proxmox
aliases: []
related:
  - [[fleet-platform-baseline]]
  - [[fleet-storage-memory-upgrade]]
  - [[ip-allocation]]
  - [[toc-cortex-pve9.2-update]]
  - [[pve-guest-park-and-adopt]]
updated: 2026-08-15
---
# Echo6 Environment Reference

## Proxmox Cluster (echo6-cluster)

Five nodes running Proxmox VE:

| Node    | Local IP      | Tailscale   | Hardware                                        | RAM            | Purpose                          |
| ------- | ------------- | ----------- | ----------------------------------------------- | -------------- | -------------------------------- |
| data    | 192.168.1.240 | 100.64.0.6  | AMD Ryzen 7 PRO 5750GE, 1TB NVMe + 1TB SATA SSD | 32GB DDR4-3200 | Database [[services]]            |
| utility | 192.168.1.241 | 100.64.0.5  | AMD Ryzen 7 PRO 5750GE, 512GB NVMe              | 32GB DDR4-3200 | Utility [[services]], monitoring     |
| cloud   | 192.168.1.242 | 100.64.0.4  | Intel i7-12700T, 512GB NVMe                     | 32GB DDR4-3200 | Cloud storage, personal [[services]] |
| media   | 192.168.1.243 | 100.64.0.3  | Intel i7-14700T, 512GB + 2TB NVMe               | 64GB DDR5-4800 | Media server, *arr stack         |
| toc     | 192.168.1.244 | 100.64.0.13 | Workstation (i9-10900X)                         | 64GB DDR4      | GPU compute, AI/ML workloads     |
### Node Storage Details

| Node | Primary Disk | Secondary Disk |
|------|-------------|----------------|
| data | Samsung SSD 980 1TB (NVMe) | SanDisk SDSSDH3 1TB (SATA SSD) |
| utility | WD PC SN740 512GB (NVMe) | — |
| cloud | SK Hynix HFS512GEJ9X164N 512GB (NVMe) | — |
| media | Intel SSDPEKNU512GZH 512GB (NVMe, boot/`pve`) | WD Green SN350 2TB (NVMe, VG `tank`) |
| toc | 512GB NVMe | — |

**media upgraded 2026-08-15:** 32 → 64 GB DDR5-4800 (2 × 32 GB, board maximum reached), and the second M.2 — which held a leftover BitLocker Windows install — replaced with a 2 TB WD Green SN350. That drive is VG `tank` as a **single 1.82 TB volume** — `tank/data`, ext4, mounted `/mnt/nvme2tb`, registered as Proxmox dir storage `media-nvme` (`nodes media`-scoped). Deliberately not split into separate pools: [[navi]]'s geo data is the intended consumer and it needs one large contiguous filesystem it can grow into. The displaced Windows data was archived to pi-nas first — see [[fleet-storage-memory-upgrade]].

**Free positions (verified live 2026-08-14):** data has none — only three external PCIe root ports exist and all are populated (NVMe, NIC, Wi-Fi), and the Wi-Fi M.2 is E-keyed so it cannot take a storage drive. utility and cloud each have an empty 2.5" SATA bay. media's second M.2 holds a leftover BitLocker Windows install (serial `PHKA142402U8512A`; the Proxmox drive is `PHKA142504HP512A`). toc has 8 unpopulated SATA ports and 4 free PCIe slots. pi-nas has one free SATA port (`ata5`). Placement plan is [[fleet-storage-memory-upgrade]].

**Memory ceilings:** data and utility are hard-capped at 32 GB and already there. cloud and media max at 64 GB. toc has 2 of 8 DIMM slots free but takes full-size DDR4 DIMMs, not SODIMM, and currently runs all modules at 2133 MT/s because rated speeds are mixed (3200/2666/2133).

### Cluster Backup Storage

`pinas-backup` — NFS `192.168.1.245:/export/pvebackup`, `vers=3`, content `backup`, mounted at `/mnt/pve/pinas-backup` on all five nodes. Backed by pi-nas `sdc1` (~19 TB free), deliberately not `sdd1` which carries PeerTube. Added 2026-08-14 for park-and-adopt guest relocation — see [[pve-guest-park-and-adopt]] and [[omv-add-nfs-share]].

### Non-Cluster Physical Hosts

| Host | Local IP | Tailscale | Hardware |
|------|----------|-----------|----------|
| pi-nas | 192.168.1.245 | 100.64.0.21 | Raspberry Pi 5, 8GB soldered, JMicron JMB58x 5-port SATA HBA, 2x 3TB btrfs RAID1 + 2x 24TB single ext4 |
| aida-nebra | 192.168.1.253 | 100.64.0.9 | Raspberry Pi Compute Module 3, Cortex-A53, 906MiB soldered, eMMC boot, RAK4631 on ttyACM1 |

Neither is upgradable — soldered RAM, no DIMM sockets.

### Node Hardware Identifiers

| Node | Make/Model | Serial/Service Tag |
|------|-----------|---------------------|
| data | Lenovo ThinkCentre M75q Gen 2 (11JN002RUS) | MZ010LPV |
| utility | Lenovo ThinkCentre M75q Gen 2 (11JN002RUS) | MJ0LZNYT |
| cloud | Lenovo ThinkCentre M70q Gen 3 (11T3000RUS) | MJ0LQCGJ |
| media | Dell OptiPlex Micro 7020 | 58SM6X3 (BIOS 1.20.0, 2025-09-04) |
| toc | HP Z4 G4 Workstation | MXL2383MVK |

**No BMC/IPMI/iDRAC on any node** (verified `dmidecode -t 38` empty on all five, 2026-07-14) — there is no remote power-cycle path for any Proxmox host. Recovery from a hung or powered-off node requires physical access.

### Network Notes

- **media NIC:** Original Intel e1000e NIC (`nic0`, MAC `e8:cf:83:20:8b:cb`) crashes under sustained NFS load — present but DOWN/unused. Sole uplink is a USB Realtek RTL8153 GbE dongle (MAC `0c:37:96:0e:e8:53`) on vmbr0.
- **Tailscale [[dns]] bootstrap:** All LXC containers with Tailscale have a systemd drop-in (`/etc/systemd/system/tailscaled.service.d/dns-bootstrap.conf`) that ensures fallback [[dns]] exists before tailscaled starts, preventing chicken-and-egg [[dns]] resolution failures on reboot

### TOC Node Details

- **Hardware:** Intel i9-10900X (20 threads), 64GB RAM (4×8GB + 2×16GB DDR4), 512GB NVMe, RTX A4000
- **GPU:** Passed through via VFIO to VM 150 (cortex), not used on host
- **VMID ranges:** 100-149 (LXC), 150-199 (VMs)
- **Presave backup:** `/home/zvx/toc-presave/` on Contabo (1.8G) — contains old Ubuntu config

## Virtual Machines

| VM | Host | VMID | Local IP | Tailscale | Purpose |
|----|------|------|----------|-----------|---------|
| cortex | toc | 150 | 192.168.1.150 | 100.64.0.14 | GPU compute — Open WebUI, Ollama, Qdrant, TEI, Claude Code |
| recon-vm | data | 1130 | 192.168.1.130 | 100.64.0.24 | [[recon]] + [[navi]] platforms, Files, Kiwix, geo backends |
| arr | media | 105 | 192.168.1.160 | 100.64.0.18 | ARR media automation stack (Jellyfin, Sonarr, Radarr, etc.) |

### cortex VM Details

- **OS:** Ubuntu 24.04 (cloud-init), kernel 6.8.0-100-generic
- **Resources:** 16 threads, 32GB RAM, 300GB disk
- **Swap:** 32GB swapfile (`/swapfile`), `vm.swappiness=10` (prefer RAM, swap under pressure only). Provisioned 2026-04-23 to support bursty jobs (gdal_contour, tippecanoe) alongside Docker ML containers.
- **GPU:** RTX A4000 (passthrough), NVIDIA driver 580.126.09, CUDA 13.0
- **Software:** Docker 29.2.1 + nvidia-container-toolkit 1.18.2, Node.js 22.22.0, Python 3.12.3
- **Docker containers:** open-webui (8080), ollama (11434 w/ GPU), qdrant (6333), tei (8090)
- **User:** zvx (sudo, SSH keys from cluster)
- **Claude Code:** installed

### recon-vm Details

- **OS:** Ubuntu 24.04.4 LTS (cloud-init), kernel 6.8.0-110-generic
- **Resources:** 4 cores, 24GB RAM, 180GB disk
- **Software:** Docker 29.4.0, Python 3.12.3, nginx, sqlite3, Tailscale
- **Platforms:** [[recon]] (knowledge extraction pipeline, :8420) and [[navi]] (offline navigation, navi.echo6.co, :8440) with geo backends (Valhalla :8002, Nominatim :8010, Photon :2322, PostgreSQL/PostGIS :5432)
- **Systemd [[services]]:** [[recon]] (8420), recon-watchdog, kiwix (8430), nginx (8888)
- **NFS mounts:** pi-nas:/export/library → /mnt/library, /mnt/nav, /mnt/kiwix
- **User:** zvx (sudo, SSH key auth)
- **Migrated from:** CT 130 (LXC) on 2026-04-19. Tailscale identity preserved (100.64.0.24).
- **Note:** Old CT 130 (LXC) has since been destroyed (verified 2026-06-15) — only VM 1130 remains.

### arr VM Details

- **OS:** Ubuntu 24.04 (cloud-init)
- **Resources:** 4 cores, 8GB RAM, 30GB disk on local-lvm
- **Software:** Docker 29.2.1, Tailscale, NFS client, sshpass, qemu-guest-agent
- **User:** zvx (sudo, SSH key from cortex)
- **NFS:** pi-nas:/export/arr → /mnt/arr (22TB, movies/tv/downloads)
- **Docker containers:** jellyfin (8096), jellyseer (5055), sonarr (8989), radarr (7878), prowlarr (9696), sabnzbd (8080), lidarr (8686/6595), navidrome (4533)
- **Docker network:** arr-net (bridge)
- **Config dirs:** /opt/arr/{jellyfin,jellyseer,sonarr,radarr,prowlarr,sabnzbd,lidarr}

## Key Servers

| Server | Local IP | Tailscale | Purpose |
|--------|----------|-----------|---------|
| aida-nebra | 192.168.1.253 | 100.64.0.9 | AIDA-N2(RPT,LLM) — meshtasticd node !27780c47, Nebra 2W hat, port 4403. [[meshai]] (CT 108) connects here via TCP |
| mt-isr | 192.168.1.141 | 100.100.0.5 (IdahoMesh) | Meshtastic sidecar Pi (G2 WiFi bridge, meshtasticd, CLI) |
| mt-burleybutte | 192.168.1.185 | — | Meshtastic node (meshtasticd, Nebra 2W hat, IdahoMesh VPN) |
| pi-nas | 192.168.1.245 | 100.64.0.21 | Raspberry Pi NAS |
| matt-desktop | 192.168.1.254 | 100.64.0.10 | Personal workstation (Windows, your PC) |
| ha | 192.168.1.151 | 100.64.0.16 | Home Assistant (VM 151 on cloud, Docker, home automation) |

| **edge1** (rebuilt Contabo VPS) | 5.189.158.149 | 100.64.0.40 | Debian 12 + Proxmox 8.4.19, **mail-only** — Mailcow in CT 101; host [[caddy]] + mailcow-dnat.service; rebuilt [[2026-06-19]] |
| edge2 | 184.174.35.153 | 100.64.0.26 | Contabo Cloud VPS 30 NVMe — Proxmox VE 8.4.19 (LXC-only), 8c/24GB/400GB — **permanent front door** for vault/forge/notes/auth/matrix/element/vpn/proxmox.echo6.co + idahomesh.com |

*Last updated: [[2026-06-19]] — Contabo VPS rebuilt as edge1 (mail-only, Debian 12 + Proxmox 8.4.19, 5.189.158.149 / tailnet 100.64.0.40); Mailcow CT 101 (10.10.10.2) on edge1; edge2 is now the permanent front door for all other [[services]]; Headscale node `contabo` moved to 100.64.0.40; previously added edge2 CT 107 (headscale), CT 106 (matrix), CT 105 ([[authentik]]), CT 104 (livesync), CT 103 (forgejo), CT 102 (vaultwarden)*

## LXC Containers

| Container | Host | Local IP | Tailscale | Purpose |
|-----------|------|----------|-----------|---------|
| meshmonitor | utility (CT 100) | 192.168.1.100 | 100.64.0.7 | Meshtastic mesh monitoring (zvx-echo6/meshmonitor fork, multi-channel) |
| [[caddy]] | utility (CT 101) | 192.168.1.101 | 100.64.0.8 | Home reverse proxy |
| [[searxng]] | utility (CT 102) | 192.168.1.102 | 100.64.0.15 | Echo6 Search homepage ([[searxng]], echo6.co) |
| immich | cloud (CT 120) | 192.168.1.182 | 100.64.0.2 | Immich photo management |
| nextcloud | cloud (CT 121) | 192.168.1.183 | 100.64.0.11 | Nextcloud AIO |
| meshtastic-hs | utility (CT 106) | 192.168.1.106 | — | IdahoMesh Headscale VPN coordination |
| mesh-bridge | utility (CT 107) | 192.168.1.107 | 100.64.0.22 | Dual-tailscaled bridge (echo6 ↔ idahomesh) |
| [[meshai]] | utility (CT 108) | 192.168.1.144 | 100.64.0.32 | [[meshai]] - LLM-powered Meshtastic assistant |
| [[archivist]] | utility (CT 118) | 192.168.1.118 | — | [[archivist]] knowledge pipeline |
| [[argus]] | utility (CT 103) | 192.168.1.103 | 100.64.0.25 | [[argus]] - OSINT intelligence gathering platform |
| [[central]] | utility (CT 104) | 192.168.1.104 | 100.64.0.12 | Data-hub spine (central.echo6.mesh) — ~25 adapters, NATS/JetStream, TimescaleDB/PostGIS — see [[central]] |
| peertube | media (CT 110) | 192.168.1.170 | 100.64.0.17 | PeerTube video streaming — Tailscale identity is `peertube-4hve9pdr` (collision-suffixed) since ~2026-06-22; `.23` is stale/gone |
| mcc | media (CT 111) | 192.168.1.111 | 100.64.0.19 | pymc console web app ([[caddy]] + Postfix, /api+/auth+/ws → aida-nebra :8000) |
| mailcow | edge1 (CT 101) | 10.10.10.2 | — | Mailcow email server (privileged LXC, mail-only host; reached via host DNAT + [[caddy]]) |
| pdm | edge2 (CT 100) | 10.10.10.10 | 100.64.0.28 | Proxmox Datacenter Manager |
| wordpress | edge2 (CT 101) | 10.10.10.11 | 100.64.0.31 | Grav CMS 2.0.11 for idahomesh.com (migrated from WordPress 2026-07-17; hostname `wordpress` unchanged; MariaDB purged — flat-file, no DB) |
| vaultwarden | edge2 (CT 102) | 10.10.10.20 | 100.64.0.33 | Vaultwarden password manager (migrated from Contabo 2026-06-16) |
| forgejo | edge2 (CT 103) | 10.10.10.21 | 100.64.0.34 | Forgejo git server (migrated from Contabo 2026-06-16) |
| livesync | edge2 (CT 104) | 10.10.10.22 | 100.64.0.35 | LiveSync Obsidian sync (CouchDB + provisioner; migrated from Contabo 2026-06-16) |
| [[authentik]] | edge2 (CT 105) | 10.10.10.23 | 100.64.0.36 | [[authentik]] SSO platform (migrated from Contabo 2026-06-18) |
| matrix | edge2 (CT 106) | 10.10.10.24 | 100.64.0.37 | Matrix stack ([[synapse]] + MAS + Element + [[mautrix_signal]]; migrated from Contabo 2026-06-18) |
| headscale | edge2 (CT 107) | 10.10.10.25 | 100.64.0.38 | Headscale + Headplane tailnet control plane (migrated from Contabo [[2026-06-19]]) |

> **Note ([[2026-06-19]]):** edge2 CT placements CT 102–107 confirmed; Forge git-SSH DNAT (`forgejo-ssh-dnat.service`) is a permanent systemd unit on edge2 host.

## IP Allocation Scheme

| Range | Purpose |
|-------|---------|
| .1-.10 | Network infrastructure |
| .11-.99 | DHCP clients |
| .100-.149 | LXC containers |
| .150-.199 | VMs |
| .240-.250 | Proxmox hosts + bare metal |
| .251-.254 | Meshtastic nodes |

Full details: [[ip-allocation]]

## Headscale Node List

Current registered nodes (25 total):

| Node | Tailscale IP | Type |
|------|-------------|------|
| contabo | 100.64.0.40 | VPS (edge1, mail-only, rebuilt [[2026-06-19]]; formerly 100.64.0.1) |
| immich | 100.64.0.2 | LXC |
| media | 100.64.0.3 | Proxmox |
| cloud | 100.64.0.4 | Proxmox |
| utility | 100.64.0.5 | Proxmox |
| data | 100.64.0.6 | Proxmox |
| meshmonitor | 100.64.0.7 | LXC |
| [[caddy]] | 100.64.0.8 | LXC |
| aida-nebra | 100.64.0.9 | Pi |
| matt-desktop | 100.64.0.10 | Desktop |
| nextcloud | 100.64.0.11 | LXC |
| toc | 100.64.0.13 | Proxmox |
| cortex | 100.64.0.14 | VM |
| [[searxng]] | 100.64.0.15 | LXC |
| ha | 100.64.0.16 | VM (cloud VM 151 — node id 54, user echo6; home automation) |
| arr | 100.64.0.18 | VM |
| pi-nas | 100.64.0.21 | Pi |
| mesh-bridge | 100.64.0.22 | LXC |
| peertube-4hve9pdr | 100.64.0.17 | LXC (collision-suffixed identity since ~2026-06-22; formerly `peertube` at 100.64.0.23, now stale/gone) |
| [[recon]] | 100.64.0.24 | VM |
| [[argus]] | 100.64.0.25 | LXC |
| [[central]] | 100.64.0.12 | LXC (utility CT 104 — central.echo6.mesh) |
| edge2 | 100.64.0.26 | Proxmox/Contabo VPS |
| gl-a1300 | 100.64.0.29 | Router |
| bluefin | 100.64.0.30 | Desktop |
| wordpress | 100.64.0.31 | LXC (edge2 CT 101 — now runs Grav CMS; hostname unchanged) |
| [[meshai]] | 100.64.0.32 | LXC |
| vaultwarden | 100.64.0.33 | LXC (edge2 CT 102) |
| forgejo | 100.64.0.34 | LXC (edge2 CT 103) — node id 46 |
| livesync | 100.64.0.35 | LXC (edge2 CT 104) — migrated 2026-06-16 |
| [[authentik]] | 100.64.0.36 | LXC (edge2 CT 105) — node id 48, migrated 2026-06-18 |
| matrix | 100.64.0.37 | LXC (edge2 CT 106) — migrated 2026-06-18 |
| headscale | 100.64.0.38 | LXC (edge2 CT 107) — migrated 2026-06-19 |

## IdahoMesh Headscale Node List

Separate Headscale instance on CT 106 (192.168.1.106), prefix 100.100.0.0/16.
Reachable from echo6 tailnet via mesh-bridge (CT 107).

| Node | Tailscale IP | User | Type |
|------|-------------|------|------|
| mesh-bridge | 100.100.0.3 | malice | LXC (bridge) |
| burley-butte | 100.100.0.1 | nebra | Pi (offline) |
| mt-isr | 100.100.0.5 | nebra | Pi Zero 2 W |

## SSH Access

**Standard user:** `zvx`
**Credentials:** Source from `/home/zvx/projects/.ref/credentials`

Most servers use SSH key auth. Exceptions noted below.

```bash
# SSH to any server (key auth)
ssh zvx@<ip-address>

# Examples
ssh zvx@192.168.1.244  # TOC (Proxmox host)
ssh zvx@192.168.1.150  # cortex VM
ssh zvx@192.168.1.241  # utility Proxmox
ssh -i ~/.ssh/contabo2_ed25519 root@5.189.158.149  # edge1 (mail-only; formerly Contabo)
ssh zvx@cortex         # cortex via Tailscale hostname
ssh edge2              # edge2 via SSH config alias (admin@100.64.0.26)
```

### edge2 SSH access

edge2 is hardened: `PasswordAuthentication no`, root login disabled. SSH user is `admin` with passwordless sudo. The SSH config alias `edge2` maps to `admin@100.64.0.26`.

Authorized keys on edge2 (`/home/admin/.ssh/authorized_keys`):
- `echo6-contabo2-184.174.35.153` — the original provisioning key (cortex `~/.ssh/contabo2_ed25519`)
- `cortex` — cortex default `id_ed25519`
- `matt-desktop-wsl2` — WSL2 default key
- `cookie-sync` — Windows native SSH key

To access CTs on edge2: `ssh edge2` then `sudo pct exec <vmid> -- bash`.

### Password-auth hosts

These require password authentication (no SSH keys installed):

| Host | User | Password | Access |
|------|------|----------|--------|
| aida-nebra | zvx | 7redditGold | `sshpass -p '7redditGold' ssh zvx@aida-nebra` |
| mt-isr | isr | UfPsfwyMIUIKb1 | `sshpass -p 'UfPsfwyMIUIKb1' ssh isr@192.168.1.141` |
| mt-burleybutte | bb | (see credentials) | `sshpass -p '<pw>' ssh bb@192.168.1.185` |
| matt-desktop | administrator | Qw1290opzx | `ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no administrator@192.168.1.254` |
| toc | root | 7redditGold | `sshpass -p '7redditGold' ssh -o PubkeyAuthentication=no root@100.64.0.13` |

Use the Tailscale hostname (`aida-nebra`) or local IP (`192.168.1.253`) — both work for aida-nebra.
mt-isr is on IdahoMesh tailnet (100.100.0.5) — reachable from echo6 via bridge.
matt-desktop is accessible via local IP (192.168.1.254) or Tailscale (100.64.0.10) — requires explicit password auth flags.

## Key External IPs

| Purpose | IP |
|---------|-----|
| Home external (public [[services]]) | 199.6.36.163 |
| edge1 (mail-only, rebuilt Contabo VPS) | 5.189.158.149 |
| edge2 (front door: vault/forge/notes/auth/matrix/element/vpn/proxmox + idahomesh.com) | 184.174.35.153 |