--- title: pymc-repeater KISS TNC Re-enumeration Fix type: runbook tags: - mesh related: - [[meshtastic-sidecar-node]] updated: 2026-07-01 --- # pymc-repeater KISS TNC Re-enumeration Fix `pymc-repeater.service` on **aida-nebra** (Tailscale 100.64.0.9) runs the Meshcore LoRa repeater node `EchoBase` on 910.525 MHz SF7. It talks to a RAK4631 USB KISS TNC (VID 239a / PID 8029) via the `/dev/pymc-radio` udev symlink. The Nebra SX1262 hat on the same machine runs a separate `meshtasticd` instance on its own radio — see [[meshtastic-sidecar-node]] for that hardware's setup. Do not touch `meshtasticd` or the SX1262 hat during any pymc-repeater work. --- ## Background — Why This Breaks `KissModemWrapper` holds the serial file descriptor open but has no reopen logic. When the RAK4631 re-enumerates (power cycle, USB glitch, kernel rescan), the kernel assigns a new node: observed `ttyACM0` → `ttyACM1` around 2026-06-23. The service keeps a stale fd pointing at `/dev/ttyACMx (deleted)`. Systemd still reports `active (running)` — the service is silently deaf and mute. --- ## Symptom Every ~30 seconds in the journal: ``` KissModemWrapper - ERROR - Serial write error: ... [Errno 5] Input/output error TX frame write failed, dropping frame ``` Zero RF traffic in or out despite a green `systemctl status`. --- ## Diagnose ### 1. Check the journal for the I/O error spam ```bash ssh zvx@100.64.0.9 journalctl -u pymc-repeater.service --since '5 min ago' --no-pager ``` Look for `[Errno 5] Input/output error` or `TX frame write failed` — either confirms stale fd. ### 2. Inspect the process's open file descriptors ```bash sudo ls -l /proc/$(systemctl show -p MainPID --value pymc-repeater.service)/fd/ | grep ttyACM ``` - **Stale (broken):** symlink target contains `(deleted)` — smoking gun. - **Healthy:** target points at a real `/dev/ttyACMx`. ### 3. Confirm where the device landed after re-enumeration ```bash ls -l /dev/pymc-radio ``` Shows which `ttyACM*` the live RAK4631 is on now. --- ## Manual Recovery A restart clears the stale fd and re-opens the device via the udev symlink: ```bash sudo systemctl restart pymc-repeater.service ``` ### Verify recovery ```bash # fd should now point at a real ttyACMx (not deleted) sudo ls -l /proc/$(systemctl show -p MainPID --value pymc-repeater.service)/fd/ | grep ttyACM # Journal should show reconnect and normal TX journalctl -u pymc-repeater.service -n 20 --no-pager ``` Expected healthy journal lines: ``` KISS modem connected to /dev/pymc-radio at 115200 baud Retransmitted packet ... ``` The `[Errno 5]` spam must be absent. --- ## Permanent Fix (applied 2026-07-01) Two changes make pymc-repeater self-healing across any future re-enumeration. Original files were backed up as `.bak` alongside each. ### 1. udev rule — `/etc/udev/rules.d/99-pymc-radio.rules` Added `TAG+="systemd"` and `ENV{SYSTEMD_WANTS}="pymc-repeater.service"` to the existing VID 239a / PID 8029 rule: ```udev SUBSYSTEM=="tty", ATTRS{idVendor}=="239a", ATTRS{idProduct}=="8029", \ SYMLINK+="pymc-radio", \ TAG+="systemd", \ ENV{SYSTEMD_WANTS}="pymc-repeater.service" ``` On any future re-enumeration, udev creates the `dev-pymc\x2dradio.device` unit for the new node and systemd activates `pymc-repeater.service` against it automatically. ### 2. systemd unit — `/etc/systemd/system/pymc-repeater.service` `[Unit]` additions: ```ini BindsTo=dev-pymc\x2dradio.device After=dev-pymc\x2dradio.device StartLimitIntervalSec=300 StartLimitBurst=10 ``` `[Service]` additions: ```ini Restart=always RestartSec=5 ``` `BindsTo` stops the service cleanly when the TNC drops (the device unit disappears) and restarts it when the device returns. `Restart=always` with `RestartSec=5` ensures fast recovery from any modem crash without the device unit cycling. --- ## Verify the Resilience Wiring Is Intact Run these on aida-nebra after any future maintenance to confirm the permanent fix is still in place: ```bash # BindsTo and After wired to the device unit systemctl show pymc-repeater.service -p BindsTo -p After | grep -i pymc # udev tagging correct udevadm info -q property -n /dev/pymc-radio | grep -i SYSTEMD # device unit tracked by systemd systemctl list-units --type=device --all | grep -i pymc ``` Expected output: ``` BindsTo=dev-pymc\x2dradio.device After=... dev-pymc\x2dradio.device ... SYSTEMD_WANTS=pymc-repeater.service dev-pymc\x2dradio.device loaded active plugged ... ``` --- ## Known Secondary Issue (non-blocking) RRD graphing fails with `rrdtool not available`. Metrics/graphing only — no impact on RF operation. --- ## Quick Reference | Item | Value | |------|-------| | Host | aida-nebra (100.64.0.9) | | Service | `pymc-repeater.service` | | Node / freq | EchoBase, 910.525 MHz SF7 | | TNC hardware | RAK4631 USB KISS TNC (VID 239a / PID 8029) | | udev symlink | `/dev/pymc-radio` | | udev rule | `/etc/udev/rules.d/99-pymc-radio.rules` | | Unit file | `/etc/systemd/system/pymc-repeater.service` | | Unrelated radio | Nebra SX1262 hat → `meshtasticd` (do not touch) | ```bash # Restart (manual recovery) sudo systemctl restart pymc-repeater.service # Tail journal journalctl -u pymc-repeater.service -f # Check fd state sudo ls -l /proc/$(systemctl show -p MainPID --value pymc-repeater.service)/fd/ | grep ttyACM ```