MeshCore reconnect persistence: implement 0=unlimited max-reconnect-attempts sentinel

connect() now translates a configured meshcore_max_reconnect_attempts of 0
(or <=0) into an effectively-unbounded count before handing it to the
meshcore library's ConnectionManager, so its retry loop never
permanently exhausts. config.py's comment already documented \"0 =
unlimited\" but that sentinel was never actually implemented -- literal 0
meant zero attempts, and the shipped default of 5 (at ~1s/attempt) gave up
after ~5s with no external supervisor to retry again, leaving MeshCore dead
until a manual container restart. Also flips the repo default from 5 to 0
so fresh deploys get unlimited retries without extra config.

Proven via a 60s forced-outage auto-recovery test: the link recovers from
any-length vnode/radio outage instead of giving up after ~5s.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Matt Johnson 2026-08-02 04:10:20 +00:00
commit bdbc2afe7f
2 changed files with 34 additions and 1 deletions

View file

@ -51,7 +51,7 @@ class ConnectionConfig:
meshcore_host: str = "" # pyMC companion frame server host meshcore_host: str = "" # pyMC companion frame server host
meshcore_port: int = 5050 # pyMC companion frame server port meshcore_port: int = 5050 # pyMC companion frame server port
meshcore_auto_reconnect: bool = True # enable meshcore lib auto-reconnect meshcore_auto_reconnect: bool = True # enable meshcore lib auto-reconnect
meshcore_max_reconnect_attempts: int = 5 # max reconnect attempts (0 = unlimited) meshcore_max_reconnect_attempts: int = 0 # max reconnect attempts (0 = unlimited)
meshcore_advert_interval_seconds: int = 86400 # periodic self-advert interval, 24h (0 = disabled) meshcore_advert_interval_seconds: int = 86400 # periodic self-advert interval, 24h (0 = disabled)
# MeshCore connection type: tcp | serial | ble (default tcp for back-compat) # MeshCore connection type: tcp | serial | ble (default tcp for back-compat)
meshcore_conn_type: str = "tcp" meshcore_conn_type: str = "tcp"

View file

@ -43,6 +43,29 @@ _TELEMETRY_MAX_FAILURES = 3
# inside the 300s reaper window with margin to spare. # inside the 300s reaper window with margin to spare.
_KEEPALIVE_INTERVAL_SECONDS = 120 _KEEPALIVE_INTERVAL_SECONDS = 120
# --- Reconnect persistence ("0 = unlimited" sentinel) ----------------------
# config.py documents meshcore_max_reconnect_attempts as "0 = unlimited", but
# that sentinel was never implemented here — the value was passed straight
# through to the meshcore lib's ConnectionManager, whose retry loop is
# `while self._reconnect_attempts < self.max_reconnect_attempts`. Taken
# literally, 0 means ZERO attempts (immediate give-up), the opposite of
# "unlimited", and any small bounded value (the shipped default is 5, at the
# lib's flat 1s-per-attempt cadence) exhausts after ~5 seconds and then the
# link stays down PERMANENTLY — there is no external supervisor for MeshCore
# (see main.py's watchdog guard: "MeshCoreTransport manages its own
# reconnect via the meshcore lib's auto_reconnect parameter"), so nothing
# ever notices and retries again after that. A radio/vnode bounce longer
# than ~5s (e.g. the 2026-08-02 device-perm heal test) killed MeshCore for
# good until a manual container restart.
#
# Fix: honor the documented sentinel for real. connect() below translates a
# configured 0 into this effectively-unbounded count, so the lib's own
# proven-safe retry loop (still local TCP only, still ~1 attempt/sec, still
# WITHOUT re-sending the connect-time self-advert — see
# _post_reconnect_setup_async) just keeps going until the vnode/radio comes
# back, no matter how long the outage lasts.
_MC_RECONNECT_ATTEMPTS_UNLIMITED = 2_147_483_647
# Numeric Cayenne-LPP type id → decoded field name. Ids not in this map are # Numeric Cayenne-LPP type id → decoded field name. Ids not in this map are
# passed through as ``lpp_<id>`` so nothing is silently dropped. # passed through as ``lpp_<id>`` so nothing is silently dropped.
_LPP_ID_TO_FIELD = { _LPP_ID_TO_FIELD = {
@ -1976,6 +1999,16 @@ class MeshCoreTransport(MeshTransport):
ble_address = getattr(self.config, "meshcore_ble_address", "") ble_address = getattr(self.config, "meshcore_ble_address", "")
auto_reconnect = getattr(self.config, "meshcore_auto_reconnect", True) auto_reconnect = getattr(self.config, "meshcore_auto_reconnect", True)
max_attempts = getattr(self.config, "meshcore_max_reconnect_attempts", 5) max_attempts = getattr(self.config, "meshcore_max_reconnect_attempts", 5)
if max_attempts <= 0:
# Documented sentinel (config.py: "0 = unlimited") — see
# _MC_RECONNECT_ATTEMPTS_UNLIMITED's docstring for why this was
# never actually unlimited before and why translating it here is
# the fix.
logger.info(
"MeshCoreTransport: meshcore_max_reconnect_attempts=%s (unlimited) -> %d",
max_attempts, _MC_RECONNECT_ATTEMPTS_UNLIMITED,
)
max_attempts = _MC_RECONNECT_ATTEMPTS_UNLIMITED
# Human-readable target for logging — from the same descriptor that # Human-readable target for logging — from the same descriptor that
# self_info() reports, so the log and the API never disagree. # self_info() reports, so the log and the API never disagree.