echo6-docs/projects/advbbs-project.md

# advBBS — Claude Code Project Context

## Source of Truth

**GitHub repo**: https://github.com/zvx-echo6/advbbs (always pull latest before working)

## What is advBBS?

A federated, encryption-first BBS for Meshtastic mesh radio networks. Users interact by sending text DMs to a Meshtastic node running the BBS. Multi-hop mail routing between BBS nodes over LoRa radio. Runs on Raspberry Pi Zero 2 W (~100MB RAM).

Built with Python 3.11, SQLite (WAL mode), Meshtastic Python API. Docker-deployed. This is a "vibe-coded" project built with AI assistance — functional but may have rough edges.

---

## Package Structure

```
advbbs/
├── __init__.py
├── __main__.py              # Entry point
├── config.py                # TOML config loading, dataclasses
├── cli/
│   ├── config_rich.py       # Rich-based interactive config TUI
├── commands/
│   ├── dispatcher.py        # Command parser + all !commands
├── core/
│   ├── bbs.py               # Main BBS class, event loop, session mgmt
│   ├── boards.py            # Board service (CRUD, access control)
│   ├── crypto.py            # Argon2id + ChaCha20-Poly1305 encryption
│   ├── mail.py              # Mail service (inbox, send, read, delete)
│   ├── maintenance.py       # Scheduled cleanup tasks
│   ├── rate_limiter.py      # Per-node rate limiting
├── db/
│   ├── connection.py        # SQLite connection, schema, migrations
│   ├── models.py            # Dataclasses (User, Message, Board, etc.)
│   ├── messages.py          # MessageRepository (CRUD)
│   ├── users.py             # UserRepository, NodeRepository, UserNodeRepository
├── mesh/
│   ├── interface.py         # Meshtastic radio interface, send/receive DMs
├── sync/
│   ├── manager.py           # SyncManager — federation orchestrator (RAP, mail routing, retry logic)
│   ├── compat/
│   │   ├── advbbs_native.py # Wire protocol handler (HELLO, SYNC_ACK, bulletin format)
├── utils/
│   ├── formatting.py        # Text formatting helpers
│   ├── pagination.py        # Message pagination for mesh constraints
tests/
├── test_boards.py
├── test_crypto.py
├── test_mail.py
├── test_maintenance.py
├── test_pagination.py
├── test_sync.py
docs/
├── commands.md
├── mail.md
├── boards.md
├── sync.md                  # Federation + RAP protocol docs
├── configuration.md
├── deployment.md
├── security.md
├── rap-testing.md           # Multi-hop RAP test procedures
├── migration.md             # fq51bbs → advBBS migration
├── quickstart.md
├── USER-QUICKSTART.md
├── ELI5.md
```

### Non-obvious file placements

- `crypto.py` and `rate_limiter.py` → `core/` (not root)
- `interface.py` (Meshtastic mesh interface) → `mesh/` (not root)
- `advbbs_native.py` (wire protocol) → `sync/compat/` (not root)
- `dispatcher.py` (all user commands) → `commands/` (not root)
- `config_rich.py` (TUI config) → `cli/` (not root)
- `formatting.py`, `pagination.py` → `utils/`

---

## Database

SQLite with WAL mode, autocommit, `check_same_thread=False`. Row factory enabled for dict-like access.

### Schema (3 migrations)

**Migration 001 — Core tables:**
- `users` — id, username, password_hash, salt, encryption_key, recovery_key_enc, is_admin, is_banned, ban fields
- `nodes` — Meshtastic nodes (node_id like `!abcdef12`, short_name, long_name, SNR/RSSI)
- `user_nodes` — Multi-node identity (user_id ↔ node_id, is_primary)
- `messages` — uuid (UNIQUE), msg_type (`mail`/`bulletin`/`system`), board_id, sender/recipient user/node IDs, subject_enc, body_enc (BLOB NOT NULL), timestamps, origin_bbs, forwarded_to, hop_count, delivery_attempts
- `boards` — name, description, board_type, board_key_enc
- `board_access` — Per-user restricted board access
- `board_states` — Per-user read position
- `bbs_peers` — node_id, bbs_name, protocol, sync_enabled, trust_level
- `sync_log` — message_uuid, peer_id, direction, status, attempts

**Migration 002 — Settings/maintenance:**
- Added columns: `messages.deleted_at_us`, `bbs_peers.callsign/name/capabilities/last_seen_us`
- New tables: `bbs_settings` (KV store), `board_read_positions`

**Migration 003 — RAP:**
- Added peer columns: `health_status`, `failed_heartbeats`, `last_heartbeat_us`, `last_pong_us`, `quality_score`
- New tables: `rap_routes` (dest_bbs, via_peer_id, hop_count, quality_score, expires_at_us), `rap_pending_mail` (queued mail for offline routes)

### Timestamps

All timestamps are microseconds since epoch (`int(time.time() * 1_000_000)`), stored as INTEGER. Column suffix `_us`.

---

## Wire Protocol

All inter-BBS messages sent as Meshtastic DMs. Format: `advBBS|1|<MSG_TYPE>|<payload>`

### RAP Messages (Route Announcement Protocol)

| Message | Purpose | Payload |
|---------|---------|---------|
| `RAP_PING` | Heartbeat | `timestamp_us` |
| `RAP_PONG` | Response + routes | `timestamp_us\|route_table` |
| `RAP_ROUTES` | Route table broadcast | `route_table` |

Route table format: `BBS1:hop:quality;BBS2:hop:quality` (e.g., `MV51:0:1.0;J51B:1:1.00`)

### Mail Protocol Messages

| Message | Format | Purpose |
|---------|--------|---------|
| `MAILREQ` | `MAILREQ\|uuid\|from_user\|from_bbs\|to_user\|to_bbs\|hop\|num_parts\|route` | Request delivery |
| `MAILACK` | `MAILACK\|uuid\|OK` | Accept, ready for chunks |
| `MAILNAK` | `MAILNAK\|uuid\|reason` | Reject (NOUSER, NOROUTE, MAXHOPS, LOOP) |
| `MAILDAT` | `MAILDAT\|uuid\|part/total\|data` | Message chunk (max 150 chars × 3) |
| `MAILDLV` | `MAILDLV\|uuid\|OK\|user@BBS` | Delivery confirmation |

### Mail Flow

```
Sender BBS              Destination BBS
    │                        │
    │── MAILREQ ────────────▶│  (pre-flight: user exists?)
    │◀── MAILACK ────────────│  (ready for chunks)
    │── MAILDAT 1/1 ────────▶│  (body chunk)
    │                        │  (store in DB)
    │◀── MAILDLV ────────────│  (confirmed)
```

Multi-hop: intermediate BBS relays MAILREQ/MAILDAT, tracked via `_relay_mail` dict. Max 5 hops. Route list in MAILREQ prevents loops.

---

## Key Architecture Patterns

### Threading Model

- **Main thread**: asyncio event loop (`bbs._loop`) — runs tick(), scheduled tasks
- **Meshtastic callback thread**: `on_receive` fires from Meshtastic library thread
- **Bridge**: `_schedule_async(coro)` uses `asyncio.run_coroutine_threadsafe()` to schedule work from callback thread onto main loop

### Session Management

Sessions keyed by Meshtastic node_id. Login requires both password AND a registered node (node-based 2FA). Sessions expire after inactivity.

### Encryption

- **At rest**: All message bodies encrypted with user-derived keys (Argon2id KDF → ChaCha20-Poly1305)
- **Transport**: Meshtastic PSK encryption (AES-256) recommended
- **Remote mail**: Stored plaintext on receiving BBS (encrypted at read time by recipient's key)

### Message Constraints

- LoRa max ~150 bytes usable per packet
- Remote mail body max 450 chars (3 chunks × 150)
- Pagination helper chunks long responses for mesh delivery
- TX queue collision avoidance: 2.5s delay between protocol DMs

### Peer Security

Federation traffic whitelisted by peer — only configured peers accepted. Non-peer protocol messages rejected.

---

## Configuration

TOML config file. Key sections: `[bbs]`, `[database]`, `[meshtastic]`, `[crypto]`, `[features]`, `[operating_mode]`, `[sync]`, `[rate_limits]`, `[web_reader]`, `[cli_config]`, `[logging]`.

Operating modes: `full`, `mail_only`, `boards_only`, `repeater`.

Peers configured as `[[sync.peers]]` arrays with `node_id`, `name`, `protocol`, `enabled`.

RAP timing defaults are conservative for mesh (12h heartbeat, 36h route expiry, 24h route share).

---

## Current Live Federation Topology

```
MV51 (Old Man Malice / Matt)  ◀──▶  J51B (JeepnJonny)
   node: !00ff0001                    node: !60a43e58
```

Both running Docker containers. Meshtastic simulator (meshtasticd) for testing.

---

## Known Bug: Federation Mail Delivery Failure

### Symptom
```
[ERROR] advbbs.sync.manager: DELIVER b8e78195: Failed to store in database
```

### Root Cause
`create_incoming_remote_mail()` in `db/messages.py` returns `None` for both duplicates (logged at DEBUG — invisible) and real DB errors. Mesh radio retransmissions deliver the same MAILDAT twice, triggering duplicate detection, but the caller can't distinguish this from a real failure.

Additionally, when a duplicate IS detected, no MAILDLV confirmation is sent back, causing the sender to retry indefinitely.

### Fix Required (3 changes)

1. **`db/messages.py`** — `create_incoming_remote_mail`: Return `"duplicate"` sentinel instead of `None` for duplicate UUID. Promote log from DEBUG → INFO. Add traceback to exception path.

2. **`sync/manager.py`** — `_deliver_remote_mail`: Handle `"duplicate"` return: log at INFO, still send MAILDLV confirmation, clean up state. Improve error message for real failures.

3. **`sync/manager.py`** — `handle_maildat` (the `_handle_maildat` section around line 1068): Add `delivering` flag guard to prevent double scheduling from mesh retransmissions.

---

## Development Notes

- Tests: `pytest tests/` — unit tests for crypto, mail, boards, maintenance, pagination, sync
- Docker build: `docker compose build` (can take 10-15 min on Pi, may need swap)
- Config TUI: `advbbs-config` or `python -m advbbs.cli.config_rich`
- Logs: `docker compose logs -f`
- DB inspection: `sqlite3 /data/advbbs.db ".tables"` inside container

---

## Style / Conventions

- Logging: `logger = logging.getLogger(__name__)` per module
- DB access: Repository pattern (MessageRepository, UserRepository, etc.) wrapping Database methods
- All DB timestamps: microseconds (`_us` suffix)
- UUIDs: `str(uuid.uuid4())` for message dedup
- Meshtastic node IDs: hex string with `!` prefix (e.g., `!00ff0001`)
- Commands: `!` prefix, case-insensitive, short aliases
- Config: TOML with dataclass parsing in `config.py`