echo6-docs/vault/archive/projects/meshai-native-fire-severity-audit-cc-handoff.md

---
title: "MeshAI Native Fire-Severity Audit — CC Handoff (archived)"
type: project
tags:
  - mesh
status: archived
updated: 2026-07-11
---
> **ARCHIVED / SUPERSEDED — fix deployed.** This was a one-time Claude Code handoff for the native fire-severity regression (audit 2026-07-08). The fix has been deployed; this runbook is retained for history only. Its host/access details are STALE — meshai runs as utility CT 108 (`pct exec 108` from root@utility; zvx has no sudo), NOT at 192.168.1.144 with sudo. Do not execute.

# MeshAI Audit Handoff — Native Fire Severity Regression

**For:** Claude Code (CC), operating from cortex against the MeshAI LXC (192.168.1.144)
**Repo:** zvx-echo6/meshai (deployed via docker compose on the LXC)
**Date of audit:** 2026-07-08, against HEAD `aef9877` (#95)
**Scope:** One minimal backend fix in `work/meshai/env/store.py`, plus a mandatory DB pre-seed. Nothing else. No drive-by refactors.

---

## 0. Standing operational rules (non-negotiable)

1. **SSH prereq gate.** Before ANY work: verify SSH from cortex to the MeshAI LXC (192.168.1.144) using the configured method in `~/.ssh/config`. If auth fails, STOP and report. Never assume SSH works.
2. **Never use `docker compose run`.** Ever. Use `docker exec` against the running container, or host tooling.
3. **Verbatim diffs only.** Apply the diff in §10 exactly. If the anchor lines don't match, STOP and report — do not improvise.
4. **Plain-English explanation before every commit.** Matt's understanding is the gate. Present the summary, wait for his confirmation, then commit.
5. **Every status report back to Matt ends with `sudo docker ps`.**
6. **Rebuild/deploy is ONLY:** `sudo docker compose build meshai && sudo docker compose up -d`.
7. **Mesh-spam stop gate** — see §1. Binding.

---

## 1. ⚠️ MESH-SPAM STOP GATE — BINDING

Deploying the fix in this document **will release a burst of fire broadcasts** unless the pre-seed in §9 is completed and verified first.

Mechanics: every fire currently blocked by this bug retries its "New" broadcast **on every poll** and dies silently at the dispatcher severity floor. Because delivery never succeeds, the decider's commit never runs, so its `fires` row keeps `last_broadcast_at = NULL`. The first poll after the fixed code starts will pass **every one of those fires at once** — and there is no pacer on the native path (`FirePacer` is wired to the Central consumer only, `main.py:128–136`; its own docstring says it serves the Central reconnect/drain path).

**Rule:** Phase −1 (§7) and Phase 0 (§8) are read-only and may run immediately. **Do not execute anything in §9 onward until Matt replies verbatim: `I know and I accept`.** When you reach that point, report the burst size from Phase 0 step D and ask for the ack explicitly. Even with the pre-seed done, note to Matt that legitimate post-fix fire traffic will begin (that's the point), and multiple genuinely-new fires arriving in one poll batch will emit together.

---

## 2. Symptom

Since the central→native swap (Jul 4–6), the mesh carries only WZDx, NWS weather alerts, satpass, and band conditions. Fire alerts — which should be active in southern Idaho in July — are silent. (Avalanche silence is seasonal and expected; it is NOT part of this problem.)

## 3. Root cause — plain English

The central→native swap dropped the decider's severity promotion, and native fire events now die at the dispatcher's `min_severity` floor — silently, with no log line and no drop counter.

Evidence chain (all verified by reading the code at HEAD):

1. **Central path (old behavior):** `central/wfigs_handler.py:238, 348` stamped `data["_severity_override"] = "priority"` on every fire New/Update. `central/consumer.py:649–654` promoted it onto the Event: `severity=sev_override or map_severity(...)`. Every fire reached the dispatcher as **priority**.
2. **Native adapter severity:** `env/fires.py` (~lines 139–148) assigns severity by anchor proximity — `"priority"` only within 25 km of a region anchor, **`"routine"` otherwise**. `to_event` (line 333+) passes it through unchanged.
3. **The decider still stamps the override:** `notifications/gating/fire.py::_broadcast_patch` sets `"_severity_override": "priority"` on every New and Update GateResult. The intent is unchanged from Central.
4. **The promotion is lost:** `env/store.py::_emit_event` (line 678+) builds the Event from `adapter.to_event()` FIRST, then merges the decider patch into **`event.data` only** (`event.data.update(gate.data_patch)`, ~line 731). Nothing anywhere in the native delivery path promotes `_severity_override` onto `event.severity` — verified by grep across dispatcher, composer, and events modules: zero consumers.
5. **The floor:** `notifications/pipeline/dispatcher.py` Section 2 (~line 558): `if event_rank < SEVERITY_RANK[tog.min_severity]: return` — a bare return. **No log, no counter** (the four persisted counters cover cold-start/stale/cooldown/dedup, not the severity floor). The matrix path (Section 1.5) has an equivalent per-cell floor with an "authoritative no-send" return.
6. **The default floor is "priority":** `config.py:641` (`NotificationToggle.min_severity: str = "priority"`) and again at the factory default, `config.py:681`.

Net: routine-severity fire → silent drop, every poll, forever. Fire is forced through the decider regardless of cutover state (`NATIVE_ALWAYS_DECIDE`, `notifications/cutover.py:44`), so every native fire New/Update is affected.

**Timeline:** WFIGS→decider migration `8bc9b14` (Jul 4, 22:51), received-delta gate `0c76a80` (Jul 5), native WFIGS routed through decider `8d61b16` (Jul 6, 15:35). Fire silence began at the native flip — matches the operator's report exactly.

## 4. Why the surviving feeds survive

They handle severity in-adapter, so they never depended on the lost promotion:
- `env/satpass.py:250` — reads `severity=data.get("_severity_override", "routine")` itself. **This is the correct pattern the fix replicates at the store level.**
- `env/nws.py:39` — derives severity in-adapter via `_map_nws_severity` from CAP severity; warnings clear the floor.
- `env/wzdx.py:365` — full closures are `"priority"`.
- Band conditions — scheduled broadcaster (`BandConditionsScheduler`) that calls the dispatcher directly.

Of the six deciders that emit `_severity_override` (fire, nws, quake, swpc, avalanche, firms), only **firms** (`env/firms.py:456`) and **satpass** consume it in-adapter.

## 5. Ruled out during the audit

- Avalanche: out of season; expected silence. Not a bug.
- Received-delta gate (`_delta_emit`, commit `0c76a80`): fires deliberately bypass it (`store.py::_ingest`, `nifc` branch) — the decider is the fire gate. Not the drop point.
- FirePacer black hole: pacer is Central-consumer-only; native fires go straight to the bus. Not a factor.
- Toggle mapping: `wildfire_declared`/`wildfire_incident` both map to the `fire` toggle (`notifications/categories.py:~380–390`). Not a factor.
- Coverage filter: fail-open for non-weather categories; fires carry centroids. Not the primary drop.
- ToggleFilter/inhibitor/grouper: all log their drops; fire family is presumed enabled (verify in Phase 0).

## 6. Secondary findings — explicitly OUT OF SCOPE for this change (backlog)

1. **Restart-window ignition swallowing.** `_fires_seeded` is per-process (`store.py::_ingest_fires`); every restart's first poll silent-seeds any fire that first appeared while the container was down — it is never announced as New. Deliberate anti-backlog design, but during dense deploy periods in fire season it eats real ignitions. Needs a design decision (e.g., persisted seen-baseline), not a hotfix.
2. **Zero-acre New fires deferred.** `env/fires.py::to_event` returns None when `not acres`; fresh WFIGS ignitions often report 0 acres initially. Self-heals when acreage posts (row exists with NULL `last_broadcast_at` → decider fires "New"), but delays first announcements.
3. **Same bug class armed for quake/swpc/avalanche/nws.** Their deciders emit `_severity_override` with no native consumer; currently masked behind `MESHAI_CUTOVER_CATEGORIES` shadow gating. The §10 fix resolves this at the shared choke point for all cut-over categories too.
4. **No native fire pacer.** Multiple legit new fires in one poll batch will emit back-to-back. Consider extending FirePacer to the native emit path later.

---

## 7. Phase −1 — SSH prereq gate (mandatory first step)

From cortex:

```bash
ssh <configured-alias-or-user>@192.168.1.144 'echo SSH_OK && hostname'
```

If this fails for any reason: STOP. Report to Matt. Do not proceed.

Then locate the deployment on the LXC (do not guess paths):

```bash
sudo docker ps --format '{{.Names}}\t{{.Image}}' | grep -i meshai
find /opt /root /home -maxdepth 3 -name docker-compose.yml 2>/dev/null | xargs grep -l meshai 2>/dev/null
```

Record the compose project directory and the container name (assumed `meshai` below — substitute if different). Confirm `git status` in the repo checkout is clean before any edit; if dirty, STOP and report.

## 8. Phase 0 — read-only diagnostics (safe to run now)

Run all of these and report results to Matt before anything else. Determine the DB path first:

```bash
sudo docker exec meshai ls /app/data
```

**A. Are fire events reaching the bus and dying downstream?** (Diagnosis predicts: repeatedly nonzero.)

```bash
sudo docker logs meshai --since 24h 2>&1 | grep -c "Emitted nifc"
sudo docker logs meshai --since 24h 2>&1 | grep "Emitted nifc" | tail -5
```

**B. Live fire family config — the floor:**

```bash
sudo docker exec meshai grep -B2 -A8 -i "fire" /app/config/config.yaml | grep -i -B1 -A4 "min_severity\|enabled\|region"
```

**C. Cutover state (determines blast radius of the bug class):**

```bash
sudo docker exec meshai printenv MESHAI_CUTOVER_CATEGORIES
```

**D. Burst size if fixed today** — this number goes in the stop-gate report to Matt:

```bash
sudo docker exec meshai sqlite3 /app/data/<DBFILE> \
  "SELECT COUNT(*) FROM fires WHERE last_broadcast_at IS NULL;"
sudo docker exec meshai sqlite3 /app/data/<DBFILE> \
  "SELECT irwin_id, incident_name, current_acres, current_contained_pct, last_broadcast_at
     FROM fires ORDER BY last_event_at DESC LIMIT 20;"
```

**E. Dispatcher drop counters** (severity-floor drops are NOT counted — expected to look quiet; that's part of the finding):

```bash
sudo docker exec meshai sqlite3 /app/data/<DBFILE> "SELECT * FROM dispatcher_state;"
```

**Decision matrix:**
- A nonzero AND B shows fire `min_severity: priority` (or matrix fire cells with priority floors) → **diagnosis confirmed.** Proceed to the stop gate.
- A is ZERO → fires aren't reaching the bus at all; the diagnosis does NOT hold as-is. STOP. Report findings; do not apply the fix. (Check adapter enablement, NIFC fetch errors: `sudo docker logs meshai --since 24h 2>&1 | grep -i "nifc\|wfigs" | tail -20`.)
- A nonzero but B shows fire floor already `routine` → drop is elsewhere (matrix cells or region scope). STOP. Report B plus the matrix config; do not apply the fix blind.

End the Phase 0 report with `sudo docker ps`.

---

## ⛔ EVERYTHING BELOW REQUIRES MATT'S VERBATIM ACK: `I know and I accept`

Report the Phase 0 results including the §8-D burst count, restate the risk (§1), and wait.

## 9. Phase 1 — pre-seed the fires table (mandatory BEFORE the fixed code ever starts)

Run against the **still-running old container** (old code keeps suppressing at the floor, so this changes nothing live; after pre-seed the decider simply returns cooldown for those rows):

```bash
sudo docker exec meshai sqlite3 /app/data/<DBFILE> "
UPDATE fires
   SET last_broadcast_at      = strftime('%s','now'),
       first_broadcast_at     = COALESCE(first_broadcast_at, strftime('%s','now')),
       last_broadcast_acres   = current_acres,
       last_broadcast_contained = current_contained_pct
 WHERE last_broadcast_at IS NULL;"
```

Verify — MUST return 0 before proceeding:

```bash
sudo docker exec meshai sqlite3 /app/data/<DBFILE> \
  "SELECT COUNT(*) FROM fires WHERE last_broadcast_at IS NULL;"
```

If `sqlite3` is not present in the image: stop the container, run the same SQL with host `sqlite3` against the bind-mounted data directory (check the volume mapping in docker-compose.yml), verify count 0, and only then continue. Do NOT start the new image before the verify shows 0.

Consequence to state to Matt: currently-active fires will not get a retroactive "New" announcement; they will broadcast on their next real growth/containment change. Only genuinely new post-deploy ignitions announce as New.

## 10. Phase 2 — the fix (verbatim diff)

Target: `work/meshai/env/store.py`, inside `_emit_event` (function starts line 678). Anchor by grepping `event.data.update(gate.data_patch)` — it occurs once. Verify both anchor lines exist verbatim, then apply exactly:

```diff
--- a/work/meshai/env/store.py
+++ b/work/meshai/env/store.py
@@ def _emit_event(self, adapter, raw_evt: dict):
                     # Apply data_patch into event.data
                     event.data.update(gate.data_patch)
+                    # Promote decider overrides onto the Event itself, mirroring
+                    # the Central path (central/consumer.py:649-653). The gating
+                    # deciders stamp _severity_override (fire: "priority" on every
+                    # New/Update) and, for fire New/tombstone, a category
+                    # override. Merging them into event.data alone leaves
+                    # event.severity at the adapter's value ("routine" for fires
+                    # >=25 km from an anchor), which silently fails the
+                    # toggle/matrix min_severity floors (default "priority",
+                    # config.py:641). Native and Central must share identical
+                    # broadcast decisions.
+                    _sev_override = gate.data_patch.get("_severity_override")
+                    if _sev_override:
+                        event.severity = _sev_override
+                    _cat_override = gate.data_patch.get("category")
+                    if _cat_override:
+                        event.category = _cat_override
                     if gate.commit is not None:
                         event.data["_on_broadcast_committed"] = gate.commit
```

Notes for review:
- `Event` is a plain mutable `@dataclass` (`notifications/events.py:38`); attribute assignment is valid.
- Category promotion is intentional and safe: `wildfire_declared` maps to the `fire` toggle (`categories.py:~387`); formatter branch selection already falls back to `data["category"]`; cooldown keys become category-scoped, matching Central behavior (New vs Update cooldowns separate).
- This is the shared choke point: any category later added to `MESHAI_CUTOVER_CATEGORIES` (quake, swpc, avalanche, nws) gets the correct promotion automatically.

**Tests:** if a pytest environment exists on the LXC checkout, run the targeted subsets and compare against the known 10-failure baseline (per commit `0c76a80`, 1697 passed):

```bash
python3 -m pytest work/tests -k "fire or store or dispatch or gating" -q
```

If no local pytest env exists, note that in the report and proceed — the change is 16 lines with fully-traced semantics. Do NOT use `docker compose run` to run tests.

## 11. Phase 3 — commit and deploy

1. Present Matt a plain-English summary of the change (what/why in 3–5 sentences, drawn from §3). Wait for his confirmation. His understanding is the gate.
2. Commit with a message in the repo's convention, e.g.:
   `fix(native): promote decider _severity_override/category onto Event — restores Central-parity fire severity (silent min_severity floor drop)`
3. Deploy exactly:

```bash
sudo docker compose build meshai && sudo docker compose up -d
```

## 12. Phase 4 — post-deploy verification

Watch the first 2–3 NIFC poll cycles (read the poll interval from the wfigs/nifc adapter config during Phase 0):

```bash
sudo docker logs -f meshai 2>&1 | grep -Ei "Emitted nifc|matrix: dispatched|dispatched|SUPPRESSED|cold-start"
```

Expected: NO burst at startup (pre-seed verified 0); cold-start seed lines for any brand-new first-sight fires; subsequent genuine New/growth events showing `matrix: dispatched` or toggle-path delivery lines. Confirm on-air on the mesh when the first legit event lands.

Then confirm the counters are stable and report:

```bash
sudo docker exec meshai sqlite3 /app/data/<DBFILE> "SELECT * FROM dispatcher_state;"
sudo docker ps
```

Every report back to Matt ends with `sudo docker ps`.

## 13. Backlog (do NOT do now — list for future sessions)

1. Persist the fire seen-baseline across restarts to close the restart-window ignition gap (§6.1).
2. Decide policy for zero-acre first sightings (§6.2).
3. Extend pacing to the native emit path for multi-fire poll batches (§6.4).
4. Before cutting over quake/swpc/avalanche/nws via `MESHAI_CUTOVER_CATEGORIES`, re-verify override promotion end-to-end (covered by this fix, but confirm with shadow logs).
5. Add a drop counter + log line for the dispatcher severity-floor return (it is currently the only fully silent drop in the pipeline).