refactored-recon/phases/phase-6c-code-cleanup.md

# Phase 6c: Code Cleanup

## Objective

Remove dead code paths left over from the refactor. Investigation first,
deletion second — only remove what's confirmed dead.

## Investigation Findings

### Expected dead code vs reality

| Item | Expected status | Actual status |
|------|----------------|---------------|
| `scanner_loop` | Dead function in recon.py | **Already removed** in Phase 5c-1 |
| `peertube_scanner_loop` | Dead function in recon.py | **Already removed** in Phase 5c-1 |
| `crawler_scheduler_loop` | Dead function in recon.py | **Already removed** in Phase 5c-1 |
| `organizer_loop` | Dead function in recon.py | **Already removed** in Phase 5c-1 |
| Extract worker thread | Vestigial in cmd_service() | **Confirmed dead** — 0 items queued, silent 24h+ |
| `lib/crawler.py` | Legacy module | **Confirmed dead** — only used by CLI subcommand |
| `lib/web_scraper.py` | Legacy module | **ALIVE** — `chunk_text()` used by transcript_processor |
| `lib/new_pipeline.py` | Legacy module | **ALIVE** — active Stream B library management tool (1,637 lines, created Apr 13) |
| `lib/peertube_scraper.py` | Legacy module | **ALIVE** — only mechanism for transcript ingestion |
| `lib/extractor.py` | Dead module | **ALIVE** — used by `cmd_run` CLI for batch processing |

### Additional findings

- **24 `.bak` files** found across `/opt/recon/` (untracked, manual pre-edit safety backups from Feb-Apr 2026). All originals preserved in git history.
- **File ownership**: All 21 `.py` files + `recon.py` correctly owned by zvx. No corrections needed.
- **No TODO/DEPRECATED comments** found in any lib/ file.
- **All imports in recon.py** confirmed used (no dead imports at module level).
- **PeerTube transcript ingestion** has no automatic mechanism since Phase 5c-1 removed `peertube_scanner_loop`. Ingestion is manual only (CLI or dashboard API endpoint).

## What Was Removed

### recon.py edits (-89 lines, +3 lines)

1. **Extract worker thread** removed from `cmd_service()`:
   - `from lib.extractor import run_extraction` import
   - `extract_workers` variable
   - `'extract': 0` from totals dict
   - Extract `threading.Thread(target=stage_loop, ...)` from thread list
   - Extract workers from startup log message

2. **`cmd_crawl` function** deleted (65 lines) — CLI handler for `recon crawl`

3. **Crawl argparse subparser** deleted (15 lines) — `recon crawl` subcommand registration

4. **Docstring** updated to remove `crawl` from subcommand list

### Files deleted

| File | Lines | Reason |
|------|-------|--------|
| `lib/crawler.py` | 432 | Only referenced by deleted `cmd_crawl` CLI subcommand |

### .bak files deleted (24 files, untracked)

| File | Size |
|------|------|
| `recon.py.bak-pre-streamb` | 48K |
| `recon.py.bak-pre-ux` | 35K |
| `recon.py.bak-pre-crawler` | 35K |
| `recon.py.bak.202602171647` | 33K |
| `config.yaml.bak-pre-crawler` | 4K |
| `config.yaml.bak-pre-streamb` | 13K |
| `lib/api.py.bak` + 5 more api.py backups | 498K total |
| `lib/embedder.py.bak` | 15K |
| `lib/enricher.py.bak` | 17K |
| `lib/extractor.py.bak` | 18K |
| `lib/status.py.bak-pre-ux` | 10K |
| `lib/status.py.bak-pre-streamb` | 13K |
| `scripts/validate.py.bak` | 6K |
| `scripts/rebuild_qdrant.py.bak` | 6K |
| `static/js/dashboard.js.bak` | 11K |
| `static/js/peertube.js.bak.20260223` | 5K |
| `templates/search.html.bak` | 2K |
| `templates/knowledge/dashboard.html.bak` | 3K |

## What Was Kept (and why)

| Module | Lines | Why kept |
|--------|-------|----------|
| `lib/web_scraper.py` | 324 | `transcript_processor.py` imports `chunk_text()` |
| `lib/new_pipeline.py` | 1,637 | Active Stream B library management CLI (created Apr 13) |
| `lib/peertube_scraper.py` | 580 | Only way to ingest PeerTube transcripts |
| `lib/extractor.py` | 601 | Used by `cmd_run` CLI for batch PDF processing |

## Verification

| Check | Result |
|-------|--------|
| Compile (recon.py) | OK |
| Import (recon module) | OK |
| Import (dispatcher, filing, processors) | OK |
| cmd_service assertions | extract worker absent, dispatch_loop present, filing_worker_loop present |
| Zero crawler references in .py files | Confirmed |
| Service restart | Clean, active |
| Thread count | 13 tasks (was 14 — extract removed) |
| Threads started | enrich, embed, dispatcher, filing, progress, dashboard, metrics |
| Extract thread | Absent (confirmed by logs: no `[extract] Stage started`) |
| Errors (60s window) | 0 |
| DB rows | catalogue=29,812, documents=29,812 (unchanged) |
| Dashboard | Responsive |
| Hopper | Empty |

## Commit

- **Commit:** `efae402` on `refactor` branch
- **Diff:** 2 files changed, 3 insertions(+), 521 deletions(-)
- **Pushed to:** `forge.echo6.co/matt/recon` (origin/refactor)