68 lines
2.8 KiB
Markdown
68 lines
2.8 KiB
Markdown
|
# Echo6 Vault Engine
|
||
|
|
|
||
|
|
Self-contained maintenance engine for the Obsidian knowledge vault at
|
||
|
|
`/home/zvx/projects/.ref/vault/`. Runs entirely on cortex (RTX A4000).
|
||
|
|
|
||
|
|
## What it does
|
||
|
|
|
||
|
|
| Layer | Tool | Job |
|
||
|
|
|-------|------|-----|
|
||
|
|
| Tagger | Qwen2.5-7B-Instruct (Ollama) | Classifies docs; assigns topic tags, type, entity refs |
|
||
|
|
| Embeddings | bge-m3 via TEI (reused service) | Embeds docs into Qdrant for similarity / related links |
|
||
|
|
| Lint | `lib/lint.py` (deterministic) | Enforces frontmatter schema; fixes safe violations automatically |
|
||
|
|
| Agent | `lib/agent.py` | Orchestrates tagger + embeddings over changed docs |
|
||
|
|
| Sweep | `sweep.sh` | Daily entry point; GPU guard, lint, agent, changelog |
|
||
|
|
|
||
|
|
## File layout
|
||
|
|
|
||
|
|
```
|
||
|
|
engine/
|
||
|
|
config.yaml — single source of truth (endpoints, vocab, schema, schedule)
|
||
|
|
Modelfile — pinned vault-tagger build (qwen2.5:7b-instruct-q8_0, temp 0.1)
|
||
|
|
bootstrap.sh — idempotent setup: verify services, build model, install cron
|
||
|
|
sweep.sh — daily maintenance sweep (called by cron)
|
||
|
|
prompts/
|
||
|
|
system.md — canonical system prompt (source of truth; synced into Modelfile)
|
||
|
|
fewshot.md — tagged examples for prompt engineering (added in Step 5)
|
||
|
|
lib/
|
||
|
|
vocab_gen.py — generates vocab.json from live infra inventory (Step 3)
|
||
|
|
lint.py — deterministic frontmatter linter (Step 4)
|
||
|
|
agent.py — tagger + embeddings agent (Step 5)
|
||
|
|
vocab.json — generated entity lexicon (not committed; built by vocab_gen.py)
|
||
|
|
changelog.md — append-only audit log of all automated changes
|
||
|
|
```
|
||
|
|
|
||
|
|
## How it runs
|
||
|
|
|
||
|
|
- **Daily cron** (`0 9 * * *` UTC): `sweep.sh` checks GPU VRAM, runs lint, runs agent
|
||
|
|
over docs changed since last sweep, appends to `changelog.md`.
|
||
|
|
- **Git pre-commit hook** (Step 4): runs lint against staged vault docs before commit.
|
||
|
|
- **Manual**: `./sweep.sh` or `python3 lib/agent.py --full` to reprocess all docs.
|
||
|
|
|
||
|
|
## Setup
|
||
|
|
|
||
|
|
```bash
|
||
|
|
./bootstrap.sh
|
||
|
|
```
|
||
|
|
|
||
|
|
`bootstrap.sh` is idempotent and documents every step. Base model weights (~8 GB) are
|
||
|
|
pulled by bootstrap — they are not committed to the repo. Rebuilding from scratch:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
./bootstrap.sh # pulls qwen2.5:7b-instruct-q8_0, builds vault-tagger, installs cron
|
||
|
|
```
|
||
|
|
|
||
|
|
## Vocabulary
|
||
|
|
|
||
|
|
- **Tier 1 — topic tags**: stable list in `config.yaml` under `topic_categories`
|
||
|
|
- **Tier 2 — entity lexicon**: generated into `vocab.json` by `lib/vocab_gen.py`
|
||
|
|
from live Proxmox, Docker, and Headscale inventory. Regenerate anytime with:
|
||
|
|
`python3 lib/vocab_gen.py`
|
||
|
|
|
||
|
|
## Configuration
|
||
|
|
|
||
|
|
All tunables are in `config.yaml`. Key settings:
|
||
|
|
|
||
|
|
- `behavior.auto_apply` — write changes directly (true) or dry-run only (false)
|
||
|
|
- `behavior.confidence_threshold` — below this, changes are flagged not applied
|
||
|
|
- `schedule.defer_if_gpu_busy_mib` — skip sweep if GPU is already under load
|