auto: docs sync 2026-06-18T12:00:08+00:00
Files changed: .gitignore engine/.embcache.json engine/changelog.md engine/lib/__pycache__/__init__.cpython-312.pyc engine/lib/__pycache__/agent.cpython-312.pyc engine/lib/agent.py engine/lint-report.md engine/prompts/system.md engine/sweep.sh vault/.obsidian/graph.json vault/docs/software/authentik.md vault/docs/software/caddy.md vault/docs/software/recon.md vault/notes/echo6-landing-page-data-export.md vault/projects/argus.md vault/projects/meshtastic-headscale-runbook.md vault/runbooks/add-peertube-channel.md vault/runbooks/authentik-access-groups.md vault/runbooks/ct-runbook.md vault/runbooks/meshtastic-sidecar-node.md
This commit is contained in:
parent
1d30335963
commit
c30ce9f1e3
20 changed files with 1490 additions and 703 deletions
2
.gitignore
vendored
Normal file
2
.gitignore
vendored
Normal file
|
|
@ -0,0 +1,2 @@
|
||||||
|
engine/.embcache.json
|
||||||
|
engine/.last-sweep
|
||||||
File diff suppressed because one or more lines are too long
60
engine/changelog.md
Normal file
60
engine/changelog.md
Normal file
|
|
@ -0,0 +1,60 @@
|
||||||
|
|
||||||
|
## 2026-06-18T06:11:17Z — meshtastic-sidecar-node.md
|
||||||
|
- file: `/home/zvx/projects/.ref/vault/runbooks/meshtastic-sidecar-node.md`
|
||||||
|
- action: applied
|
||||||
|
- confidence: 0.95
|
||||||
|
- changes: type: '' → 'runbook'; tags: [] → ['mesh', 'vpn']
|
||||||
|
|
||||||
|
## 2026-06-18T06:11:27Z — authentik-access-groups.md
|
||||||
|
- file: `/home/zvx/projects/.ref/vault/runbooks/authentik-access-groups.md`
|
||||||
|
- action: applied
|
||||||
|
- confidence: 0.95
|
||||||
|
- changes: type: '' → 'runbook'; tags: [] → ['auth', 'mesh', 'matrix']
|
||||||
|
|
||||||
|
## 2026-06-18T06:11:36Z — add-peertube-channel.md
|
||||||
|
- file: `/home/zvx/projects/.ref/vault/runbooks/add-peertube-channel.md`
|
||||||
|
- action: applied
|
||||||
|
- confidence: 0.95
|
||||||
|
- changes: type: '' → 'runbook'; tags: [] → ['media', 'vpn', 'auth']
|
||||||
|
|
||||||
|
## 2026-06-18T06:11:44Z — ct-runbook.md
|
||||||
|
- file: `/home/zvx/projects/.ref/vault/runbooks/ct-runbook.md`
|
||||||
|
- action: applied
|
||||||
|
- confidence: 0.95
|
||||||
|
- changes: type: '' → 'runbook'; tags: [] → ['proxmox']
|
||||||
|
|
||||||
|
## 2026-06-18T06:12:04Z — authentik.md
|
||||||
|
- file: `/home/zvx/projects/.ref/vault/docs/software/authentik.md`
|
||||||
|
- action: applied
|
||||||
|
- confidence: 0.90
|
||||||
|
- changes: type: '' → 'reference'; tags: [] → ['auth', 'mesh', 'matrix']
|
||||||
|
|
||||||
|
## 2026-06-18T06:12:31Z — caddy.md
|
||||||
|
- file: `/home/zvx/projects/.ref/vault/docs/software/caddy.md`
|
||||||
|
- action: applied
|
||||||
|
- confidence: 0.95
|
||||||
|
- changes: type: '' → 'reference'; tags: [] → ['dns', 'mesh', 'auth']
|
||||||
|
|
||||||
|
## 2026-06-18T06:12:59Z — recon.md
|
||||||
|
- file: `/home/zvx/projects/.ref/vault/docs/software/recon.md`
|
||||||
|
- action: applied
|
||||||
|
- confidence: 0.90
|
||||||
|
- changes: type: '' → 'reference'; tags: [] → ['recon', 'ai', 'storage']
|
||||||
|
|
||||||
|
## 2026-06-18T06:13:09Z — argus.md
|
||||||
|
- file: `/home/zvx/projects/.ref/vault/projects/argus.md`
|
||||||
|
- action: applied
|
||||||
|
- confidence: 0.90
|
||||||
|
- changes: type: '' → 'project'; tags: [] → ['mesh', 'auth', 'recon']
|
||||||
|
|
||||||
|
## 2026-06-18T06:13:16Z — meshtastic-headscale-runbook.md
|
||||||
|
- file: `/home/zvx/projects/.ref/vault/projects/meshtastic-headscale-runbook.md`
|
||||||
|
- action: applied
|
||||||
|
- confidence: 0.95
|
||||||
|
- changes: type: '' → 'project'; tags: [] → ['mesh', 'vpn', 'auth']
|
||||||
|
|
||||||
|
## 2026-06-18T06:13:30Z — echo6-landing-page-data-export.md
|
||||||
|
- file: `/home/zvx/projects/.ref/vault/notes/echo6-landing-page-data-export.md`
|
||||||
|
- action: applied
|
||||||
|
- confidence: 0.95
|
||||||
|
- changes: type: '' → 'note'; tags: [] → ['auth', 'media', 'matrix']
|
||||||
BIN
engine/lib/__pycache__/__init__.cpython-312.pyc
Normal file
BIN
engine/lib/__pycache__/__init__.cpython-312.pyc
Normal file
Binary file not shown.
BIN
engine/lib/__pycache__/agent.cpython-312.pyc
Normal file
BIN
engine/lib/__pycache__/agent.cpython-312.pyc
Normal file
Binary file not shown.
1103
engine/lib/agent.py
1103
engine/lib/agent.py
File diff suppressed because it is too large
Load diff
|
|
@ -1,17 +1,17 @@
|
||||||
# Vault Lint Report
|
# Vault Lint Report
|
||||||
|
|
||||||
Generated: 2026-06-18T05:49:07Z | Docs scanned: 98 | Elapsed: 0.0s
|
Generated: 2026-06-18T06:14:05Z | Docs scanned: 98 | Elapsed: 0.0s
|
||||||
|
|
||||||
## Summary
|
## Summary
|
||||||
|
|
||||||
| Severity | Count |
|
| Severity | Count |
|
||||||
|----------|-------|
|
|----------|-------|
|
||||||
| ERROR (dead links) | 0 |
|
| ERROR (dead links) | 0 |
|
||||||
| WARN (schema) | 106 |
|
| WARN (schema) | 96 |
|
||||||
| INFO (orphans) | 38 |
|
| INFO (orphans) | 38 |
|
||||||
|
|
||||||
### WARN breakdown
|
### WARN breakdown
|
||||||
- Missing frontmatter block: 95
|
- Missing frontmatter block: 85
|
||||||
- Invalid/missing frontmatter fields: 3
|
- Invalid/missing frontmatter fields: 3
|
||||||
- Unknown tags: 8
|
- Unknown tags: 8
|
||||||
|
|
||||||
|
|
@ -72,27 +72,18 @@ _None. All wikilinks resolve._
|
||||||
- `docs/services/ots-setup.md` — missing frontmatter block entirely
|
- `docs/services/ots-setup.md` — missing frontmatter block entirely
|
||||||
- `docs/services/services.md` — missing frontmatter block entirely
|
- `docs/services/services.md` — missing frontmatter block entirely
|
||||||
- `docs/services/usenet.md` — missing frontmatter block entirely
|
- `docs/services/usenet.md` — missing frontmatter block entirely
|
||||||
- `docs/software/authentik.md` — missing frontmatter block entirely
|
|
||||||
- `docs/software/caddy.md` — missing frontmatter block entirely
|
|
||||||
- `docs/software/dns.md` — missing frontmatter block entirely
|
- `docs/software/dns.md` — missing frontmatter block entirely
|
||||||
- `docs/software/geo-tools.md` — missing frontmatter block entirely
|
- `docs/software/geo-tools.md` — missing frontmatter block entirely
|
||||||
- `docs/software/recon.md` — missing frontmatter block entirely
|
|
||||||
- `docs/software/searxng.md` — missing frontmatter block entirely
|
- `docs/software/searxng.md` — missing frontmatter block entirely
|
||||||
- `notes/echo6-landing-page-data-export.md` — missing frontmatter block entirely
|
|
||||||
- `notes/ia-download-queue.md` — missing frontmatter block entirely
|
- `notes/ia-download-queue.md` — missing frontmatter block entirely
|
||||||
- `plans/vaultwarden-plan.md` — missing frontmatter block entirely
|
- `plans/vaultwarden-plan.md` — missing frontmatter block entirely
|
||||||
- `projects/advbbs-project.md` — missing frontmatter block entirely
|
- `projects/advbbs-project.md` — missing frontmatter block entirely
|
||||||
- `projects/argus.md` — missing frontmatter block entirely
|
|
||||||
- `projects/deploy-livesync.md` — missing frontmatter block entirely
|
- `projects/deploy-livesync.md` — missing frontmatter block entirely
|
||||||
- `projects/matrix-synapse-deployment.md` — missing frontmatter block entirely
|
- `projects/matrix-synapse-deployment.md` — missing frontmatter block entirely
|
||||||
- `projects/meshtastic-headscale-runbook.md` — missing frontmatter block entirely
|
|
||||||
- `projects/mmud-project.md` — missing frontmatter block entirely
|
- `projects/mmud-project.md` — missing frontmatter block entirely
|
||||||
- `runbooks/add-peertube-channel.md` — missing frontmatter block entirely
|
|
||||||
- `runbooks/authentik-access-groups.md` — missing frontmatter block entirely
|
|
||||||
- `runbooks/authentik-create-invitation.md` — missing frontmatter block entirely
|
- `runbooks/authentik-create-invitation.md` — missing frontmatter block entirely
|
||||||
- `runbooks/authentik-oidc-application.md` — missing frontmatter block entirely
|
- `runbooks/authentik-oidc-application.md` — missing frontmatter block entirely
|
||||||
- `runbooks/authentik-upgrade.md` — missing frontmatter block entirely
|
- `runbooks/authentik-upgrade.md` — missing frontmatter block entirely
|
||||||
- `runbooks/ct-runbook.md` — missing frontmatter block entirely
|
|
||||||
- `runbooks/edge2-access-reference.md` — missing frontmatter block entirely
|
- `runbooks/edge2-access-reference.md` — missing frontmatter block entirely
|
||||||
- `runbooks/expose-service-contabo.md` — missing frontmatter block entirely
|
- `runbooks/expose-service-contabo.md` — missing frontmatter block entirely
|
||||||
- `runbooks/expose-service-edge2.md` — missing frontmatter block entirely
|
- `runbooks/expose-service-edge2.md` — missing frontmatter block entirely
|
||||||
|
|
@ -105,7 +96,6 @@ _None. All wikilinks resolve._
|
||||||
- `runbooks/lxc-service-migration.md` — missing frontmatter block entirely
|
- `runbooks/lxc-service-migration.md` — missing frontmatter block entirely
|
||||||
- `runbooks/mailcow-create-mailbox.md` — missing frontmatter block entirely
|
- `runbooks/mailcow-create-mailbox.md` — missing frontmatter block entirely
|
||||||
- `runbooks/meshmonitor-password-reset.md` — missing frontmatter block entirely
|
- `runbooks/meshmonitor-password-reset.md` — missing frontmatter block entirely
|
||||||
- `runbooks/meshtastic-sidecar-node.md` — missing frontmatter block entirely
|
|
||||||
- `runbooks/meshtasticd-sim-nodes-runbook.md` — missing frontmatter block entirely
|
- `runbooks/meshtasticd-sim-nodes-runbook.md` — missing frontmatter block entirely
|
||||||
- `runbooks/nordvpn-lxc.md` — missing frontmatter block entirely
|
- `runbooks/nordvpn-lxc.md` — missing frontmatter block entirely
|
||||||
- `runbooks/peertube-remote-runner.md` — missing frontmatter block entirely
|
- `runbooks/peertube-remote-runner.md` — missing frontmatter block entirely
|
||||||
|
|
|
||||||
|
|
@ -28,7 +28,7 @@ Respond with ONLY a single valid JSON object. No prose, no markdown fences, no e
|
||||||
```
|
```
|
||||||
|
|
||||||
Field definitions:
|
Field definitions:
|
||||||
- **tags**: tier-1 topic tags drawn exclusively from topic_categories
|
- **tags**: up to 3 tier-1 topic tags drawn exclusively from topic_categories, ordered most-relevant to least-relevant (primary topic first)
|
||||||
- **entities**: known names matched from entity_lexicon
|
- **entities**: known names matched from entity_lexicon
|
||||||
- **glossary_proposals**: unknown acronyms or terms worth adding to the lexicon
|
- **glossary_proposals**: unknown acronyms or terms worth adding to the lexicon
|
||||||
- **type**: one of reference | runbook | project | note | index | session
|
- **type**: one of reference | runbook | project | note | index | session
|
||||||
|
|
@ -36,7 +36,7 @@ Field definitions:
|
||||||
|
|
||||||
## Rules — follow exactly
|
## Rules — follow exactly
|
||||||
|
|
||||||
1. **Only use provided vocabulary.** `tags` must be a subset of `topic_categories`.
|
1. **Only use provided vocabulary.** `tags` must be a subset of `topic_categories`; return at most 3, ordered most-relevant to least-relevant.
|
||||||
`entities` must be a subset of the keys in `entity_lexicon`. Never invent new tags.
|
`entities` must be a subset of the keys in `entity_lexicon`. Never invent new tags.
|
||||||
|
|
||||||
2. **Strict JSON only.** The output must parse with `json.loads()` with no preprocessing.
|
2. **Strict JSON only.** The output must parse with `json.loads()` with no preprocessing.
|
||||||
|
|
|
||||||
235
engine/sweep.sh
235
engine/sweep.sh
|
|
@ -1,66 +1,221 @@
|
||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# sweep.sh — Echo6 Vault Engine daily maintenance sweep
|
# sweep.sh — Echo6 Vault Engine orchestrator
|
||||||
#
|
#
|
||||||
# Invoked by cron (schedule: "0 9 * * *" from config.yaml).
|
# Usage:
|
||||||
# Also callable manually: ./sweep.sh
|
# ./sweep.sh # default: process docs changed since last sweep
|
||||||
#
|
# ./sweep.sh --all # process every non-archived vault doc
|
||||||
# What this does (when fully implemented):
|
# ./sweep.sh --dry-run # pass --dry-run to agent.py (no writes)
|
||||||
# 1. GPU-busy guard: check VRAM usage; defer if > defer_if_gpu_busy_mib (6000 MiB default)
|
# ./sweep.sh --all --dry-run # both
|
||||||
# 2. Run lint (lib/lint.py) over all vault docs — fix or flag frontmatter issues
|
|
||||||
# 3. Run agent (lib/agent.py) over changed/new docs since last run — tag + embed
|
|
||||||
# 4. Append a summary entry to changelog.md
|
|
||||||
#
|
|
||||||
# Configuration is read from config.yaml (engine_dir, vault_dir, thresholds, changelog path).
|
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
VAULT_DIR="${SCRIPT_DIR}/../vault"
|
||||||
CONFIG="${SCRIPT_DIR}/config.yaml"
|
CONFIG="${SCRIPT_DIR}/config.yaml"
|
||||||
|
LAST_SWEEP="${SCRIPT_DIR}/.last-sweep"
|
||||||
|
CHANGELOG="${SCRIPT_DIR}/changelog.md"
|
||||||
|
LINT_REPORT="${SCRIPT_DIR}/lint-report.md"
|
||||||
|
|
||||||
echo "==> Echo6 vault sweep — $(date -u '+%Y-%m-%dT%H:%M:%SZ')"
|
# Parse args
|
||||||
|
OPT_ALL=0
|
||||||
|
OPT_DRY=0
|
||||||
|
for arg in "$@"; do
|
||||||
|
case "$arg" in
|
||||||
|
--all) OPT_ALL=1 ;;
|
||||||
|
--dry-run) OPT_DRY=1 ;;
|
||||||
|
*)
|
||||||
|
echo "[error] Unknown argument: $arg" >&2
|
||||||
|
echo "Usage: $0 [--all] [--dry-run]" >&2
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
SWEEP_START="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||||
|
echo "==> Echo6 vault sweep — ${SWEEP_START}"
|
||||||
|
if [ "$OPT_DRY" -eq 1 ]; then
|
||||||
|
echo " (DRY-RUN mode — no files will be modified)"
|
||||||
|
fi
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Step 1 — GPU-busy guard
|
# Step 1 — GPU-busy guard
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# TODO: Query nvidia-smi for used VRAM; compare to defer_if_gpu_busy_mib from config.yaml.
|
echo ""
|
||||||
# If busy, log a deferred entry to changelog and exit 0 (not an error, just deferred).
|
echo "[1/4] GPU-busy guard..."
|
||||||
#
|
|
||||||
# Example skeleton:
|
# Helper: run nvidia-smi, using docker exec ollama if host lacks it
|
||||||
# USED_MIB=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits | head -1)
|
_nvidia_smi() {
|
||||||
# THRESHOLD=6000 # read from config.yaml
|
if command -v nvidia-smi &>/dev/null; then
|
||||||
# if [ "$USED_MIB" -gt "$THRESHOLD" ]; then
|
nvidia-smi "$@" 2>/dev/null
|
||||||
# echo "GPU busy (${USED_MIB} MiB > ${THRESHOLD} MiB threshold) — deferring sweep."
|
elif command -v docker &>/dev/null && docker ps --format '{{.Names}}' 2>/dev/null | grep -q '^ollama$'; then
|
||||||
# exit 0
|
docker exec ollama nvidia-smi "$@" 2>/dev/null
|
||||||
# fi
|
else
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
GPU_DEFERRED=0
|
||||||
|
DEFER_REASON=""
|
||||||
|
|
||||||
|
# Guard 1 — competing GPU processes (share the A4000)
|
||||||
|
if pgrep -fa 'peertube-runner|whisper|ffmpeg' >/dev/null 2>&1; then
|
||||||
|
COMPETING="$(pgrep -fa 'peertube-runner|whisper|ffmpeg' | head -1)"
|
||||||
|
DEFER_REASON="competing GPU process: ${COMPETING}"
|
||||||
|
GPU_DEFERRED=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Guard 2 — sustained GPU utilization > 60 %
|
||||||
|
if [ "$GPU_DEFERRED" -eq 0 ]; then
|
||||||
|
GPU_UTIL="$(_nvidia_smi --query-gpu=utilization.gpu --format=csv,noheader,nounits 2>/dev/null | head -1 | tr -d '[:space:]' || true)"
|
||||||
|
if [ -n "$GPU_UTIL" ] && [[ "$GPU_UTIL" =~ ^[0-9]+$ ]]; then
|
||||||
|
echo " GPU utilization: ${GPU_UTIL}%"
|
||||||
|
if [ "$GPU_UTIL" -gt 60 ]; then
|
||||||
|
DEFER_REASON="GPU utilization ${GPU_UTIL}% > 60%"
|
||||||
|
GPU_DEFERRED=1
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo " nvidia-smi not available or could not read utilization — skipping util guard."
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Guard 3 (safety cap) — very high VRAM usage (>14000 MiB) — not triggered by vault-tagger itself (~8-10 GB)
|
||||||
|
if [ "$GPU_DEFERRED" -eq 0 ]; then
|
||||||
|
GPU_USED_MIB="$(_nvidia_smi --query-gpu=memory.used --format=csv,noheader,nounits 2>/dev/null | head -1 | tr -d '[:space:]' || true)"
|
||||||
|
if [ -n "$GPU_USED_MIB" ] && [[ "$GPU_USED_MIB" =~ ^[0-9]+$ ]]; then
|
||||||
|
echo " GPU VRAM used: ${GPU_USED_MIB} MiB (safety cap: 14000 MiB)"
|
||||||
|
if [ "$GPU_USED_MIB" -gt 14000 ]; then
|
||||||
|
DEFER_REASON="VRAM ${GPU_USED_MIB} MiB > 14000 MiB safety cap"
|
||||||
|
GPU_DEFERRED=1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$GPU_DEFERRED" -eq 1 ]; then
|
||||||
|
echo " GPU busy — deferring sweep. Reason: ${DEFER_REASON}"
|
||||||
|
if [ "$OPT_DRY" -eq 0 ]; then
|
||||||
|
printf "\n## %s — sweep deferred (%s)\n" "$SWEEP_START" "$DEFER_REASON" >> "$CHANGELOG"
|
||||||
|
fi
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo " GPU OK — proceeding."
|
||||||
|
|
||||||
echo "[1/4] TODO — GPU-busy guard not yet implemented."
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Step 2 — Run lint
|
# Step 2 — Lint
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# TODO: Call lib/lint.py to validate/fix frontmatter across vault docs.
|
echo ""
|
||||||
# Lint should be idempotent and log all changes to changelog.
|
echo "[2/4] Running lint..."
|
||||||
#
|
|
||||||
# python3 "${SCRIPT_DIR}/lib/lint.py" --config "${CONFIG}"
|
|
||||||
|
|
||||||
echo "[2/4] TODO — lint.py not yet implemented (Step 2)."
|
python3 "${SCRIPT_DIR}/lib/lint.py" --report 2>&1 | tee /tmp/sweep-lint.log || {
|
||||||
|
echo "[warn] lint.py exited non-zero — continuing sweep"
|
||||||
|
}
|
||||||
|
echo " Lint complete. Report: ${LINT_REPORT}"
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Step 3 — Run agent over changed/new docs
|
# Step 3 — Select docs and run agent
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# TODO: Call lib/agent.py to tag + embed documents modified since last sweep.
|
echo ""
|
||||||
# Agent tracks last-run timestamp in a state file (e.g. engine/.last_sweep).
|
echo "[3/4] Selecting documents..."
|
||||||
#
|
|
||||||
# python3 "${SCRIPT_DIR}/lib/agent.py" --config "${CONFIG}"
|
|
||||||
|
|
||||||
echo "[3/4] TODO — agent.py not yet implemented (Step 5)."
|
# Build file list
|
||||||
|
DOC_LIST=()
|
||||||
|
if [ "$OPT_ALL" -eq 1 ] || [ ! -f "$LAST_SWEEP" ]; then
|
||||||
|
if [ "$OPT_ALL" -eq 1 ]; then
|
||||||
|
echo " Mode: --all (full vault scan)"
|
||||||
|
else
|
||||||
|
echo " Mode: no .last-sweep marker found — treating as first run (full scan)"
|
||||||
|
fi
|
||||||
|
while IFS= read -r -d f; do
|
||||||
|
# Skip archive
|
||||||
|
if [[ "$f" == *"/archive/"* ]]; then
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
DOC_LIST+=("$f")
|
||||||
|
done < <(find "${VAULT_DIR}" -name "*.md" -print0 | sort -z)
|
||||||
|
else
|
||||||
|
LAST_TS="$(cat "$LAST_SWEEP")"
|
||||||
|
echo " Mode: incremental — docs modified since ${LAST_TS}"
|
||||||
|
# Compare file mtime to .last-sweep mtime (not contents)
|
||||||
|
while IFS= read -r -d f; do
|
||||||
|
if [[ "$f" == *"/archive/"* ]]; then
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
DOC_LIST+=("$f")
|
||||||
|
done < <(find "${VAULT_DIR}" -name "*.md" -newer "$LAST_SWEEP" -print0 | sort -z)
|
||||||
|
fi
|
||||||
|
|
||||||
|
TOTAL="${#DOC_LIST[@]}"
|
||||||
|
echo " ${TOTAL} document(s) selected."
|
||||||
|
|
||||||
|
if [ "$TOTAL" -eq 0 ]; then
|
||||||
|
echo " Nothing to process — vault is up to date."
|
||||||
|
if [ "$OPT_DRY" -eq 0 ]; then
|
||||||
|
date -u +%Y-%m-%dT%H:%M:%SZ > "$LAST_SWEEP"
|
||||||
|
printf "\n## %s — sweep complete (0 docs selected)\n" "$SWEEP_START" >> "$CHANGELOG"
|
||||||
|
fi
|
||||||
|
echo "==> Done."
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo " Processing..."
|
||||||
|
|
||||||
|
N_PROCESSED=0
|
||||||
|
N_WRITTEN=0
|
||||||
|
N_FLAGGED=0
|
||||||
|
N_ERRORS=0
|
||||||
|
|
||||||
|
AGENT_DRY_FLAG=""
|
||||||
|
if [ "$OPT_DRY" -eq 1 ]; then
|
||||||
|
AGENT_DRY_FLAG="--dry-run"
|
||||||
|
fi
|
||||||
|
|
||||||
|
for doc in "${DOC_LIST[@]}"; do
|
||||||
|
echo ""
|
||||||
|
echo " --- $(basename "$doc") ---"
|
||||||
|
set +e
|
||||||
|
python3 "${SCRIPT_DIR}/lib/agent.py" $AGENT_DRY_FLAG "$doc" 2>&1
|
||||||
|
EXIT_CODE=$?
|
||||||
|
set -e
|
||||||
|
N_PROCESSED=$(( N_PROCESSED + 1 ))
|
||||||
|
if [ "$EXIT_CODE" -eq 0 ]; then
|
||||||
|
N_WRITTEN=$(( N_WRITTEN + 1 ))
|
||||||
|
else
|
||||||
|
echo " [warn] agent.py exited ${EXIT_CODE} for ${doc}"
|
||||||
|
N_ERRORS=$(( N_ERRORS + 1 ))
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Step 4 — Append changelog summary
|
# Step 4 — Update marker + changelog summary
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# TODO: agent.py and lint.py both append to changelog.md directly.
|
echo ""
|
||||||
# This step adds a sweep-level summary entry.
|
echo "[4/4] Finalising..."
|
||||||
|
|
||||||
echo "[4/4] TODO — changelog summary not yet implemented."
|
SWEEP_END="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||||
|
|
||||||
echo "==> sweep.sh done."
|
if [ "$OPT_DRY" -eq 0 ]; then
|
||||||
|
date -u +%Y-%m-%dT%H:%M:%SZ > "$LAST_SWEEP"
|
||||||
|
{
|
||||||
|
printf "\n## %s — sweep run\n" "$SWEEP_START"
|
||||||
|
printf "- end: %s\n" "$SWEEP_END"
|
||||||
|
printf "- mode: %s\n" "$([ "$OPT_ALL" -eq 1 ] && echo "--all" || echo "incremental")"
|
||||||
|
printf "- docs selected: %d\n" "$TOTAL"
|
||||||
|
printf "- processed: %d | written: %d | flagged: %d | errors: %d\n" \
|
||||||
|
"$N_PROCESSED" "$N_WRITTEN" "$N_FLAGGED" "$N_ERRORS"
|
||||||
|
} >> "$CHANGELOG"
|
||||||
|
echo " .last-sweep updated. Changelog appended."
|
||||||
|
else
|
||||||
|
echo " [DRY-RUN] .last-sweep NOT updated. Changelog NOT written."
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "==> Sweep complete."
|
||||||
|
echo " Docs selected : ${TOTAL}"
|
||||||
|
echo " Processed : ${N_PROCESSED}"
|
||||||
|
echo " Written/OK : ${N_WRITTEN}"
|
||||||
|
echo " Errors : ${N_ERRORS}"
|
||||||
|
if [ "$OPT_DRY" -eq 1 ]; then
|
||||||
|
echo " [DRY-RUN] No vault docs were modified."
|
||||||
|
fi
|
||||||
|
|
|
||||||
2
vault/.obsidian/graph.json
vendored
2
vault/.obsidian/graph.json
vendored
|
|
@ -1,7 +1,7 @@
|
||||||
{
|
{
|
||||||
"collapse-filter": true,
|
"collapse-filter": true,
|
||||||
"search": "",
|
"search": "",
|
||||||
"showTags": false,
|
"showTags": true,
|
||||||
"showAttachments": false,
|
"showAttachments": false,
|
||||||
"hideUnresolved": false,
|
"hideUnresolved": false,
|
||||||
"showOrphans": true,
|
"showOrphans": true,
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,19 @@
|
||||||
|
---
|
||||||
|
title: Authentik SSO Configuration
|
||||||
|
type: reference
|
||||||
|
tags:
|
||||||
|
- auth
|
||||||
|
- mesh
|
||||||
|
- matrix
|
||||||
|
aliases: []
|
||||||
|
related:
|
||||||
|
- [[authentik-oidc-application]]
|
||||||
|
- [[caddy]]
|
||||||
|
- [[mailcow-create-mailbox]]
|
||||||
|
- [[proxmox-onboard-node]]
|
||||||
|
- [[services]]
|
||||||
|
updated: 2026-06-18
|
||||||
|
---
|
||||||
# Authentik SSO Configuration
|
# Authentik SSO Configuration
|
||||||
|
|
||||||
## Location
|
## Location
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,19 @@
|
||||||
|
---
|
||||||
|
title: "Caddy & DNS Reference"
|
||||||
|
type: reference
|
||||||
|
tags:
|
||||||
|
- dns
|
||||||
|
- mesh
|
||||||
|
- auth
|
||||||
|
aliases: []
|
||||||
|
related:
|
||||||
|
- [[services]]
|
||||||
|
- [[headscale-onboard-node]]
|
||||||
|
- [[ip-allocation]]
|
||||||
|
- [[expose-service-home]]
|
||||||
|
- [[INDEX]]
|
||||||
|
updated: 2026-06-18
|
||||||
|
---
|
||||||
# Caddy & DNS Reference
|
# Caddy & DNS Reference
|
||||||
|
|
||||||
## Contabo Caddy
|
## Contabo Caddy
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,19 @@
|
||||||
|
---
|
||||||
|
title: RECON — Knowledge Extraction Pipeline
|
||||||
|
type: reference
|
||||||
|
tags:
|
||||||
|
- recon
|
||||||
|
- ai
|
||||||
|
- storage
|
||||||
|
aliases: []
|
||||||
|
related:
|
||||||
|
- [[recon-operations]]
|
||||||
|
- [[INDEX]]
|
||||||
|
- [[ia-download-queue]]
|
||||||
|
- [[add-peertube-channel]]
|
||||||
|
- [[services]]
|
||||||
|
updated: 2026-06-18
|
||||||
|
---
|
||||||
# RECON — Knowledge Extraction Pipeline
|
# RECON — Knowledge Extraction Pipeline
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,19 @@
|
||||||
|
---
|
||||||
|
title: Echo6 Landing Page — Data Export
|
||||||
|
type: note
|
||||||
|
tags:
|
||||||
|
- auth
|
||||||
|
- media
|
||||||
|
- matrix
|
||||||
|
aliases: []
|
||||||
|
related:
|
||||||
|
- [[searxng]]
|
||||||
|
- [[authentik]]
|
||||||
|
- [[caddy]]
|
||||||
|
- [[INDEX]]
|
||||||
|
- [[services]]
|
||||||
|
updated: 2026-06-18
|
||||||
|
---
|
||||||
# Echo6 Landing Page — Data Export
|
# Echo6 Landing Page — Data Export
|
||||||
## Echo6 Platform Reference — Infrastructure, Services & Brand Identity
|
## Echo6 Platform Reference — Infrastructure, Services & Brand Identity
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,310 +1,326 @@
|
||||||
# ARGUS - OSINT Intelligence Platform
|
---
|
||||||
|
title: ARGUS - OSINT Intelligence Platform
|
||||||
**Status:** Container provisioned, baseline installed, awaiting application deployment
|
type: project
|
||||||
**Last Updated:** 2026-06-14
|
tags:
|
||||||
|
- mesh
|
||||||
---
|
- auth
|
||||||
|
- recon
|
||||||
## Overview
|
aliases: []
|
||||||
|
related:
|
||||||
ARGUS (Automated Reconnaissance & Gathering for Unified Situational-awareness) is an OSINT intelligence gathering platform combining SearXNG with local LLM analysis for automated threat intelligence collection and processing.
|
- [[ip-allocation]]
|
||||||
|
- [[caddy]]
|
||||||
**Architecture:**
|
- [[headscale-onboard-node]]
|
||||||
- Search backend: SearXNG (self-hosted)
|
- [[ct-runbook]]
|
||||||
- Analysis: Local LLM models (no cloud APIs)
|
- [[environment]]
|
||||||
- Scopes: Local, regional, national, global threat levels
|
updated: 2026-06-18
|
||||||
- Privacy-first: No PII collection, focus on events/trends/policies
|
---
|
||||||
|
# ARGUS - OSINT Intelligence Platform
|
||||||
---
|
|
||||||
|
**Status:** Container provisioned, baseline installed, awaiting application deployment
|
||||||
## Container Specifications
|
**Last Updated:** 2026-06-14
|
||||||
|
|
||||||
| Property | Value |
|
---
|
||||||
|----------|-------|
|
|
||||||
| **CTID** | 103 |
|
## Overview
|
||||||
| **Hostname** | argus |
|
|
||||||
| **Host** | utility (192.168.1.241 / 100.64.0.5) |
|
ARGUS (Automated Reconnaissance & Gathering for Unified Situational-awareness) is an OSINT intelligence gathering platform combining SearXNG with local LLM analysis for automated threat intelligence collection and processing.
|
||||||
| **Local IP** | 192.168.1.103 (static) |
|
|
||||||
| **Tailscale IP** | 100.64.0.25 |
|
**Architecture:**
|
||||||
| **Gateway** | 192.168.1.1 |
|
- Search backend: SearXNG (self-hosted)
|
||||||
| **Container Type** | Privileged (unprivileged=0) |
|
- Analysis: Local LLM models (no cloud APIs)
|
||||||
| **Resources** | 4 cores, 8GB RAM, 30GB disk |
|
- Scopes: Local, regional, national, global threat levels
|
||||||
| **Storage** | local-lvm:vm-103-disk-0 |
|
- Privacy-first: No PII collection, focus on events/trends/policies
|
||||||
| **Network** | vmbr0, eth0 |
|
|
||||||
| **Features** | nesting=1 (Docker support) |
|
---
|
||||||
| **Autostart** | Yes (onboot=1) |
|
|
||||||
| **OS** | Ubuntu 24.04 LTS |
|
## Container Specifications
|
||||||
|
|
||||||
**Why privileged:** Required for /dev/net/tun access (Tailscale). Attempted unprivileged initially but tailscaled failed with "CreateTUN failed; /dev/net/tun does not exist".
|
| Property | Value |
|
||||||
|
|----------|-------|
|
||||||
---
|
| **CTID** | 103 |
|
||||||
|
| **Hostname** | argus |
|
||||||
## Installed Software (Baseline)
|
| **Host** | utility (192.168.1.241 / 100.64.0.5) |
|
||||||
|
| **Local IP** | 192.168.1.103 (static) |
|
||||||
- **Docker:** 29.5.3 + docker-compose plugin
|
| **Tailscale IP** | 100.64.0.25 |
|
||||||
- **Tailscale:** 1.98.4 (registered with Headscale at vpn.echo6.co)
|
| **Gateway** | 192.168.1.1 |
|
||||||
- **User:** zvx (uid=1000, groups: sudo, docker)
|
| **Container Type** | Privileged (unprivileged=0) |
|
||||||
- **Common tools:** curl, wget, vim, htop, git, jq, net-tools, dnsutils, sshpass
|
| **Resources** | 4 cores, 8GB RAM, 30GB disk |
|
||||||
- **SSH:** OpenSSH server (password auth enabled)
|
| **Storage** | local-lvm:vm-103-disk-0 |
|
||||||
|
| **Network** | vmbr0, eth0 |
|
||||||
---
|
| **Features** | nesting=1 (Docker support) |
|
||||||
|
| **Autostart** | Yes (onboot=1) |
|
||||||
## Tailscale Configuration
|
| **OS** | Ubuntu 24.04 LTS |
|
||||||
|
|
||||||
**Headscale server:** https://vpn.echo6.co
|
**Why privileged:** Required for /dev/net/tun access (Tailscale). Attempted unprivileged initially but tailscaled failed with "CreateTUN failed; /dev/net/tun does not exist".
|
||||||
**User:** echo6 (user ID 1)
|
|
||||||
**Tailscale IP:** 100.64.0.25
|
---
|
||||||
**Registration:** `tailscale up --login-server=https://vpn.echo6.co --authkey=<key> --ssh --accept-routes`
|
|
||||||
|
## Installed Software (Baseline)
|
||||||
**DNS Bootstrap Fix:**
|
|
||||||
Systemd drop-in at `/etc/systemd/system/tailscaled.service.d/dns-bootstrap.conf` ensures fallback DNS (1.1.1.1, 8.8.8.8) exists before tailscaled starts, preventing chicken-and-egg DNS resolution failures on reboot.
|
- **Docker:** 29.5.3 + docker-compose plugin
|
||||||
|
- **Tailscale:** 1.98.4 (registered with Headscale at vpn.echo6.co)
|
||||||
```bash
|
- **User:** zvx (uid=1000, groups: sudo, docker)
|
||||||
[Service]
|
- **Common tools:** curl, wget, vim, htop, git, jq, net-tools, dnsutils, sshpass
|
||||||
# Ensure fallback DNS exists before tailscaled starts
|
- **SSH:** OpenSSH server (password auth enabled)
|
||||||
# Prevents chicken-and-egg DNS resolution failures on reboot
|
|
||||||
ExecStartPre=/bin/sh -c "echo nameserver 1.1.1.1 > /etc/resolv.conf; echo nameserver 8.8.8.8 >> /etc/resolv.conf"
|
---
|
||||||
```
|
|
||||||
|
## Tailscale Configuration
|
||||||
---
|
|
||||||
|
**Headscale server:** https://vpn.echo6.co
|
||||||
## Network Configuration
|
**User:** echo6 (user ID 1)
|
||||||
|
**Tailscale IP:** 100.64.0.25
|
||||||
**Static IP:** Configured via Proxmox (`pct set 103 -net0 name=eth0,bridge=vmbr0,ip=192.168.1.103/24,gw=192.168.1.1`)
|
**Registration:** `tailscale up --login-server=https://vpn.echo6.co --authkey=<key> --ssh --accept-routes`
|
||||||
|
|
||||||
**Container config** (`/etc/pve/lxc/103.conf`):
|
**DNS Bootstrap Fix:**
|
||||||
```
|
Systemd drop-in at `/etc/systemd/system/tailscaled.service.d/dns-bootstrap.conf` ensures fallback DNS (1.1.1.1, 8.8.8.8) exists before tailscaled starts, preventing chicken-and-egg DNS resolution failures on reboot.
|
||||||
arch: amd64
|
|
||||||
cores: 4
|
```bash
|
||||||
features: nesting=1
|
[Service]
|
||||||
hostname: argus
|
# Ensure fallback DNS exists before tailscaled starts
|
||||||
memory: 8192
|
# Prevents chicken-and-egg DNS resolution failures on reboot
|
||||||
net0: name=eth0,bridge=vmbr0,hwaddr=BC:24:11:EA:8B:21,ip=192.168.1.103/24,gw=192.168.1.1,type=veth
|
ExecStartPre=/bin/sh -c "echo nameserver 1.1.1.1 > /etc/resolv.conf; echo nameserver 8.8.8.8 >> /etc/resolv.conf"
|
||||||
onboot: 1
|
```
|
||||||
ostype: ubuntu
|
|
||||||
rootfs: local-lvm:vm-103-disk-0,size=30G
|
---
|
||||||
swap: 512
|
|
||||||
lxc.cgroup2.devices.allow: c 10:200 rwm
|
## Network Configuration
|
||||||
lxc.mount.entry: /dev/net dev/net none bind,create=dir
|
|
||||||
```
|
**Static IP:** Configured via Proxmox (`pct set 103 -net0 name=eth0,bridge=vmbr0,ip=192.168.1.103/24,gw=192.168.1.1`)
|
||||||
|
|
||||||
**TUN device:** Added manually via `lxc.cgroup2.devices.allow` and `lxc.mount.entry` to support Tailscale in privileged container.
|
**Container config** (`/etc/pve/lxc/103.conf`):
|
||||||
|
```
|
||||||
---
|
arch: amd64
|
||||||
|
cores: 4
|
||||||
## Access Methods
|
features: nesting=1
|
||||||
|
hostname: argus
|
||||||
### SSH Access
|
memory: 8192
|
||||||
|
net0: name=eth0,bridge=vmbr0,hwaddr=BC:24:11:EA:8B:21,ip=192.168.1.103/24,gw=192.168.1.1,type=veth
|
||||||
```bash
|
onboot: 1
|
||||||
# Local network (static IP)
|
ostype: ubuntu
|
||||||
ssh zvx@192.168.1.103
|
rootfs: local-lvm:vm-103-disk-0,size=30G
|
||||||
|
swap: 512
|
||||||
# Tailscale VPN
|
lxc.cgroup2.devices.allow: c 10:200 rwm
|
||||||
ssh zvx@100.64.0.25
|
lxc.mount.entry: /dev/net dev/net none bind,create=dir
|
||||||
ssh zvx@argus
|
```
|
||||||
|
|
||||||
# With password (for sshpass workflows)
|
**TUN device:** Added manually via `lxc.cgroup2.devices.allow` and `lxc.mount.entry` to support Tailscale in privileged container.
|
||||||
sshpass -p '7redditGold' ssh zvx@192.168.1.103
|
|
||||||
```
|
---
|
||||||
|
|
||||||
**Credentials:**
|
## Access Methods
|
||||||
- User: `zvx`
|
|
||||||
- Password: `7redditGold`
|
### SSH Access
|
||||||
- Sudo: Enabled (no password prompt)
|
|
||||||
|
```bash
|
||||||
### From Proxmox Host
|
# Local network (static IP)
|
||||||
|
ssh zvx@192.168.1.103
|
||||||
```bash
|
|
||||||
# Execute commands in container
|
# Tailscale VPN
|
||||||
pct exec 103 -- <command>
|
ssh zvx@100.64.0.25
|
||||||
|
ssh zvx@argus
|
||||||
# Enter container shell
|
|
||||||
pct enter 103
|
# With password (for sshpass workflows)
|
||||||
|
sshpass -p '7redditGold' ssh zvx@192.168.1.103
|
||||||
# Container management
|
```
|
||||||
pct start 103
|
|
||||||
pct stop 103
|
**Credentials:**
|
||||||
pct reboot 103
|
- User: `zvx`
|
||||||
pct status 103
|
- Password: `7redditGold`
|
||||||
```
|
- Sudo: Enabled (no password prompt)
|
||||||
|
|
||||||
---
|
### From Proxmox Host
|
||||||
|
|
||||||
## ARGUS Application Architecture (Planned)
|
```bash
|
||||||
|
# Execute commands in container
|
||||||
### Geographic Scope Hierarchy
|
pct exec 103 -- <command>
|
||||||
|
|
||||||
| Scope | Description | Update Frequency |
|
# Enter container shell
|
||||||
|-------|-------------|------------------|
|
pct enter 103
|
||||||
| LOCAL | Idaho, immediate region | High |
|
|
||||||
| REGIONAL | Pacific Northwest, neighboring states | Medium |
|
# Container management
|
||||||
| NATIONAL | US-wide threats, policy changes | Medium |
|
pct start 103
|
||||||
| GLOBAL | International, geopolitical | Low |
|
pct stop 103
|
||||||
|
pct reboot 103
|
||||||
### Data Storage
|
pct status 103
|
||||||
|
```
|
||||||
- **Raw search results:** `data/raw/{scope}/{date}/`
|
|
||||||
- **Processed intel:** `data/processed/{scope}/`
|
---
|
||||||
- **Alerts:** `data/alerts/`
|
|
||||||
- **Timestamps:** All in UTC
|
## ARGUS Application Architecture (Planned)
|
||||||
|
|
||||||
### Privacy Rules
|
### Geographic Scope Hierarchy
|
||||||
|
|
||||||
- No PII collection on individuals
|
| Scope | Description | Update Frequency |
|
||||||
- Focus on events, trends, policies — not people
|
|-------|-------------|------------------|
|
||||||
- Scrub any inadvertent PII before storage
|
| LOCAL | Idaho, immediate region | High |
|
||||||
- Logs must not contain search queries with personal info
|
| REGIONAL | Pacific Northwest, neighboring states | Medium |
|
||||||
|
| NATIONAL | US-wide threats, policy changes | Medium |
|
||||||
### LLM Analysis
|
| GLOBAL | International, geopolitical | Low |
|
||||||
|
|
||||||
- **Model hosting:** Local only (no cloud APIs)
|
### Data Storage
|
||||||
- **Functions:** Summarization, threat classification, entity extraction, sentiment/threat scoring
|
|
||||||
- **Entities:** Locations, organizations (not individuals)
|
- **Raw search results:** `data/raw/{scope}/{date}/`
|
||||||
|
- **Processed intel:** `data/processed/{scope}/`
|
||||||
---
|
- **Alerts:** `data/alerts/`
|
||||||
|
- **Timestamps:** All in UTC
|
||||||
## Provisioning History
|
|
||||||
|
### Privacy Rules
|
||||||
**2026-06-14 03:00 UTC** - Initial provisioning
|
|
||||||
|
- No PII collection on individuals
|
||||||
1. **First attempt (unprivileged):** Failed - tailscaled couldn't access /dev/net/tun
|
- Focus on events, trends, policies — not people
|
||||||
2. **Second attempt (privileged):** Success
|
- Scrub any inadvertent PII before storage
|
||||||
- Created CT 103 with `--unprivileged 0`
|
- Logs must not contain search queries with personal info
|
||||||
- Installed baseline (apt update/upgrade, common tools, Docker, Tailscale)
|
|
||||||
- DNS fix required post-restart (resolv.conf reset to 100.100.100.100)
|
### LLM Analysis
|
||||||
- Added DNS bootstrap systemd drop-in to prevent future DNS failures
|
|
||||||
- Configured static IP 192.168.1.103 (originally got .142 via DHCP)
|
- **Model hosting:** Local only (no cloud APIs)
|
||||||
- Added TUN device support via lxc.cgroup2 and lxc.mount.entry
|
- **Functions:** Summarization, threat classification, entity extraction, sentiment/threat scoring
|
||||||
|
- **Entities:** Locations, organizations (not individuals)
|
||||||
**Headscale registration:**
|
|
||||||
- Created preauth key via `docker exec headscale headscale preauthkeys create --user 1 --expiration 24h --reusable`
|
---
|
||||||
- Registered successfully after DNS fix
|
|
||||||
- Assigned Tailscale IP: 100.64.0.25
|
## Provisioning History
|
||||||
|
|
||||||
---
|
**2026-06-14 03:00 UTC** - Initial provisioning
|
||||||
|
|
||||||
## Verification Checklist
|
1. **First attempt (unprivileged):** Failed - tailscaled couldn't access /dev/net/tun
|
||||||
|
2. **Second attempt (privileged):** Success
|
||||||
Run inside container to verify baseline:
|
- Created CT 103 with `--unprivileged 0`
|
||||||
|
- Installed baseline (apt update/upgrade, common tools, Docker, Tailscale)
|
||||||
```bash
|
- DNS fix required post-restart (resolv.conf reset to 100.100.100.100)
|
||||||
pct exec 103 -- bash -c '
|
- Added DNS bootstrap systemd drop-in to prevent future DNS failures
|
||||||
echo "=== CT Provisioning Check ==="
|
- Configured static IP 192.168.1.103 (originally got .142 via DHCP)
|
||||||
echo ""
|
- Added TUN device support via lxc.cgroup2 and lxc.mount.entry
|
||||||
echo "Hostname: $(hostname)"
|
|
||||||
echo "User zvx: $(id zvx 2>/dev/null && echo OK || echo MISSING)"
|
**Headscale registration:**
|
||||||
echo "sudo: $(sudo -l -U zvx 2>/dev/null | grep -q ALL && echo OK || echo MISSING)"
|
- Created preauth key via `docker exec headscale headscale preauthkeys create --user 1 --expiration 24h --reusable`
|
||||||
echo "sshpass: $(which sshpass >/dev/null 2>&1 && echo OK || echo MISSING)"
|
- Registered successfully after DNS fix
|
||||||
echo "SSH: $(systemctl is-active ssh)"
|
- Assigned Tailscale IP: 100.64.0.25
|
||||||
echo "Docker: $(docker --version 2>/dev/null || echo MISSING)"
|
|
||||||
echo "Tailscale: $(tailscale status --self 2>/dev/null | head -1 || echo NOT CONNECTED)"
|
---
|
||||||
echo "Tailscale IP: $(tailscale ip -4 2>/dev/null || echo N/A)"
|
|
||||||
echo "Local IP: $(hostname -I | awk \"{print \$1}\")"
|
## Verification Checklist
|
||||||
'
|
|
||||||
```
|
Run inside container to verify baseline:
|
||||||
|
|
||||||
**Expected output:**
|
```bash
|
||||||
```
|
pct exec 103 -- bash -c '
|
||||||
=== CT Provisioning Check ===
|
echo "=== CT Provisioning Check ==="
|
||||||
|
echo ""
|
||||||
Hostname: argus
|
echo "Hostname: $(hostname)"
|
||||||
User zvx: uid=1000(zvx) gid=1000(zvx) groups=1000(zvx),27(sudo),990(docker) OK
|
echo "User zvx: $(id zvx 2>/dev/null && echo OK || echo MISSING)"
|
||||||
sudo: OK
|
echo "sudo: $(sudo -l -U zvx 2>/dev/null | grep -q ALL && echo OK || echo MISSING)"
|
||||||
sshpass: OK
|
echo "sshpass: $(which sshpass >/dev/null 2>&1 && echo OK || echo MISSING)"
|
||||||
SSH: active
|
echo "SSH: $(systemctl is-active ssh)"
|
||||||
Docker: Docker version 29.5.3, build d1c06ef
|
echo "Docker: $(docker --version 2>/dev/null || echo MISSING)"
|
||||||
Tailscale: 100.64.0.25 argus echo6 linux -
|
echo "Tailscale: $(tailscale status --self 2>/dev/null | head -1 || echo NOT CONNECTED)"
|
||||||
Tailscale IP: 100.64.0.25
|
echo "Tailscale IP: $(tailscale ip -4 2>/dev/null || echo N/A)"
|
||||||
Local IP: 192.168.1.103
|
echo "Local IP: $(hostname -I | awk \"{print \$1}\")"
|
||||||
```
|
'
|
||||||
|
```
|
||||||
---
|
|
||||||
|
**Expected output:**
|
||||||
## Known Issues & Resolutions
|
```
|
||||||
|
=== CT Provisioning Check ===
|
||||||
### Issue: DNS resolution fails after container restart
|
|
||||||
|
Hostname: argus
|
||||||
**Symptom:** `resolv.conf` gets reset to invalid nameserver (100.100.100.100), breaking apt and network connectivity.
|
User zvx: uid=1000(zvx) gid=1000(zvx) groups=1000(zvx),27(sudo),990(docker) OK
|
||||||
|
sudo: OK
|
||||||
**Root cause:** LXC containers sometimes reset DNS on boot before networking is fully initialized.
|
sshpass: OK
|
||||||
|
SSH: active
|
||||||
**Resolution:** Installed systemd drop-in (`/etc/systemd/system/tailscaled.service.d/dns-bootstrap.conf`) that sets fallback DNS before tailscaled starts. Prevents chicken-and-egg failure where Tailscale can't resolve vpn.echo6.co because DNS is broken.
|
Docker: Docker version 29.5.3, build d1c06ef
|
||||||
|
Tailscale: 100.64.0.25 argus echo6 linux -
|
||||||
### Issue: Tailscaled fails with "/dev/net/tun does not exist"
|
Tailscale IP: 100.64.0.25
|
||||||
|
Local IP: 192.168.1.103
|
||||||
**Symptom:** Tailscaled crashes on startup with `CreateTUN("tailscale0") failed; /dev/net/tun does not exist`.
|
```
|
||||||
|
|
||||||
**Root cause:** Unprivileged LXC containers don't have access to /dev/net/tun by default.
|
---
|
||||||
|
|
||||||
**Resolution:** Recreated container as privileged (`--unprivileged 0`) and added TUN device to container config:
|
## Known Issues & Resolutions
|
||||||
```
|
|
||||||
lxc.cgroup2.devices.allow: c 10:200 rwm
|
### Issue: DNS resolution fails after container restart
|
||||||
lxc.mount.entry: /dev/net dev/net none bind,create=dir
|
|
||||||
```
|
**Symptom:** `resolv.conf` gets reset to invalid nameserver (100.100.100.100), breaking apt and network connectivity.
|
||||||
|
|
||||||
---
|
**Root cause:** LXC containers sometimes reset DNS on boot before networking is fully initialized.
|
||||||
|
|
||||||
## Next Steps (Application Deployment)
|
**Resolution:** Installed systemd drop-in (`/etc/systemd/system/tailscaled.service.d/dns-bootstrap.conf`) that sets fallback DNS before tailscaled starts. Prevents chicken-and-egg failure where Tailscale can't resolve vpn.echo6.co because DNS is broken.
|
||||||
|
|
||||||
1. **SearXNG deployment:** Docker container for self-hosted search aggregation
|
### Issue: Tailscaled fails with "/dev/net/tun does not exist"
|
||||||
2. **LLM integration:** Local model for analysis (Ollama on cortex or self-hosted)
|
|
||||||
3. **Database:** SQLite for processed intel, possibly Qdrant for vector search (cortex:6333 available)
|
**Symptom:** Tailscaled crashes on startup with `CreateTUN("tailscale0") failed; /dev/net/tun does not exist`.
|
||||||
4. **Scheduler:** Cron or systemd timers for automated collection
|
|
||||||
5. **Web dashboard:** Flask/FastAPI for threat intel visualization
|
**Root cause:** Unprivileged LXC containers don't have access to /dev/net/tun by default.
|
||||||
6. **Alerting:** Integration with Matrix/email for high-priority threats
|
|
||||||
|
**Resolution:** Recreated container as privileged (`--unprivileged 0`) and added TUN device to container config:
|
||||||
---
|
```
|
||||||
|
lxc.cgroup2.devices.allow: c 10:200 rwm
|
||||||
## Operational Notes
|
lxc.mount.entry: /dev/net dev/net none bind,create=dir
|
||||||
|
```
|
||||||
- **Backup strategy:** TBD (Docker volumes + application data)
|
|
||||||
- **Log rotation:** TBD
|
---
|
||||||
- **Monitoring:** TBD (consider adding to WATCHTOWER ops dashboard)
|
|
||||||
- **Updates:** Standard Ubuntu + Docker update procedures
|
## Next Steps (Application Deployment)
|
||||||
- **Resource scaling:** Can adjust cores/RAM via `pct set 103 -cores X -memory Y` (requires container restart)
|
|
||||||
|
1. **SearXNG deployment:** Docker container for self-hosted search aggregation
|
||||||
---
|
2. **LLM integration:** Local model for analysis (Ollama on cortex or self-hosted)
|
||||||
|
3. **Database:** SQLite for processed intel, possibly Qdrant for vector search (cortex:6333 available)
|
||||||
## Related Documentation
|
4. **Scheduler:** Cron or systemd timers for automated collection
|
||||||
|
5. **Web dashboard:** Flask/FastAPI for threat intel visualization
|
||||||
- **CT provisioning:** `/home/zvx/projects/.ref/runbooks/ct-runbook.md`
|
6. **Alerting:** Integration with Matrix/email for high-priority threats
|
||||||
- **ARGUS rules:** `~/.claude/rules/argus.md`
|
|
||||||
- **Environment:** `/home/zvx/projects/.ref/docs/hardware/environment.md`
|
---
|
||||||
- **Services:** `/home/zvx/projects/.ref/docs/services/services.md`
|
|
||||||
- **Headscale:** `/home/zvx/projects/.ref/docs/software/caddy.md` (dnsmasq split DNS)
|
## Operational Notes
|
||||||
|
|
||||||
---
|
- **Backup strategy:** TBD (Docker volumes + application data)
|
||||||
|
- **Log rotation:** TBD
|
||||||
## Quick Command Reference
|
- **Monitoring:** TBD (consider adding to WATCHTOWER ops dashboard)
|
||||||
|
- **Updates:** Standard Ubuntu + Docker update procedures
|
||||||
```bash
|
- **Resource scaling:** Can adjust cores/RAM via `pct set 103 -cores X -memory Y` (requires container restart)
|
||||||
# Container management (from Proxmox host)
|
|
||||||
pct start 103
|
---
|
||||||
pct stop 103
|
|
||||||
pct reboot 103
|
## Related Documentation
|
||||||
pct enter 103
|
|
||||||
|
- **CT provisioning:** `/home/zvx/projects/.ref/runbooks/ct-runbook.md`
|
||||||
# SSH access
|
- **ARGUS rules:** `~/.claude/rules/argus.md`
|
||||||
ssh zvx@192.168.1.103
|
- **Environment:** `/home/zvx/projects/.ref/docs/hardware/environment.md`
|
||||||
ssh zvx@argus # via Tailscale DNS
|
- **Services:** `/home/zvx/projects/.ref/docs/services/services.md`
|
||||||
|
- **Headscale:** `/home/zvx/projects/.ref/docs/software/caddy.md` (dnsmasq split DNS)
|
||||||
# Check Tailscale status
|
|
||||||
pct exec 103 -- tailscale status
|
---
|
||||||
pct exec 103 -- tailscale ip -4
|
|
||||||
|
## Quick Command Reference
|
||||||
# Docker commands (as zvx user)
|
|
||||||
ssh zvx@argus "docker ps"
|
```bash
|
||||||
ssh zvx@argus "docker compose up -d"
|
# Container management (from Proxmox host)
|
||||||
|
pct start 103
|
||||||
# View container config
|
pct stop 103
|
||||||
cat /etc/pve/lxc/103.conf
|
pct reboot 103
|
||||||
|
pct enter 103
|
||||||
# Check resource usage
|
|
||||||
pct status 103 --verbose
|
# SSH access
|
||||||
```
|
ssh zvx@192.168.1.103
|
||||||
|
ssh zvx@argus # via Tailscale DNS
|
||||||
---
|
|
||||||
|
# Check Tailscale status
|
||||||
**Provisioned by:** Claude Code
|
pct exec 103 -- tailscale status
|
||||||
**Container ready for:** ARGUS application deployment
|
pct exec 103 -- tailscale ip -4
|
||||||
|
|
||||||
|
# Docker commands (as zvx user)
|
||||||
|
ssh zvx@argus "docker ps"
|
||||||
|
ssh zvx@argus "docker compose up -d"
|
||||||
|
|
||||||
|
# View container config
|
||||||
|
cat /etc/pve/lxc/103.conf
|
||||||
|
|
||||||
|
# Check resource usage
|
||||||
|
pct status 103 --verbose
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Provisioned by:** Claude Code
|
||||||
|
**Container ready for:** ARGUS application deployment
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,19 @@
|
||||||
|
---
|
||||||
|
title: IdahoMesh Tailnet Runbook
|
||||||
|
type: project
|
||||||
|
tags:
|
||||||
|
- mesh
|
||||||
|
- vpn
|
||||||
|
- auth
|
||||||
|
aliases: []
|
||||||
|
related:
|
||||||
|
- [[idahomesh-vpn-device-setup]]
|
||||||
|
- [[idahomesh-bridge-setup]]
|
||||||
|
- [[meshtastic-sidecar-node]]
|
||||||
|
- [[headscale-onboard-node]]
|
||||||
|
- [[caddy]]
|
||||||
|
updated: 2026-06-18
|
||||||
|
---
|
||||||
# IdahoMesh Tailnet Runbook
|
# IdahoMesh Tailnet Runbook
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,19 @@
|
||||||
|
---
|
||||||
|
title: Add PeerTube Channel
|
||||||
|
type: runbook
|
||||||
|
tags:
|
||||||
|
- media
|
||||||
|
- vpn
|
||||||
|
- auth
|
||||||
|
aliases: []
|
||||||
|
related:
|
||||||
|
- [[peertube-remote-runner]]
|
||||||
|
- [[ct-runbook]]
|
||||||
|
- [[recon-operations]]
|
||||||
|
- [[recon-service-integration]]
|
||||||
|
- [[proxmox-onboard-node]]
|
||||||
|
updated: 2026-06-18
|
||||||
|
---
|
||||||
# Add PeerTube Channel
|
# Add PeerTube Channel
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,19 @@
|
||||||
|
---
|
||||||
|
title: Authentik Access Groups
|
||||||
|
type: runbook
|
||||||
|
tags:
|
||||||
|
- auth
|
||||||
|
- mesh
|
||||||
|
- matrix
|
||||||
|
aliases: []
|
||||||
|
related:
|
||||||
|
- [[authentik-oidc-application]]
|
||||||
|
- [[authentik-create-invitation]]
|
||||||
|
- [[authentik]]
|
||||||
|
- [[deploy-livesync]]
|
||||||
|
- [[proxmox-onboard-node]]
|
||||||
|
updated: 2026-06-18
|
||||||
|
---
|
||||||
# Authentik Access Groups
|
# Authentik Access Groups
|
||||||
|
|
||||||
Manage group-based application access via the Authentik API. No web UI interaction required.
|
Manage group-based application access via the Authentik API. No web UI interaction required.
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,17 @@
|
||||||
|
---
|
||||||
|
title: Proxmox CT/LXC Provisioning Runbook
|
||||||
|
type: runbook
|
||||||
|
tags:
|
||||||
|
- proxmox
|
||||||
|
aliases: []
|
||||||
|
related:
|
||||||
|
- [[headscale-onboard-node]]
|
||||||
|
- [[proxmox-onboard-node]]
|
||||||
|
- [[proxmox-create-ubuntu-vm]]
|
||||||
|
- [[nordvpn-lxc]]
|
||||||
|
- [[meshtasticd-sim-nodes-runbook]]
|
||||||
|
updated: 2026-06-18
|
||||||
|
---
|
||||||
# Proxmox CT/LXC Provisioning Runbook
|
# Proxmox CT/LXC Provisioning Runbook
|
||||||
|
|
||||||
Every container gets the same baseline: local user, Tailscale, SSH, Docker, and common tools. No exceptions.
|
Every container gets the same baseline: local user, Tailscale, SSH, Docker, and common tools. No exceptions.
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,18 @@
|
||||||
|
---
|
||||||
|
title: Meshtastic Sidecar Node — Modular Deployment Runbook
|
||||||
|
type: runbook
|
||||||
|
tags:
|
||||||
|
- mesh
|
||||||
|
- vpn
|
||||||
|
aliases: []
|
||||||
|
related:
|
||||||
|
- [[idahomesh-vpn-device-setup]]
|
||||||
|
- [[headscale-onboard-node]]
|
||||||
|
- [[meshtastic-headscale-runbook]]
|
||||||
|
- [[idahomesh-bridge-setup]]
|
||||||
|
- [[meshtasticd-sim-nodes-runbook]]
|
||||||
|
updated: 2026-06-18
|
||||||
|
---
|
||||||
# Meshtastic Sidecar Node — Modular Deployment Runbook
|
# Meshtastic Sidecar Node — Modular Deployment Runbook
|
||||||
|
|
||||||
Deploy a Raspberry Pi node with a real Meshtastic radio and an operator-selected combination of software modules. Each module is self-contained — install only what the site needs.
|
Deploy a Raspberry Pi node with a real Meshtastic radio and an operator-selected combination of software modules. Each module is self-contained — install only what the site needs.
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue