refactored-recon/AUTH-PUBLIC-FRONTEND.md

AUTH-PUBLIC-FRONTEND.md — Public Navi Frontend with Selective Backend Auth

Status: DRAFT Created: 2026-04-25 Author: Matt + Claude Target: Phase 3 dedicated session Location: recon_refactor repo on cortex

---

1. Principle Statement

Single rule: Auth-lock anything that (a) costs real money to serve or (b) is tied to a specific user account. Everything else is public.

This enables a public-facing navigation frontend at navi.echo6.co while protecting:

• Paid API calls (Google Places, TomTom commercial endpoints)
• Per-user data (contacts, saved locations)
• Administrative functions (key management, service restart)

---

2. Endpoint Inventory

2.1 Main API Routes (api.py)

Route	Method	Classification	Rationale
/	GET	PUBLIC	Dashboard landing page
/search	GET	PUBLIC	Vector search UI (uses local embedding)
/catalogue	GET	PUBLIC	Document catalogue browser
/upload	GET	PUBLIC	Upload page (form only)
/web-ingest	GET	PUBLIC	Web ingest form
/failures	GET	PUBLIC	Failure browser
/deleted-contacts	GET	AUTHED-USER	Per-user deleted contacts page
/nav-i	GET	AUTHED-USER	Nav-I landing (shows user contact count)
/nav-i/api-keys	GET	AUTHED-USER	API key management UI
/peertube	GET	PUBLIC	PeerTube dashboard
/peertube/channels	GET	PUBLIC	Channel listing
/settings/keys	GET	AUTHED-USER	Gemini key management
/settings/cookies	GET	AUTHED-USER	YouTube cookie management
/settings/vpn	GET	AUTHED-USER	VPN control panel
/settings/health	GET	PUBLIC	Service health dashboard
/kiwix	GET	PUBLIC	Kiwix library browser
/kiwix/scraper	GET	PUBLIC	Scraper status
/api/upload	POST	AUTHED-USER	Document upload (disk usage concern)
/api/upload/<hash>/status	GET	PUBLIC	Upload status check
/api/upload/categories	GET	PUBLIC	Category listing
/api/quick-stats	GET	PUBLIC	Cached stats
/api/retry-all	POST	AUTHED-USER	Bulk retry (admin action)
/api/ingest-url	POST	AUTHED-USER	URL ingestion (disk usage)
/api/ingest-urls	POST	AUTHED-USER	Batch URL ingestion
/api/crawl	POST	AUTHED-USER	Site crawl (resource intensive)
/api/crawl/<id>/status	GET	PUBLIC	Crawl status
/api/ingest-peertube	POST	AUTHED-USER	PeerTube ingestion
/api/ingest-peertube/<id>/status	GET	PUBLIC	Ingestion status
/api/peertube/stats	GET	PUBLIC	Stats endpoint
/api/search	POST	PUBLIC	Vector search (local embedding)
/api/status	GET	PUBLIC	Pipeline status
/api/retry/<hash>	POST	AUTHED-USER	Single retry
/api/ingest	POST	AUTHED-USER	Raw intel ingestion
/api/knowledge-stats	GET	PUBLIC	Cached knowledge stats
/api/traffic/flow/<z>/<x>/<y>.png	GET	AUTHED-COST	TomTom traffic tiles (paid API)
/api/place/<type>/<id>	GET	MIXED	See Section 8
/api/landclass	GET	PUBLIC	PAD-US lookup (local DuckDB)
/api/config	GET	PUBLIC	Feature flags
/api/health	GET	PUBLIC	Health check
/api/service/restart	POST	AUTHED-USER	Service restart (admin)
/api/keys	GET	AUTHED-USER	List Gemini keys
/api/keys	POST	AUTHED-USER	Add Gemini key
/api/keys/<idx>	PUT	AUTHED-USER	Replace key
/api/keys/<idx>	DELETE	AUTHED-USER	Remove key
/api/keys/validate	POST	AUTHED-USER	Validate all keys
/api/keys/<idx>/validate	POST	AUTHED-USER	Validate one key
/api/keys/reload	POST	AUTHED-USER	Reload from env
/api/nav-i/api-keys/list	GET	AUTHED-USER	Nav-I key listing
/api/nav-i/api-keys/update	POST	AUTHED-USER	Update key
/api/nav-i/api-keys/test	POST	AUTHED-USER	Test key
/api/nav-i/api-keys/restart-recon	POST	AUTHED-USER	Restart service
/api/cookies/status	GET	AUTHED-USER	Cookie freshness
/api/cookies/upload	POST	AUTHED-USER	Upload cookies
/api/vpn/*	*	AUTHED-USER	VPN control (all 5 routes)
/api/peertube/channels/stats	GET	PUBLIC	Channel stats
/api/peertube/channels	GET	PUBLIC	Channel listing
/api/peertube/channels/add	POST	AUTHED-USER	Add channel
/api/peertube/channels/<name>	DELETE	AUTHED-USER	Remove channel
/api/peertube/dashboard	GET	PUBLIC	Cached dashboard
/api/kiwix/sources	GET	PUBLIC	ZIM source listing
/api/kiwix/toggle-ingest/<id>	POST	AUTHED-USER	Toggle ingest
/api/kiwix/trigger-ingest/<id>	POST	AUTHED-USER	Trigger ingest
/api/kiwix/upload	POST	AUTHED-USER	ZIM upload (disk usage)
/api/kiwix/remove/<id>	POST	AUTHED-USER	Remove ZIM
/api/scraper/submit	POST	AUTHED-USER	Submit scrape job
/api/scraper/jobs	GET	PUBLIC	Job listing
/api/scraper/cancel/<id>	POST	AUTHED-USER	Cancel job
/api/scraper/retry/<id>	POST	AUTHED-USER	Retry job
/api/scraper/delete/<id>	POST	AUTHED-USER	Delete job
/api/scraper/clear-failed	POST	AUTHED-USER	Clear failed jobs
/api/metrics/history	GET	PUBLIC	Metrics history

2.2 Contacts API (contacts_api.py) — ALL AUTHED-USER

All routes use @require_auth decorator. Per-user phone book.

Route	Method	Classification
/api/contacts	GET	AUTHED-USER
/api/contacts	POST	AUTHED-USER
/api/contacts/nearby	GET	AUTHED-USER
/api/contacts/deleted	GET	AUTHED-USER
/api/contacts/<id>	GET	AUTHED-USER
/api/contacts/<id>	PATCH	AUTHED-USER
/api/contacts/<id>	DELETE	AUTHED-USER
/api/contacts/<id>/restore	POST	AUTHED-USER
/api/contacts/<id>/restore-as	POST	AUTHED-USER
/api/contacts/<id>/purge	DELETE	AUTHED-USER

2.3 Address Book API (address_book_api.py) — ALL PUBLIC

Static address lookups from YAML. No auth required.

Route	Method	Classification
/api/address_book/lookup	GET	PUBLIC
/api/address_book/list	GET	PUBLIC

2.4 Geocode API (netsyms_api.py) — ALL PUBLIC

Local Photon geocoding and Netsyms address lookup.

Route	Method	Classification
/api/netsyms/lookup	GET	PUBLIC
/api/netsyms/health	GET	PUBLIC
/api/geocode	GET	PUBLIC
/api/reverse	GET	PUBLIC

2.5 Routes NOT on RECON Backend

The following routes mentioned in the original spec do not exist on RECON:

• /api/photon/* — Photon is accessed via /api/geocode and /api/reverse
• /api/nominatim/* — Nominatim is accessed via /api/place internally
• /api/valhalla/* — Valhalla is called directly by Aurora (cortex), not proxied through RECON
• /api/overture/* — Overture is an enrichment layer inside place_detail.py, not a direct route
• /api/padus/* — PAD-US is accessed via /api/landclass
• /api/aurora/* — Aurora runs on Open WebUI (cortex), not on RECON

---

3. Current Auth Architecture

3.1 Authentik Forward Auth (CT 101 Caddy)

Current state: Unable to read Caddy config from CT 101 (SSH access issue during doc creation).

Expected configuration:

• navi.echo6.co domain likely uses forward_auth directive to Authentik
• All paths currently gated at Caddy level
• Authentik application name: likely "navi" or "recon"

3.2 X-Authentik-Username Header

The backend checks this header in two places:

auth.py:

def get_user_id():
    """Return X-Authentik-Username or None."""
    return request.headers.get('X-Authentik-Username')

def require_auth(f):
    """Decorator: 401 if no Authentik auth header."""
    @wraps(f)
    def wrapper(*args, **kwargs):
        user_id = get_user_id()
        if not user_id:
            return jsonify({'error': 'Authentication required'}), 401
        request.user_id = user_id
        return f(*args, **kwargs)
    return wrapper

Usage:

• contacts_api.py — ALL routes use @require_auth
• api.py — get_user_id() called in /deleted-contacts and /nav-i page handlers, but not enforced (returns "anonymous" if missing)

3.3 Public Exception: recon-watchdog

The monitoring system uses localhost:8420 to bypass Caddy/Authentik entirely when polling health endpoints.

---

4. Proposed Caddy Changes

4.1 Path-Based Forward Auth Split

navi.echo6.co {
    # Static frontend assets — PUBLIC
    handle /assets/* {
        reverse_proxy 192.168.1.130:8420
    }

    # Public API routes — no auth required
    handle_path /api/geocode* {
        reverse_proxy 192.168.1.130:8420
    }
    handle_path /api/reverse* {
        reverse_proxy 192.168.1.130:8420
    }
    handle_path /api/address_book/* {
        reverse_proxy 192.168.1.130:8420
    }
    handle_path /api/netsyms/* {
        reverse_proxy 192.168.1.130:8420
    }
    handle_path /api/place/* {
        reverse_proxy 192.168.1.130:8420
    }
    handle_path /api/landclass* {
        reverse_proxy 192.168.1.130:8420
    }
    handle_path /api/config* {
        reverse_proxy 192.168.1.130:8420
    }
    handle_path /api/health* {
        reverse_proxy 192.168.1.130:8420
    }
    handle_path /api/kiwix/sources* {
        reverse_proxy 192.168.1.130:8420
    }

    # Auth-required routes — forward_auth first
    handle /api/contacts/* {
        forward_auth {
            uri /outpost.goauthentik.io/auth/caddy
            copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Email
        }
        reverse_proxy 192.168.1.130:8420
    }

    handle /api/keys/* {
        forward_auth {
            uri /outpost.goauthentik.io/auth/caddy
            copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Email
        }
        reverse_proxy 192.168.1.130:8420
    }

    handle /api/nav-i/* {
        forward_auth {
            uri /outpost.goauthentik.io/auth/caddy
            copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Email
        }
        reverse_proxy 192.168.1.130:8420
    }

    handle /api/traffic/* {
        forward_auth {
            uri /outpost.goauthentik.io/auth/caddy
            copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Email
        }
        reverse_proxy 192.168.1.130:8420
    }

    # ... additional auth-required paths ...

    # Default: public (frontend pages)
    handle {
        reverse_proxy 192.168.1.130:8420
    }
}

4.2 Alternative: Path Matchers

For cleaner config, use path matchers:

@public_api path /api/geocode* /api/reverse* /api/address_book/* /api/netsyms/* /api/place/* /api/landclass* /api/config* /api/health* /api/kiwix/sources*

@authed_api path /api/contacts/* /api/keys/* /api/nav-i/* /api/traffic/* /api/upload /api/ingest* /api/crawl /api/service/* /api/cookies/* /api/vpn/*

---

5. Proposed RECON Backend Changes

5.1 Routes Needing Explicit Auth Checks

Currently these routes do not enforce auth but should:

Route	Current State	Required Change
/api/upload POST	No check	Add @require_auth
/api/ingest-url POST	No check	Add @require_auth
/api/ingest-urls POST	No check	Add @require_auth
/api/crawl POST	No check	Add @require_auth
/api/retry-all POST	No check	Add @require_auth
/api/retry/<hash> POST	No check	Add @require_auth
/api/ingest POST	No check	Add @require_auth
/api/service/restart POST	No check	Add @require_auth
/api/keys/*	No check	Add @require_auth
/api/nav-i/api-keys/*	No check	Add @require_auth
/api/cookies/*	No check	Add @require_auth
/api/vpn/*	No check	Add @require_auth
/api/peertube/channels/add POST	No check	Add @require_auth
/api/peertube/channels/<name> DELETE	No check	Add @require_auth
/api/kiwix/toggle-ingest/* POST	No check	Add @require_auth
/api/kiwix/trigger-ingest/* POST	No check	Add @require_auth
/api/kiwix/upload POST	No check	Add @require_auth
/api/kiwix/remove/* POST	No check	Add @require_auth
/api/scraper/submit POST	No check	Add @require_auth
/api/scraper/cancel/* POST	No check	Add @require_auth
/api/scraper/retry/* POST	No check	Add @require_auth
/api/scraper/delete/* POST	No check	Add @require_auth
/api/scraper/clear-failed POST	No check	Add @require_auth
/api/traffic/flow/*	No check	Add @require_auth

5.2 Implementation Pattern

For routes that currently assume auth, make explicit:

from .auth import require_auth

@app.route('/api/upload', methods=['POST'])
@require_auth
def api_upload():
    # request.user_id now guaranteed to be set
    ...

5.3 Routes That Can Remain Public

These need no changes — no user state, no cost:

• /api/geocode, /api/reverse (Photon — local)
• /api/address_book/* (YAML lookup — local)
• /api/netsyms/* (SQLite — local)
• /api/place/* (Nominatim/Overpass — local/free, but see Section 8)
• /api/landclass (DuckDB — local)
• /api/config (static config)
• /api/health (health check)
• /api/kiwix/sources (read-only listing)
• /api/search (local embedding + Qdrant)
• /api/knowledge-stats, /api/quick-stats (cached stats)
• /api/peertube/dashboard, /api/peertube/stats, /api/peertube/channels GET (read-only)
• /api/scraper/jobs GET (read-only listing)
• /api/metrics/history (read-only)

---

6. Frontend Auth-Aware UX

6.1 Options Considered

Option	Description	Pros	Cons
A. Inline auth modal	Intercept 401, show login modal, retry	Smooth UX, no context loss	Complex state management
B. Redirect to Authentik	On 401, redirect to /outpost.goauthentik.io/start	Simple, Authentik handles flow	Loses frontend context/state
C. Visually gate features	Hide/disable authed features until logged in	Prevents 401 entirely	Requires knowing auth state upfront

6.2 Recommendation: Option C (Visually Gate) + B (Fallback)

Primary: Check auth state on page load via presence of X-Authentik-Username cookie or a /api/whoami endpoint. Show authed features only when logged in.

Fallback: If a 401 occurs unexpectedly, redirect to Authentik with return URL.

Implementation sketch:

// On app init
async function checkAuth() {
    const resp = await fetch('/api/whoami');  // new endpoint
    if (resp.ok) {
        const { username, groups } = await resp.json();
        window.authState = { authenticated: true, username, groups };
    } else {
        window.authState = { authenticated: false };
    }
    renderAuthAwareUI();
}

New backend endpoint:

@app.route('/api/whoami')
def api_whoami():
    user_id = get_user_id()
    if not user_id:
        return jsonify({'authenticated': False}), 200
    return jsonify({
        'authenticated': True,
        'username': user_id,
        'groups': request.headers.get('X-Authentik-Groups', '').split(','),
    })

6.3 UI Patterns

• Contacts panel: Hidden entirely when unauthenticated
• API keys button: Hidden when unauthenticated
• Traffic layer toggle: Hidden or shows "Login for traffic data"
• Upload button: Shows "Login to upload" when clicked unauthenticated

---

7. Rate Limiting Plan

7.1 Caddy Rate Limit Configuration

Use rate_limit directive for public endpoints:

@public_api path /api/geocode* /api/reverse* /api/place/* ...

handle @public_api {
    rate_limit {
        zone geocode {
            key {remote_host}
            events 60
            window 60s
        }
    }
    reverse_proxy 192.168.1.130:8420
}

7.2 Proposed Thresholds

Endpoint Group	Requests/min	Burst	Rationale
/api/geocode, /api/reverse	60/min	10	Typical map interaction
/api/place/*	30/min	5	Heavy backend processing
/api/landclass	30/min	5	DuckDB query
/api/search	20/min	3	Embedding + vector search
/api/kiwix/sources	10/min	2	Cached, rarely changes
/api/address_book/*	120/min	20	Very cheap lookup

7.3 429 Response Shape

{
    "error": "Rate limit exceeded",
    "retry_after": 45,
    "limit": "60/min"
}

Include Retry-After header.

---

8. Mixed-Route Handling: /api/place

8.1 Current Behavior

/api/place/<osm_type>/<osm_id> performs:

1. Local Nominatim lookup (free)
2. Local Overture enrichment (free)
3. Google Places enrichment (paid) — fills phone, website, hours for business POIs

8.2 Options Considered

Option	Description	Pros	Cons
A. Return partial, flag missing	Return OSM+Overture data, add enrichment_available: "google" flag	Clean public API	Frontend complexity
B. Split routes	/api/place/basic/... and /api/place/enriched/...	Clear API contract	Breaking change
C. Auth-conditional enrichment	Check X-Authentik-Username, skip Google if missing	Backward compatible	Inconsistent response shape

8.3 Recommendation: Option C (Auth-Conditional)

Implementation:

# In place_detail.py

def _enrich_with_google(result, osm_type, osm_id):
    # Check auth header
    from flask import request
    user_id = request.headers.get('X-Authentik-Username')
    if not user_id:
        # Add metadata indicating what is missing
        result.setdefault('sources', {})['google_places'] = {
            'status': 'auth_required',
            'reason': 'Google Places enrichment requires authentication'
        }
        return result

    # ... existing enrichment logic ...

Response shape (unauthenticated):

{
    "osm_type": "N",
    "osm_id": 12345,
    "name": "Boise Coffee House",
    "sources": {
        "primary": "nominatim_local",
        "enrichment": "overture",
        "google_places": {
            "status": "auth_required",
            "reason": "Google Places enrichment requires authentication"
        }
    }
}

Response shape (authenticated):

{
    "osm_type": "N",
    "osm_id": 12345,
    "name": "Boise Coffee House",
    "extratags": {
        "phone": "+1-208-555-0123",
        "opening_hours": "Mo-Fr 06:00-18:00"
    },
    "sources": {
        "primary": "nominatim_local",
        "enrichment": "overture",
        "google_places": {
            "place_id": "ChIJ...",
            "source": "api"
        }
    }
}

---

9. Aurora Handling

9.1 Current Architecture

Aurora runs on Open WebUI (cortex), NOT on the RECON backend. It is accessed via:

• aurora.echo6.co (web interface)
• Mesh bridge integration (future)

Aurora calls RECON APIs (geocode, search) but is not exposed through navi.echo6.co.

9.2 Classification

AUTHED — Aurora is compute-expensive (LLM inference on cortex GPU).

9.3 Future Consideration: Aurora-Lite

Out of scope for this design, but noted for future:

Rate-limited "Aurora-lite" public endpoint with tight token budget per IP. Could expose /api/aurora/ask with:

• 5 requests/hour/IP
• Max 100 input tokens
• Predefined safe prompts only (directions, place info)

---

10. Migration Plan

Phase 1: Backend Auth Enforcement

Risk: Low Rollback: Revert @require_auth decorators

1. Add @require_auth to all identified routes (Section 5.1)
2. Add /api/whoami endpoint
3. Test with existing Caddy config (all routes still gated)
4. Deploy to VM 130
5. Verify Authentik headers flow correctly

Phase 2: Caddy Public Exceptions

Risk: Medium Rollback: Remove path exceptions from Caddy config

1. Update CT 101 Caddyfile with public path exceptions
2. Keep sensitive paths behind forward_auth
3. Test unauthenticated access to public endpoints
4. Verify authenticated endpoints still require login

Phase 3: Frontend Auth-Aware UI

Risk: Low Rollback: N/A (additive change)

1. Implement checkAuth() on frontend
2. Conditionally render authed features
3. Add 401 handling with Authentik redirect
4. Deploy frontend changes

Phase 4: Rate Limiting

Risk: Low Rollback: Remove rate_limit directives

1. Add rate_limit zones to Caddy config
2. Start with high limits, monitor
3. Tune thresholds based on traffic patterns
4. Add monitoring/alerting for 429s

Phase 5: DNS/CDN Consideration

Risk: Low Rollback: DNS change

1. Evaluate Cloudflare for static asset caching
2. Consider geographic distribution for tile proxying
3. Out of scope for this design — separate decision

---

11. Open Questions

Q1: Caddy Configuration Access

Issue: Unable to SSH to CT 101 to read current Caddyfile during doc creation. Action needed: User to provide current navi.echo6.co Caddy block or SSH access path.

Q2: TomTom Traffic Tile Cost Model

Issue: Unknown per-tile cost for traffic API. Decision needed: Is auth + rate limiting sufficient, or should traffic be entirely disabled for public users?

Q3: Knowledge Dashboard Admin Actions

Issue: Several routes like /api/retry-all are admin-only but do not have role checks. Decision needed: Is @require_auth sufficient, or should we add @require_admin that checks X-Authentik-Groups?

Q4: Upload Size Limits

Issue: /api/upload and /api/kiwix/upload accept large files. Decision needed: Should public users (if upload ever becomes public) have stricter size limits than authenticated users?

Q5: Existing Session State

Issue: What happens to in-progress operations (crawls, scraper jobs) when auth is enforced? Decision needed: Grandfather existing jobs or require re-auth?

Q6: Rate Limit Per-User vs Per-IP

Issue: Authenticated users could get higher limits. Decision needed: Implement tiered rate limiting based on auth state?

---

Appendix A: Route Summary by Classification

PUBLIC (28 routes)

• /api/geocode, /api/reverse, /api/address_book/*, /api/netsyms/*
• /api/place/* (basic), /api/landclass, /api/config, /api/health
• /api/kiwix/sources, /api/search, /api/knowledge-stats, /api/quick-stats
• /api/peertube/dashboard, /api/peertube/stats, /api/peertube/channels GET
• /api/scraper/jobs, /api/metrics/history, /api/status
• /api/upload/<hash>/status, /api/upload/categories
• /api/crawl/<id>/status, /api/ingest-peertube/<id>/status
• All page routes except settings/nav-i

AUTHED-USER (35 routes)

• /api/contacts/* (10 routes)
• /api/keys/* (7 routes)
• /api/nav-i/* (4 routes)
• /api/cookies/* (2 routes)
• /api/vpn/* (5 routes)
• /api/upload, /api/ingest-*, /api/crawl POST
• /api/peertube/channels/add, /api/peertube/channels/<name> DELETE
• /api/kiwix/toggle-ingest/*, /api/kiwix/trigger-ingest/*, /api/kiwix/upload, /api/kiwix/remove/*
• /api/scraper/submit, /api/scraper/cancel/*, /api/scraper/retry/*, /api/scraper/delete/*, /api/scraper/clear-failed
• /api/service/restart, /api/retry-all, /api/retry/<hash>, /api/ingest
• Settings pages, Nav-I pages

AUTHED-COST (1 route)

• /api/traffic/flow/* — TomTom paid API

MIXED (1 route)

• /api/place/* — Returns OSM+Overture freely, Google enrichment requires auth

---

Appendix B: Files to Modify

Backend (VM 130)

• /opt/recon/lib/api.py — Add @require_auth to 25+ routes
• /opt/recon/lib/place_detail.py — Auth-conditional Google enrichment
• /opt/recon/lib/auth.py — No changes needed (already has require_auth)

Caddy (CT 101)

• /etc/caddy/Caddyfile — Path-based forward_auth split + rate limiting

Frontend (TBD)

• Auth state detection
• Conditional feature rendering
• 401 handling