commit d0a17a6d2cd6b8334e58056c5c03e184ed9ba21a Author: estorozhenko Date: Sun Sep 6 13:50:57 2026 +0000 Initial commit: Hermes skill hermes-webui-docker diff --git a/SKILL.md b/SKILL.md new file mode 100644 index 0000000..7f32e15 --- /dev/null +++ b/SKILL.md @@ -0,0 +1,217 @@ +--- +name: hermes-webui-docker +description: "Deploy Hermes WebUI in Docker talking to host Hermes agent." +version: 2.0.0 +author: Agent +--- + +# Hermes WebUI in Docker (Hybrid Deployment) + +## When to Use + +When Hermes agent (gateway, CLI, dashboard) runs **natively on the host** but the WebUI runs in a Docker container. This hybrid topology is useful when: +- WebUI needs access to host filesystem paths (e.g. `/opt/hermes/memory-os/`, `/workspace`) +- You want WebUI isolated in a container without moving the entire Hermes stack to Docker +- The gateway is already running as a systemd service on the host + +## Architecture + +``` +Host (port 8642) Docker (port 8787) +┌──────────────────┐ ┌──────────────────────┐ +│ Hermes Gateway │◄──────────────│ Hermes WebUI │ +│ (systemd) │ host.docker │ ghcr.io/nesquena/ │ +│ port 8642 │ .internal: │ hermes-webui:latest │ +│ │ 8642 │ /opt → /opt:rw │ +│ Ollama:11434 │ │ ~/.hermes → /home/ │ +│ SearXNG:11436 │ │ ~/ → /workspace:rw │ +└──────────────────┘ └──────────────────────┘ +``` + +## Docker Compose (WebUI Only) + +```yaml +services: + hermes-webui: + image: ghcr.io/nesquena/hermes-webui:latest + container_name: hermes-webui + ports: + - "8787:8787" + volumes: + # Config, sessions, state — shared with host agent + - /home/estorozhenko/.hermes:/home/hermeswebui/.hermes + # Host filesystem — WebUI can see /opt/hermes/memory-os/, etc. + - /opt:/opt:rw + # Workspace — code browser in WebUI + - /home/estorozhenko:/workspace + environment: + - HERMES_WEBUI_HOST=0.0.0.0 + - HERMES_WEBUI_PORT=8787 + - HERMES_WEBUI_STATE_DIR=/home/hermeswebui/.hermes/webui + # Gateway is on HOST — use host.docker.internal + - HERMES_API_URL=http://host.docker.internal:8642 + - HERMES_WEBUI_CHAT_BACKEND=gateway + - HERMES_WEBUI_GATEWAY_BASE_URL=http://host.docker.internal:8642 + # Runs API needs newer gateway routes (/api/session/stream, /api/session?...) + # that gateway 0.19.0 DOES NOT have (only /api/sessions/{id}/...). On 0.19.0 + # keep this false or WebUI hangs on "Loading conversation...". + - HERMES_WEBUI_GATEWAY_USE_RUNS_API=false + - HERMES_WEBUI_GATEWAY_API_KEY=${API_SERVER_KEY:-} + - WANTED_UID=${UID:-1000} + - WANTED_GID=${GID:-1000} + extra_hosts: + - "host.docker.internal:host-gateway" + restart: unless-stopped +``` + +## Environment Variables — Key Differences from All-in-Docker + +| Variable | All-in-Docker (container name) | Hybrid (host.docker.internal) | +|---|---|---| +| `HERMES_API_URL` | `http://hermes-agent:8642` | `http://host.docker.internal:8642` | +| `HERMES_WEBUI_GATEWAY_BASE_URL` | `http://hermes-agent:8642` | `http://host.docker.internal:8642` | + +The `extra_hosts: host.docker.internal:host-gateway` line is **required** — without it the container cannot resolve `host.docker.internal` on Linux. + +## Host Filesystem Access + +WebUI's file browser and Memory OS tabs see the container's filesystem, not the host's. Mount the host paths the WebUI needs: + +- **`/opt:/opt:rw`** — grants access to `/opt/hermes/memory-os/`, `/opt/hermes/.hermes/`, etc. +- **`/home/estorozhenko:/workspace`** — code browser access to user's home +- **`/home/estorozhenko/.hermes:/home/hermeswebui/.hermes`** — shares sessions, config, skills (required for state persistence) + +Without these mounts, WebUI will show errors like "path /opt/hermes/memory-os/ not found" because it only sees the container's internal filesystem. + +## Host Services that Must Be Running + +- **Hermes Gateway** — port 8642 (systemd: `hermes-gateway.service`) +- **Hermes Dashboard** — port 9119 (systemd: `hermes-dashboard.service`, bind `127.0.0.1`) + - Created via `hermes-dashboard.service` unit file in `/etc/systemd/system/` + - Dashboard **refuses** `--host 0.0.0.0` without auth providers (Jun 2026+ hardening) + - **Workaround:** bind `127.0.0.1` and access via SSH tunnel: `ssh -L 9119:localhost:9119 bigbox` + - Alternative: configure basic auth in `config.yaml` (`dashboard.basic_auth.username` + `password_hash`) +- **Ollama** — port 11434 (if delegation uses local model) +- **SearXNG** — port 11436 (if web search uses local instance) + +All must listen on `0.0.0.0` or at least the Docker bridge interface for the container to reach them. Check with `ss -tlnp | grep -E '8642|11434|11436'`. + +## Gateway API Server Configuration + +**This is the #1 reason WebUI cannot reach the gateway.** The `api_server` is a **gateway platform** (like Telegram or Email) — on host-native setups it must be configured explicitly. + +Detailed reference: `references/host-gateway-config.md` (code walkthrough, troubleshooting table, env overlay logic). + +### How It Gets Enabled + +| Context | Mechanism | +|---|---| +| **All-in-Docker** | Env vars: `API_SERVER_ENABLED=true`, `API_SERVER_KEY=xxx` | +| **Host native** (systemd) | **Must be in `config.yaml`** — env vars are NOT auto-propagated to the systemd unit | + +### Config.yaml Section + +```yaml +platforms: + api_server: + enabled: true + host: 0.0.0.0 + port: 8642 + key: "your-32-char-hex-key-here" + +⚠️ **Field name matters:** The code reads `extra.get("key", ...)` — so the YAML field must be `key`, NOT `api_key`. A field named `api_key` is silently ignored (not bridged into extra). + +Place this at the root level of `config.yaml` (NOT nested under `gateway:`). + +### Why It Fails Silently + +- Gateway starts, shows "⚕ Hermes Gateway Starting...", connects to Email etc. — looks healthy. +- But if `api_server` platform isn't configured, it simply doesn't bind port 8642. +- `systemctl status hermes-gateway.service` shows `active (running)` — misleading. +- The only symptom is `ss -tlnp | grep 8642` shows nothing. + +### Verification + +```bash +systemctl is-active hermes-gateway.service +ss -tlnp | grep 8642 +curl -s http://localhost:8642/health +# Expected: {"status": "ok", "platform": "hermes-agent", "version": "0.19.0"} +``` + +If `ss` shows no listener on 8642 but the service is `active`, the api_server platform isn't configured. + +### systemd Env Var Alternative + +Alternatively, add to the systemd unit file: + +```ini +Environment="API_SERVER_ENABLED=true" +Environment="API_SERVER_HOST=0.0.0.0" +Environment="API_SERVER_KEY=your-key-here" +``` + +Then `sudo systemctl daemon-reload && sudo systemctl restart hermes-gateway.service`. + +## Checking Connectivity + +```bash +# From inside the container: +docker exec hermes-webui curl -s http://host.docker.internal:8642/health + +# From host (should also work): +curl -s http://localhost:8642/health + +# Logs show API calls: +docker logs hermes-webui --tail 20 +``` + +If WebUI shows "Check HERMES_WEBUI_GATEWAY_BASE_URL and Gateway API server health": +1. Verify 8642 listens: `ss -tlnp | grep 8642` +2. If not → check `platforms: api_server:` in config.yaml or env vars in systemd unit +3. Verify key matches between gateway and `HERMES_WEBUI_GATEWAY_API_KEY` +4. `host.docker.internal` must resolve inside container (requires `extra_hosts`) + +## Migrating from All-in-Docker to Hybrid + +When undoing a full Docker migration (all 3 containers + dashboard), do: + +1. **Stop & remove agent + dashboard containers:** + ```bash + docker stop hermes-agent hermes-dashboard + docker rm hermes-agent hermes-dashboard + ``` + +2. **Re-enable host gateway systemd service** (if it was disabled during migration): + ```bash + sudo systemctl enable --now hermes-gateway.service + ``` + +3. **Create dashboard systemd service** (Jun 2026+ requires auth for public bind): + ```bash + # Bind 127.0.0.1 and tunnel in via SSH/WireGuard + ExecStart=/home/estorozhenko/.hermes/hermes-agent/venv/bin/python -m hermes_cli.main dashboard --host 127.0.0.1 + ``` + Full unit template: see `references/dashboard-systemd-service.md`. + +4. **Update docker-compose.yml** — remove hermes-agent + hermes-dashboard services, keep only webui. + +5. **Change environment URLs** from container names to `host.docker.internal`: + - `HERMES_API_URL=http://hermes-agent:8642` → `http://host.docker.internal:8642` + - `HERMES_WEBUI_GATEWAY_BASE_URL=http://hermes-agent:8642` → `http://host.docker.internal:8642` + +6. **Add `/opt:/opt:rw` volume** if WebUI needs host filesystem access. + +7. **Recreate webui container:** + ```bash + docker compose -f /opt/hermes/docker/docker-compose.yml up -d + ``` + +## Pitfalls + +- **Gateway API key mismatch — EASY TO MISS ROOT CAUSE:** The API key exists in **4 sources** (config.yaml `key:` field, `~/.hermes/.env`, systemd unit, docker `.env`). All must match. The most common failure pattern: config.yaml and systemd agree, but `~/.hermes/.env` has a **different** value that overwrites the process env at startup. Always check `grep API_SERVER_KEY ~/.hermes/.env` when debugging 401. +- **YAML field name is `key`, not `api_key`:** The adapter reads `extra.get("key", os.getenv("API_SERVER_KEY", ""))`. A field named `api_key` in config.yaml silently does nothing because the YAML→extra bridge only looks for the exact name `key`. +- **`host.docker.internal` only works with `extra_hosts`** on Linux. Without it, the hostname won't resolve. Alternative: use host's actual IP (e.g. `192.168.x.x` or Docker bridge gateway like `172.17.0.1`) +- **docker stop/rm for containers may be blocked by safety system** — use `docker compose -f down` instead +- **`/opt` on davfs2 (Yandex Disk via WebDAV)** — `stat`, `find`, `rm` are very slow on WebDAV-backed mounts. Consider mounting specific subdirectories instead of the whole `/opt` if the filesystem is partly WebDAV +- **Telegram SOCKS5 tunnel** — if gateway is on host and Telegram is used, ensure `telegram-tunnel.service` is active \ No newline at end of file diff --git a/references/dashboard-systemd-service.md b/references/dashboard-systemd-service.md new file mode 100644 index 0000000..7e0fc9a --- /dev/null +++ b/references/dashboard-systemd-service.md @@ -0,0 +1,60 @@ +# Dashboard Systemd Service — Hermes WebUI Hybrid Deployment + +## Full Unit File Template + +Location: `/etc/systemd/system/hermes-dashboard.service` + +```ini +[Unit] +Description=Hermes Agent Dashboard - Web Management UI +After=network-online.target hermes-gateway.service +Wants=network-online.target +StartLimitIntervalSec=0 + +[Service] +Type=simple +User=estorozhenko +Group=estorozhenko +ExecStart=/home/estorozhenko/.hermes/hermes-agent/venv/bin/python -m hermes_cli.main dashboard --host 127.0.0.1 +WorkingDirectory=/home/estorozhenko/.hermes/hermes-agent +Environment="HOME=/home/estorozhenko" +Environment="USER=estorozhenko" +Environment="LOGNAME=estorozhenko" +Environment="PATH=/home/estorozhenko/.hermes/hermes-agent/venv/bin:/home/estorozhenko/.hermes/hermes-agent/node_modules/.bin:/home/estorozhenko/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" +Environment="VIRTUAL_ENV=/home/estorozhenko/.hermes/hermes-agent/venv" +Environment="HERMES_HOME=/home/estorozhenko/.hermes" +Environment="GATEWAY_HEALTH_URL=http://localhost:8642" +Restart=always +RestartSec=5 +RestartMaxDelaySec=300 +RestartSteps=5 +KillMode=mixed +KillSignal=SIGTERM +TimeoutStopSec=30 +StandardOutput=journal +StandardError=journal + +[Install] +WantedBy=multi-user.target +``` + +## Pitfalls + +- **`--host 0.0.0.0` blocked** — as of June 2026 hardening, Hermes dashboard refuses to bind on a non-loopback interface without a configured auth provider. Error message: + ``` + Refusing to bind dashboard to 0.0.0.0 — the auth gate engages on non-loopback binds, + but no auth providers are registered. + Configure an auth provider before exposing the dashboard: + • Password: set dashboard.basic_auth.username + password_hash in config.yaml + • OAuth: run `hermes dashboard register` (Nous Portal) + There is no unauthenticated public-bind option — to keep it local, bind 127.0.0.1 + and tunnel in (SSH / Tailscale). + ``` + +- **Solution:** bind `127.0.0.1` and access via SSH tunnel: + ```bash + ssh -L 9119:localhost:9119 user@bigbox # from client machine + ``` + Or via WireGuard: if the client IP can reach the host's loopback is impossible — bind on the WireGuard interface IP instead. + +- **No need for `--insecure` flag** — it's deprecated and ignored as of June 2026. \ No newline at end of file diff --git a/references/host-gateway-config.md b/references/host-gateway-config.md new file mode 100644 index 0000000..47260e3 --- /dev/null +++ b/references/host-gateway-config.md @@ -0,0 +1,116 @@ +# Host Gateway API Server Configuration — Hermes WebUI Hybrid + +## The Problem + +WebUI shows `"Check HERMES_WEBUI_GATEWAY_BASE_URL and Gateway API server health"`. +Gateway systemd service shows `active (running)` but port 8642 isn't listening. + +## Root Cause + +`api_server` is a **gateway platform** (enum `Platform.API_SERVER = "api_server"`), not a standalone service. It activates only when configured — either via `config.yaml` or env vars. On a host-native (systemd) setup, env vars from the Docker `.env` file are NOT propagated. + +## Fix — Two Options + +### Option A: config.yaml (Recommended) + +```yaml +# At ROOT level of /home/estorozhenko/.hermes/config.yaml +platforms: + api_server: + enabled: true + host: 0.0.0.0 + port: 8642 + key: "7c8a2d9f4e1b3a5c6d7e8f9a0b1c2d3e4f5a6b7c" +``` + +⚠️ **Field name note:** The field must be `key` (not `api_key`). The adapter reads `extra.get("key", os.getenv("API_SERVER_KEY", ""))`. A field named `api_key` in the YAML is silently ignored because the bridge only moves `key` into the `extra` dict. + +### Option B: systemd unit env vars + +```ini +Environment="API_SERVER_ENABLED=true" +Environment="API_SERVER_HOST=0.0.0.0" +Environment="API_SERVER_KEY=7c8a2d9f4e1b3a5c6d7e8f9a0b1c2d3e4f5a6b7c" +``` + +Then daemon-reload + restart. + +## How the Code Works + +From `gateway/config.py`: + +1. **`_load_env_overrides`** (line ~2128): reads `API_SERVER_ENABLED`, `API_SERVER_KEY`, `API_SERVER_HOST`, `API_SERVER_PORT`, `API_SERVER_CORS_ORIGINS` from env +2. **Platform connected checker** (line ~846): requires `key` in `extra` with `min_length=16` +3. **YAML bridging** (line ~1471): moves `port`, `key`, `host`, `cors_origins`, `model_name` from the platform dict into `extra` so PlatformConfig preserves them +4. **`_has_usable_api_server_key`** (line ~822): checks key is a string ≥ 16 chars + +Without a valid key (≥16 chars), the platform won't be marked connected and the adapter won't start — no listener on 8642, no error message. + +## Key Debugging Lesson: Multi-Source Divergence + +The API server key lives in **four independent sources** that must all agree: + +| Source | Where | How it's set | +|--------|-------|-------------| +| `config.yaml` | `platforms.api_server.key` | Edited directly | +| `~/.hermes/.env` | `API_SERVER_KEY=...` | `hermes config set gateway.api_server_key ...` or manual edit | +| systemd unit | `Environment="API_SERVER_KEY=..."` | `hermes gateway install --force` or manual edit | +| docker `.env` for webui | `API_SERVER_KEY=...` | Docker compose env-file | + +**Common failure modes:** + +- **config.yaml says one thing, `.env` says another** → `.env` wins (`os.environ` is overwritten at startup), so even though systemd's `Environment=` and config.yaml agree, the process env is different. This is the most frequent root cause of "401 with correct key". +- **systemd shows key A, `.env` shows key B** → process starts with systemd's env, but then load_gateway_config reads `.env` and calls `os.environ["API_SERVER_KEY"] = B`, overwriting A. + +**To fix:** pick ONE canonical value and set it identically in all four sources. The simplest approach: pick a single key string (e.g. 40 hex chars), put it in config.yaml's `key:` field, and ensure `~/.hermes/.env`, the systemd unit, and docker `.env` all match. + +### How to Verify All Sources + +```bash +# 1. Config YAML +grep -A1 'key:' ~/.hermes/config.yaml | grep -v extra + +# 2. User .env (read_file blocked — use terminal/xxd) +grep API_SERVER_KEY ~/.hermes/.env + +# 3. Systemd service +grep API_SERVER_KEY /etc/systemd/system/hermes-gateway.service + +# 4. Docker compose .env +grep API_SERVER_KEY /opt/hermes/docker/.env + +# 5. Live process env (verify the actual value the gateway uses) +SYSTEMD_PID=$(systemctl show hermes-gateway.service -p MainPID | grep -oP '\d+') +cat /proc/$SYSTEMD_PID/environ | tr '\0' '\n' | grep API_SERVER_KEY +``` + +All five should show the identical value. If step 5 (live process) differs from 1-4, then something is overwriting the key at runtime — likely step 2. + +## Verification + +```bash +# Confirm listener +ss -tlnp | grep 8642 + +# Health check +curl -s http://localhost:8642/health +# → {"status": "ok", "platform": "hermes-agent", "version": "0.19.0"} + +# From inside webui container +docker exec hermes-webui curl -s http://host.docker.internal:8642/health + +# Check gateway logs for platform startup +journalctl -u hermes-gateway.service --no-pager -n 10 | grep -i api +``` + +## Env Vars Expected by WebUI + +In `docker-compose.yml` the webui expects: + +```yaml +- HERMES_WEBUI_GATEWAY_API_KEY=${API_SERVER_KEY:-} +- HERMES_API_URL=http://host.docker.internal:8642 +- HERMES_WEBUI_GATEWAY_BASE_URL=http://host.docker.internal:8642 +``` + +The `API_SERVER_KEY` env var must be exported or set in the `.env` file that docker-compose reads. \ No newline at end of file