Initial commit: Hermes skill hermes-webui-docker

2026-09-29 09:15:10 +00:00 · 2026-09-06 13:50:57 +00:00
commit d0a17a6d2c
3 changed files with 393 additions and 0 deletions
@@ -0,0 +1,217 @@
+---
+name: hermes-webui-docker
+description: "Deploy Hermes WebUI in Docker talking to host Hermes agent."
+version: 2.0.0
+author: Agent
+---
+
+# Hermes WebUI in Docker (Hybrid Deployment)
+
+## When to Use
+
+When Hermes agent (gateway, CLI, dashboard) runs **natively on the host** but the WebUI runs in a Docker container. This hybrid topology is useful when:
+- WebUI needs access to host filesystem paths (e.g. `/opt/hermes/memory-os/`, `/workspace`)
+- You want WebUI isolated in a container without moving the entire Hermes stack to Docker
+- The gateway is already running as a systemd service on the host
+
+## Architecture
+
+```
+Host (port 8642)                    Docker (port 8787)
+┌──────────────────┐               ┌──────────────────────┐
+│  Hermes Gateway   │◄──────────────│    Hermes WebUI      │
+│  (systemd)        │  host.docker  │  ghcr.io/nesquena/   │
+│  port 8642        │  .internal:   │  hermes-webui:latest │
+│                   │  8642         │  /opt → /opt:rw     │
+│  Ollama:11434     │               │  ~/.hermes → /home/  │
+│  SearXNG:11436    │               │  ~/ → /workspace:rw │
+└──────────────────┘               └──────────────────────┘
+```
+
+## Docker Compose (WebUI Only)
+
+```yaml
+services:
+  hermes-webui:
+    image: ghcr.io/nesquena/hermes-webui:latest
+    container_name: hermes-webui
+    ports:
+      - "8787:8787"
+    volumes:
+      # Config, sessions, state — shared with host agent
+      - /home/estorozhenko/.hermes:/home/hermeswebui/.hermes
+      # Host filesystem — WebUI can see /opt/hermes/memory-os/, etc.
+      - /opt:/opt:rw
+      # Workspace — code browser in WebUI
+      - /home/estorozhenko:/workspace
+    environment:
+      - HERMES_WEBUI_HOST=0.0.0.0
+      - HERMES_WEBUI_PORT=8787
+      - HERMES_WEBUI_STATE_DIR=/home/hermeswebui/.hermes/webui
+      # Gateway is on HOST — use host.docker.internal
+      - HERMES_API_URL=http://host.docker.internal:8642
+      - HERMES_WEBUI_CHAT_BACKEND=gateway
+      - HERMES_WEBUI_GATEWAY_BASE_URL=http://host.docker.internal:8642
+      # Runs API needs newer gateway routes (/api/session/stream, /api/session?...)
+      # that gateway 0.19.0 DOES NOT have (only /api/sessions/{id}/...). On 0.19.0
+      # keep this false or WebUI hangs on "Loading conversation...".
+      - HERMES_WEBUI_GATEWAY_USE_RUNS_API=false
+      - HERMES_WEBUI_GATEWAY_API_KEY=${API_SERVER_KEY:-}
+      - WANTED_UID=${UID:-1000}
+      - WANTED_GID=${GID:-1000}
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    restart: unless-stopped
+```
+
+## Environment Variables — Key Differences from All-in-Docker
+
+| Variable | All-in-Docker (container name) | Hybrid (host.docker.internal) |
+|---|---|---|
+| `HERMES_API_URL` | `http://hermes-agent:8642` | `http://host.docker.internal:8642` |
+| `HERMES_WEBUI_GATEWAY_BASE_URL` | `http://hermes-agent:8642` | `http://host.docker.internal:8642` |
+
+The `extra_hosts: host.docker.internal:host-gateway` line is **required** — without it the container cannot resolve `host.docker.internal` on Linux.
+
+## Host Filesystem Access
+
+WebUI's file browser and Memory OS tabs see the container's filesystem, not the host's. Mount the host paths the WebUI needs:
+
+- **`/opt:/opt:rw`** — grants access to `/opt/hermes/memory-os/`, `/opt/hermes/.hermes/`, etc.
+- **`/home/estorozhenko:/workspace`** — code browser access to user's home
+- **`/home/estorozhenko/.hermes:/home/hermeswebui/.hermes`** — shares sessions, config, skills (required for state persistence)
+
+Without these mounts, WebUI will show errors like "path /opt/hermes/memory-os/ not found" because it only sees the container's internal filesystem.
+
+## Host Services that Must Be Running
+
+- **Hermes Gateway** — port 8642 (systemd: `hermes-gateway.service`)
+- **Hermes Dashboard** — port 9119 (systemd: `hermes-dashboard.service`, bind `127.0.0.1`)
+  - Created via `hermes-dashboard.service` unit file in `/etc/systemd/system/`
+  - Dashboard **refuses** `--host 0.0.0.0` without auth providers (Jun 2026+ hardening)
+  - **Workaround:** bind `127.0.0.1` and access via SSH tunnel: `ssh -L 9119:localhost:9119 bigbox`
+  - Alternative: configure basic auth in `config.yaml` (`dashboard.basic_auth.username` + `password_hash`)
+- **Ollama** — port 11434 (if delegation uses local model)
+- **SearXNG** — port 11436 (if web search uses local instance)
+
+All must listen on `0.0.0.0` or at least the Docker bridge interface for the container to reach them. Check with `ss -tlnp | grep -E '8642|11434|11436'`.
+
+## Gateway API Server Configuration
+
+**This is the #1 reason WebUI cannot reach the gateway.** The `api_server` is a **gateway platform** (like Telegram or Email) — on host-native setups it must be configured explicitly.
+
+Detailed reference: `references/host-gateway-config.md` (code walkthrough, troubleshooting table, env overlay logic).
+
+### How It Gets Enabled
+
+| Context | Mechanism |
+|---|---|
+| **All-in-Docker** | Env vars: `API_SERVER_ENABLED=true`, `API_SERVER_KEY=xxx` |
+| **Host native** (systemd) | **Must be in `config.yaml`** — env vars are NOT auto-propagated to the systemd unit |
+
+### Config.yaml Section
+
+```yaml
+platforms:
+  api_server:
+    enabled: true
+    host: 0.0.0.0
+    port: 8642
+    key: "your-32-char-hex-key-here"
+
+⚠️ **Field name matters:** The code reads `extra.get("key", ...)` — so the YAML field must be `key`, NOT `api_key`. A field named `api_key` is silently ignored (not bridged into extra).
+
+Place this at the root level of `config.yaml` (NOT nested under `gateway:`).
+
+### Why It Fails Silently
+
+- Gateway starts, shows "⚕ Hermes Gateway Starting...", connects to Email etc. — looks healthy.
+- But if `api_server` platform isn't configured, it simply doesn't bind port 8642.
+- `systemctl status hermes-gateway.service` shows `active (running)` — misleading.
+- The only symptom is `ss -tlnp | grep 8642` shows nothing.
+
+### Verification
+
+```bash
+systemctl is-active hermes-gateway.service
+ss -tlnp | grep 8642
+curl -s http://localhost:8642/health
+# Expected: {"status": "ok", "platform": "hermes-agent", "version": "0.19.0"}
+```
+
+If `ss` shows no listener on 8642 but the service is `active`, the api_server platform isn't configured.
+
+### systemd Env Var Alternative
+
+Alternatively, add to the systemd unit file:
+
+```ini
+Environment="API_SERVER_ENABLED=true"
+Environment="API_SERVER_HOST=0.0.0.0"
+Environment="API_SERVER_KEY=your-key-here"
+```
+
+Then `sudo systemctl daemon-reload && sudo systemctl restart hermes-gateway.service`.
+
+## Checking Connectivity
+
+```bash
+# From inside the container:
+docker exec hermes-webui curl -s http://host.docker.internal:8642/health
+
+# From host (should also work):
+curl -s http://localhost:8642/health
+
+# Logs show API calls:
+docker logs hermes-webui --tail 20
+```
+
+If WebUI shows "Check HERMES_WEBUI_GATEWAY_BASE_URL and Gateway API server health":
+1. Verify 8642 listens: `ss -tlnp | grep 8642`
+2. If not → check `platforms: api_server:` in config.yaml or env vars in systemd unit
+3. Verify key matches between gateway and `HERMES_WEBUI_GATEWAY_API_KEY`
+4. `host.docker.internal` must resolve inside container (requires `extra_hosts`)
+
+## Migrating from All-in-Docker to Hybrid
+
+When undoing a full Docker migration (all 3 containers + dashboard), do:
+
+1. **Stop & remove agent + dashboard containers:**
+   ```bash
+   docker stop hermes-agent hermes-dashboard
+   docker rm hermes-agent hermes-dashboard
+   ```
+
+2. **Re-enable host gateway systemd service** (if it was disabled during migration):
+   ```bash
+   sudo systemctl enable --now hermes-gateway.service
+   ```
+
+3. **Create dashboard systemd service** (Jun 2026+ requires auth for public bind):
+   ```bash
+   # Bind 127.0.0.1 and tunnel in via SSH/WireGuard
+   ExecStart=/home/estorozhenko/.hermes/hermes-agent/venv/bin/python -m hermes_cli.main dashboard --host 127.0.0.1
+   ```
+   Full unit template: see `references/dashboard-systemd-service.md`.
+
+4. **Update docker-compose.yml** — remove hermes-agent + hermes-dashboard services, keep only webui.
+
+5. **Change environment URLs** from container names to `host.docker.internal`:
+   - `HERMES_API_URL=http://hermes-agent:8642` → `http://host.docker.internal:8642`
+   - `HERMES_WEBUI_GATEWAY_BASE_URL=http://hermes-agent:8642` → `http://host.docker.internal:8642`
+
+6. **Add `/opt:/opt:rw` volume** if WebUI needs host filesystem access.
+
+7. **Recreate webui container:**
+   ```bash
+   docker compose -f /opt/hermes/docker/docker-compose.yml up -d
+   ```
+
+## Pitfalls
+
+- **Gateway API key mismatch — EASY TO MISS ROOT CAUSE:** The API key exists in **4 sources** (config.yaml `key:` field, `~/.hermes/.env`, systemd unit, docker `.env`). All must match. The most common failure pattern: config.yaml and systemd agree, but `~/.hermes/.env` has a **different** value that overwrites the process env at startup. Always check `grep API_SERVER_KEY ~/.hermes/.env` when debugging 401.
+- **YAML field name is `key`, not `api_key`:** The adapter reads `extra.get("key", os.getenv("API_SERVER_KEY", ""))`. A field named `api_key` in config.yaml silently does nothing because the YAML→extra bridge only looks for the exact name `key`.
+- **`host.docker.internal` only works with `extra_hosts`** on Linux. Without it, the hostname won't resolve. Alternative: use host's actual IP (e.g. `192.168.x.x` or Docker bridge gateway like `172.17.0.1`)
+- **docker stop/rm for containers may be blocked by safety system** — use `docker compose -f <file> down` instead
+- **`/opt` on davfs2 (Yandex Disk via WebDAV)** — `stat`, `find`, `rm` are very slow on WebDAV-backed mounts. Consider mounting specific subdirectories instead of the whole `/opt` if the filesystem is partly WebDAV
+- **Telegram SOCKS5 tunnel** — if gateway is on host and Telegram is used, ensure `telegram-tunnel.service` is active
@@ -0,0 +1,60 @@
+# Dashboard Systemd Service — Hermes WebUI Hybrid Deployment
+
+## Full Unit File Template
+
+Location: `/etc/systemd/system/hermes-dashboard.service`
+
+```ini
+[Unit]
+Description=Hermes Agent Dashboard - Web Management UI
+After=network-online.target hermes-gateway.service
+Wants=network-online.target
+StartLimitIntervalSec=0
+
+[Service]
+Type=simple
+User=estorozhenko
+Group=estorozhenko
+ExecStart=/home/estorozhenko/.hermes/hermes-agent/venv/bin/python -m hermes_cli.main dashboard --host 127.0.0.1
+WorkingDirectory=/home/estorozhenko/.hermes/hermes-agent
+Environment="HOME=/home/estorozhenko"
+Environment="USER=estorozhenko"
+Environment="LOGNAME=estorozhenko"
+Environment="PATH=/home/estorozhenko/.hermes/hermes-agent/venv/bin:/home/estorozhenko/.hermes/hermes-agent/node_modules/.bin:/home/estorozhenko/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
+Environment="VIRTUAL_ENV=/home/estorozhenko/.hermes/hermes-agent/venv"
+Environment="HERMES_HOME=/home/estorozhenko/.hermes"
+Environment="GATEWAY_HEALTH_URL=http://localhost:8642"
+Restart=always
+RestartSec=5
+RestartMaxDelaySec=300
+RestartSteps=5
+KillMode=mixed
+KillSignal=SIGTERM
+TimeoutStopSec=30
+StandardOutput=journal
+StandardError=journal
+
+[Install]
+WantedBy=multi-user.target
+```
+
+## Pitfalls
+
+- **`--host 0.0.0.0` blocked** — as of June 2026 hardening, Hermes dashboard refuses to bind on a non-loopback interface without a configured auth provider. Error message:
+  ```
+  Refusing to bind dashboard to 0.0.0.0 — the auth gate engages on non-loopback binds,
+  but no auth providers are registered.
+  Configure an auth provider before exposing the dashboard:
+    • Password: set dashboard.basic_auth.username + password_hash in config.yaml
+    • OAuth: run `hermes dashboard register` (Nous Portal)
+  There is no unauthenticated public-bind option — to keep it local, bind 127.0.0.1
+  and tunnel in (SSH / Tailscale).
+  ```
+
+- **Solution:** bind `127.0.0.1` and access via SSH tunnel:
+  ```bash
+  ssh -L 9119:localhost:9119 user@bigbox  # from client machine
+  ```
+  Or via WireGuard: if the client IP can reach the host's loopback is impossible — bind on the WireGuard interface IP instead.
+
+- **No need for `--insecure` flag** — it's deprecated and ignored as of June 2026.
@@ -0,0 +1,116 @@
+# Host Gateway API Server Configuration — Hermes WebUI Hybrid
+
+## The Problem
+
+WebUI shows `"Check HERMES_WEBUI_GATEWAY_BASE_URL and Gateway API server health"`.
+Gateway systemd service shows `active (running)` but port 8642 isn't listening.
+
+## Root Cause
+
+`api_server` is a **gateway platform** (enum `Platform.API_SERVER = "api_server"`), not a standalone service. It activates only when configured — either via `config.yaml` or env vars. On a host-native (systemd) setup, env vars from the Docker `.env` file are NOT propagated.
+
+## Fix — Two Options
+
+### Option A: config.yaml (Recommended)
+
+```yaml
+# At ROOT level of /home/estorozhenko/.hermes/config.yaml
+platforms:
+  api_server:
+    enabled: true
+    host: 0.0.0.0
+    port: 8642
+    key: "7c8a2d9f4e1b3a5c6d7e8f9a0b1c2d3e4f5a6b7c"
+```
+
+⚠️ **Field name note:** The field must be `key` (not `api_key`). The adapter reads `extra.get("key", os.getenv("API_SERVER_KEY", ""))`. A field named `api_key` in the YAML is silently ignored because the bridge only moves `key` into the `extra` dict.
+
+### Option B: systemd unit env vars
+
+```ini
+Environment="API_SERVER_ENABLED=true"
+Environment="API_SERVER_HOST=0.0.0.0"
+Environment="API_SERVER_KEY=7c8a2d9f4e1b3a5c6d7e8f9a0b1c2d3e4f5a6b7c"
+```
+
+Then daemon-reload + restart.
+
+## How the Code Works
+
+From `gateway/config.py`:
+
+1. **`_load_env_overrides`** (line ~2128): reads `API_SERVER_ENABLED`, `API_SERVER_KEY`, `API_SERVER_HOST`, `API_SERVER_PORT`, `API_SERVER_CORS_ORIGINS` from env
+2. **Platform connected checker** (line ~846): requires `key` in `extra` with `min_length=16`
+3. **YAML bridging** (line ~1471): moves `port`, `key`, `host`, `cors_origins`, `model_name` from the platform dict into `extra` so PlatformConfig preserves them
+4. **`_has_usable_api_server_key`** (line ~822): checks key is a string ≥ 16 chars
+
+Without a valid key (≥16 chars), the platform won't be marked connected and the adapter won't start — no listener on 8642, no error message.
+
+## Key Debugging Lesson: Multi-Source Divergence
+
+The API server key lives in **four independent sources** that must all agree:
+
+| Source | Where | How it's set |
+|--------|-------|-------------|
+| `config.yaml` | `platforms.api_server.key` | Edited directly |
+| `~/.hermes/.env` | `API_SERVER_KEY=...` | `hermes config set gateway.api_server_key ...` or manual edit |
+| systemd unit | `Environment="API_SERVER_KEY=..."` | `hermes gateway install --force` or manual edit |
+| docker `.env` for webui | `API_SERVER_KEY=...` | Docker compose env-file |
+
+**Common failure modes:**
+
+- **config.yaml says one thing, `.env` says another** → `.env` wins (`os.environ` is overwritten at startup), so even though systemd's `Environment=` and config.yaml agree, the process env is different. This is the most frequent root cause of "401 with correct key".
+- **systemd shows key A, `.env` shows key B** → process starts with systemd's env, but then load_gateway_config reads `.env` and calls `os.environ["API_SERVER_KEY"] = B`, overwriting A.
+
+**To fix:** pick ONE canonical value and set it identically in all four sources. The simplest approach: pick a single key string (e.g. 40 hex chars), put it in config.yaml's `key:` field, and ensure `~/.hermes/.env`, the systemd unit, and docker `.env` all match.
+
+### How to Verify All Sources
+
+```bash
+# 1. Config YAML
+grep -A1 'key:' ~/.hermes/config.yaml | grep -v extra
+
+# 2. User .env (read_file blocked — use terminal/xxd)
+grep API_SERVER_KEY ~/.hermes/.env
+
+# 3. Systemd service
+grep API_SERVER_KEY /etc/systemd/system/hermes-gateway.service
+
+# 4. Docker compose .env
+grep API_SERVER_KEY /opt/hermes/docker/.env
+
+# 5. Live process env (verify the actual value the gateway uses)
+SYSTEMD_PID=$(systemctl show hermes-gateway.service -p MainPID | grep -oP '\d+')
+cat /proc/$SYSTEMD_PID/environ | tr '\0' '\n' | grep API_SERVER_KEY
+```
+
+All five should show the identical value. If step 5 (live process) differs from 1-4, then something is overwriting the key at runtime — likely step 2.
+
+## Verification
+
+```bash
+# Confirm listener
+ss -tlnp | grep 8642
+
+# Health check
+curl -s http://localhost:8642/health
+# → {"status": "ok", "platform": "hermes-agent", "version": "0.19.0"}
+
+# From inside webui container
+docker exec hermes-webui curl -s http://host.docker.internal:8642/health
+
+# Check gateway logs for platform startup
+journalctl -u hermes-gateway.service --no-pager -n 10 | grep -i api
+```
+
+## Env Vars Expected by WebUI
+
+In `docker-compose.yml` the webui expects:
+
+```yaml
+- HERMES_WEBUI_GATEWAY_API_KEY=${API_SERVER_KEY:-}
+- HERMES_API_URL=http://host.docker.internal:8642
+- HERMES_WEBUI_GATEWAY_BASE_URL=http://host.docker.internal:8642
+```
+
+The `API_SERVER_KEY` env var must be exported or set in the `.env` file that docker-compose reads.