Initial commit: Hermes skill hermes-webui-docker

2026-09-29 09:15:10 +00:00 · 2026-09-06 13:50:57 +00:00
commit d0a17a6d2c
3 changed files with 393 additions and 0 deletions
@@ -0,0 +1,217 @@
 ---
 name: hermes-webui-docker
 description: "Deploy Hermes WebUI in Docker talking to host Hermes agent."
 version: 2.0.0
 author: Agent
 ---
 # Hermes WebUI in Docker (Hybrid Deployment)
 ## When to Use
 When Hermes agent (gateway, CLI, dashboard) runs **natively on the host** but the WebUI runs in a Docker container. This hybrid topology is useful when:
 - WebUI needs access to host filesystem paths (e.g. `/opt/hermes/memory-os/`, `/workspace`)
 - You want WebUI isolated in a container without moving the entire Hermes stack to Docker
 - The gateway is already running as a systemd service on the host
 ## Architecture
 ```
 Host (port 8642)                    Docker (port 8787)
 ┌──────────────────┐               ┌──────────────────────┐
 │  Hermes Gateway   │◄──────────────│    Hermes WebUI      │
 │  (systemd)        │  host.docker  │  ghcr.io/nesquena/   │
 │  port 8642        │  .internal:   │  hermes-webui:latest │
 │                   │  8642         │  /opt → /opt:rw     │
 │  Ollama:11434     │               │  ~/.hermes → /home/  │
 │  SearXNG:11436    │               │  ~/ → /workspace:rw │
 └──────────────────┘               └──────────────────────┘
 ```
 ## Docker Compose (WebUI Only)
 ```yaml
 services:
  hermes-webui:
    image: ghcr.io/nesquena/hermes-webui:latest
    container_name: hermes-webui
    ports:
      - "8787:8787"
    volumes:
      # Config, sessions, state — shared with host agent
      - /home/estorozhenko/.hermes:/home/hermeswebui/.hermes
      # Host filesystem — WebUI can see /opt/hermes/memory-os/, etc.
      - /opt:/opt:rw
      # Workspace — code browser in WebUI
      - /home/estorozhenko:/workspace
    environment:
      - HERMES_WEBUI_HOST=0.0.0.0
      - HERMES_WEBUI_PORT=8787
      - HERMES_WEBUI_STATE_DIR=/home/hermeswebui/.hermes/webui
      # Gateway is on HOST — use host.docker.internal
      - HERMES_API_URL=http://host.docker.internal:8642
      - HERMES_WEBUI_CHAT_BACKEND=gateway
      - HERMES_WEBUI_GATEWAY_BASE_URL=http://host.docker.internal:8642
      # Runs API needs newer gateway routes (/api/session/stream, /api/session?...)
      # that gateway 0.19.0 DOES NOT have (only /api/sessions/{id}/...). On 0.19.0
      # keep this false or WebUI hangs on "Loading conversation...".
      - HERMES_WEBUI_GATEWAY_USE_RUNS_API=false
      - HERMES_WEBUI_GATEWAY_API_KEY=${API_SERVER_KEY:-}
      - WANTED_UID=${UID:-1000}
      - WANTED_GID=${GID:-1000}
    extra_hosts:
      - "host.docker.internal:host-gateway"
    restart: unless-stopped
 ```
 ## Environment Variables — Key Differences from All-in-Docker
 | Variable | All-in-Docker (container name) | Hybrid (host.docker.internal) |
 |---|---|---|
 | `HERMES_API_URL` | `http://hermes-agent:8642` | `http://host.docker.internal:8642` |
 | `HERMES_WEBUI_GATEWAY_BASE_URL` | `http://hermes-agent:8642` | `http://host.docker.internal:8642` |
 The `extra_hosts: host.docker.internal:host-gateway` line is **required** — without it the container cannot resolve `host.docker.internal` on Linux.
 ## Host Filesystem Access
 WebUI's file browser and Memory OS tabs see the container's filesystem, not the host's. Mount the host paths the WebUI needs:
 - **`/opt:/opt:rw`** — grants access to `/opt/hermes/memory-os/`, `/opt/hermes/.hermes/`, etc.
 - **`/home/estorozhenko:/workspace`** — code browser access to user's home
 - **`/home/estorozhenko/.hermes:/home/hermeswebui/.hermes`** — shares sessions, config, skills (required for state persistence)
 Without these mounts, WebUI will show errors like "path /opt/hermes/memory-os/ not found" because it only sees the container's internal filesystem.
 ## Host Services that Must Be Running
 - **Hermes Gateway** — port 8642 (systemd: `hermes-gateway.service`)
 - **Hermes Dashboard** — port 9119 (systemd: `hermes-dashboard.service`, bind `127.0.0.1`)
  - Created via `hermes-dashboard.service` unit file in `/etc/systemd/system/`
  - Dashboard **refuses** `--host 0.0.0.0` without auth providers (Jun 2026+ hardening)
  - **Workaround:** bind `127.0.0.1` and access via SSH tunnel: `ssh -L 9119:localhost:9119 bigbox`
  - Alternative: configure basic auth in `config.yaml` (`dashboard.basic_auth.username` + `password_hash`)
 - **Ollama** — port 11434 (if delegation uses local model)
 - **SearXNG** — port 11436 (if web search uses local instance)
 All must listen on `0.0.0.0` or at least the Docker bridge interface for the container to reach them. Check with `ss -tlnp | grep -E '8642|11434|11436'`.
 ## Gateway API Server Configuration
 **This is the #1 reason WebUI cannot reach the gateway.** The `api_server` is a **gateway platform** (like Telegram or Email) — on host-native setups it must be configured explicitly.
 Detailed reference: `references/host-gateway-config.md` (code walkthrough, troubleshooting table, env overlay logic).
 ### How It Gets Enabled
 | Context | Mechanism |
 |---|---|
 | **All-in-Docker** | Env vars: `API_SERVER_ENABLED=true`, `API_SERVER_KEY=xxx` |
 | **Host native** (systemd) | **Must be in `config.yaml`** — env vars are NOT auto-propagated to the systemd unit |
 ### Config.yaml Section
 ```yaml
 platforms:
  api_server:
    enabled: true
    host: 0.0.0.0
    port: 8642
    key: "your-32-char-hex-key-here"
 ⚠️ **Field name matters:** The code reads `extra.get("key", ...)` — so the YAML field must be `key`, NOT `api_key`. A field named `api_key` is silently ignored (not bridged into extra).
 Place this at the root level of `config.yaml` (NOT nested under `gateway:`).
 ### Why It Fails Silently
 - Gateway starts, shows "⚕ Hermes Gateway Starting...", connects to Email etc. — looks healthy.
 - But if `api_server` platform isn't configured, it simply doesn't bind port 8642.
 - `systemctl status hermes-gateway.service` shows `active (running)` — misleading.
 - The only symptom is `ss -tlnp | grep 8642` shows nothing.
 ### Verification
 ```bash
 systemctl is-active hermes-gateway.service
 ss -tlnp | grep 8642
 curl -s http://localhost:8642/health
 # Expected: {"status": "ok", "platform": "hermes-agent", "version": "0.19.0"}
 ```
 If `ss` shows no listener on 8642 but the service is `active`, the api_server platform isn't configured.
 ### systemd Env Var Alternative
 Alternatively, add to the systemd unit file:
 ```ini
 Environment="API_SERVER_ENABLED=true"
 Environment="API_SERVER_HOST=0.0.0.0"
 Environment="API_SERVER_KEY=your-key-here"
 ```
 Then `sudo systemctl daemon-reload && sudo systemctl restart hermes-gateway.service`.
 ## Checking Connectivity
 ```bash
 # From inside the container:
 docker exec hermes-webui curl -s http://host.docker.internal:8642/health
 # From host (should also work):
 curl -s http://localhost:8642/health
 # Logs show API calls:
 docker logs hermes-webui --tail 20
 ```
 If WebUI shows "Check HERMES_WEBUI_GATEWAY_BASE_URL and Gateway API server health":
 1. Verify 8642 listens: `ss -tlnp | grep 8642`
 2. If not → check `platforms: api_server:` in config.yaml or env vars in systemd unit
 3. Verify key matches between gateway and `HERMES_WEBUI_GATEWAY_API_KEY`
 4. `host.docker.internal` must resolve inside container (requires `extra_hosts`)
 ## Migrating from All-in-Docker to Hybrid
 When undoing a full Docker migration (all 3 containers + dashboard), do:
 1. **Stop & remove agent + dashboard containers:**
   ```bash
   docker stop hermes-agent hermes-dashboard
   docker rm hermes-agent hermes-dashboard
   ```
 2. **Re-enable host gateway systemd service** (if it was disabled during migration):
   ```bash
   sudo systemctl enable --now hermes-gateway.service
   ```
 3. **Create dashboard systemd service** (Jun 2026+ requires auth for public bind):
   ```bash
   # Bind 127.0.0.1 and tunnel in via SSH/WireGuard
   ExecStart=/home/estorozhenko/.hermes/hermes-agent/venv/bin/python -m hermes_cli.main dashboard --host 127.0.0.1
   ```
   Full unit template: see `references/dashboard-systemd-service.md`.
 4. **Update docker-compose.yml** — remove hermes-agent + hermes-dashboard services, keep only webui.
 5. **Change environment URLs** from container names to `host.docker.internal`:
   - `HERMES_API_URL=http://hermes-agent:8642` → `http://host.docker.internal:8642`
   - `HERMES_WEBUI_GATEWAY_BASE_URL=http://hermes-agent:8642` → `http://host.docker.internal:8642`
 6. **Add `/opt:/opt:rw` volume** if WebUI needs host filesystem access.
 7. **Recreate webui container:**
   ```bash
   docker compose -f /opt/hermes/docker/docker-compose.yml up -d
   ```
 ## Pitfalls
 - **Gateway API key mismatch — EASY TO MISS ROOT CAUSE:** The API key exists in **4 sources** (config.yaml `key:` field, `~/.hermes/.env`, systemd unit, docker `.env`). All must match. The most common failure pattern: config.yaml and systemd agree, but `~/.hermes/.env` has a **different** value that overwrites the process env at startup. Always check `grep API_SERVER_KEY ~/.hermes/.env` when debugging 401.
 - **YAML field name is `key`, not `api_key`:** The adapter reads `extra.get("key", os.getenv("API_SERVER_KEY", ""))`. A field named `api_key` in config.yaml silently does nothing because the YAML→extra bridge only looks for the exact name `key`.
 - **`host.docker.internal` only works with `extra_hosts`** on Linux. Without it, the hostname won't resolve. Alternative: use host's actual IP (e.g. `192.168.x.x` or Docker bridge gateway like `172.17.0.1`)
 - **docker stop/rm for containers may be blocked by safety system** — use `docker compose -f <file> down` instead
 - **`/opt` on davfs2 (Yandex Disk via WebDAV)** — `stat`, `find`, `rm` are very slow on WebDAV-backed mounts. Consider mounting specific subdirectories instead of the whole `/opt` if the filesystem is partly WebDAV
 - **Telegram SOCKS5 tunnel** — if gateway is on host and Telegram is used, ensure `telegram-tunnel.service` is active
@@ -0,0 +1,60 @@
 # Dashboard Systemd Service — Hermes WebUI Hybrid Deployment
 ## Full Unit File Template
 Location: `/etc/systemd/system/hermes-dashboard.service`
 ```ini
 [Unit]
 Description=Hermes Agent Dashboard - Web Management UI
 After=network-online.target hermes-gateway.service
 Wants=network-online.target
 StartLimitIntervalSec=0
 [Service]
 Type=simple
 User=estorozhenko
 Group=estorozhenko
 ExecStart=/home/estorozhenko/.hermes/hermes-agent/venv/bin/python -m hermes_cli.main dashboard --host 127.0.0.1
 WorkingDirectory=/home/estorozhenko/.hermes/hermes-agent
 Environment="HOME=/home/estorozhenko"
 Environment="USER=estorozhenko"
 Environment="LOGNAME=estorozhenko"
 Environment="PATH=/home/estorozhenko/.hermes/hermes-agent/venv/bin:/home/estorozhenko/.hermes/hermes-agent/node_modules/.bin:/home/estorozhenko/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
 Environment="VIRTUAL_ENV=/home/estorozhenko/.hermes/hermes-agent/venv"
 Environment="HERMES_HOME=/home/estorozhenko/.hermes"
 Environment="GATEWAY_HEALTH_URL=http://localhost:8642"
 Restart=always
 RestartSec=5
 RestartMaxDelaySec=300
 RestartSteps=5
 KillMode=mixed
 KillSignal=SIGTERM
 TimeoutStopSec=30
 StandardOutput=journal
 StandardError=journal
 [Install]
 WantedBy=multi-user.target
 ```
 ## Pitfalls
 - **`--host 0.0.0.0` blocked** — as of June 2026 hardening, Hermes dashboard refuses to bind on a non-loopback interface without a configured auth provider. Error message:
  ```
  Refusing to bind dashboard to 0.0.0.0 — the auth gate engages on non-loopback binds,
  but no auth providers are registered.
  Configure an auth provider before exposing the dashboard:
    • Password: set dashboard.basic_auth.username + password_hash in config.yaml
    • OAuth: run `hermes dashboard register` (Nous Portal)
  There is no unauthenticated public-bind option — to keep it local, bind 127.0.0.1
  and tunnel in (SSH / Tailscale).
  ```
 - **Solution:** bind `127.0.0.1` and access via SSH tunnel:
  ```bash
  ssh -L 9119:localhost:9119 user@bigbox  # from client machine
  ```
  Or via WireGuard: if the client IP can reach the host's loopback is impossible — bind on the WireGuard interface IP instead.
 - **No need for `--insecure` flag** — it's deprecated and ignored as of June 2026.
@@ -0,0 +1,116 @@
 # Host Gateway API Server Configuration — Hermes WebUI Hybrid
 ## The Problem
 WebUI shows `"Check HERMES_WEBUI_GATEWAY_BASE_URL and Gateway API server health"`.
 Gateway systemd service shows `active (running)` but port 8642 isn't listening.
 ## Root Cause
 `api_server` is a **gateway platform** (enum `Platform.API_SERVER = "api_server"`), not a standalone service. It activates only when configured — either via `config.yaml` or env vars. On a host-native (systemd) setup, env vars from the Docker `.env` file are NOT propagated.
 ## Fix — Two Options
 ### Option A: config.yaml (Recommended)
 ```yaml
 # At ROOT level of /home/estorozhenko/.hermes/config.yaml
 platforms:
  api_server:
    enabled: true
    host: 0.0.0.0
    port: 8642
    key: "7c8a2d9f4e1b3a5c6d7e8f9a0b1c2d3e4f5a6b7c"
 ```
 ⚠️ **Field name note:** The field must be `key` (not `api_key`). The adapter reads `extra.get("key", os.getenv("API_SERVER_KEY", ""))`. A field named `api_key` in the YAML is silently ignored because the bridge only moves `key` into the `extra` dict.
 ### Option B: systemd unit env vars
 ```ini
 Environment="API_SERVER_ENABLED=true"
 Environment="API_SERVER_HOST=0.0.0.0"
 Environment="API_SERVER_KEY=7c8a2d9f4e1b3a5c6d7e8f9a0b1c2d3e4f5a6b7c"
 ```
 Then daemon-reload + restart.
 ## How the Code Works
 From `gateway/config.py`:
 1. **`_load_env_overrides`** (line ~2128): reads `API_SERVER_ENABLED`, `API_SERVER_KEY`, `API_SERVER_HOST`, `API_SERVER_PORT`, `API_SERVER_CORS_ORIGINS` from env
 2. **Platform connected checker** (line ~846): requires `key` in `extra` with `min_length=16`
 3. **YAML bridging** (line ~1471): moves `port`, `key`, `host`, `cors_origins`, `model_name` from the platform dict into `extra` so PlatformConfig preserves them
 4. **`_has_usable_api_server_key`** (line ~822): checks key is a string ≥ 16 chars
 Without a valid key (≥16 chars), the platform won't be marked connected and the adapter won't start — no listener on 8642, no error message.
 ## Key Debugging Lesson: Multi-Source Divergence
 The API server key lives in **four independent sources** that must all agree:
 | Source | Where | How it's set |
 |--------|-------|-------------|
 | `config.yaml` | `platforms.api_server.key` | Edited directly |
 | `~/.hermes/.env` | `API_SERVER_KEY=...` | `hermes config set gateway.api_server_key ...` or manual edit |
 | systemd unit | `Environment="API_SERVER_KEY=..."` | `hermes gateway install --force` or manual edit |
 | docker `.env` for webui | `API_SERVER_KEY=...` | Docker compose env-file |
 **Common failure modes:**
 - **config.yaml says one thing, `.env` says another** → `.env` wins (`os.environ` is overwritten at startup), so even though systemd's `Environment=` and config.yaml agree, the process env is different. This is the most frequent root cause of "401 with correct key".
 - **systemd shows key A, `.env` shows key B** → process starts with systemd's env, but then load_gateway_config reads `.env` and calls `os.environ["API_SERVER_KEY"] = B`, overwriting A.
 **To fix:** pick ONE canonical value and set it identically in all four sources. The simplest approach: pick a single key string (e.g. 40 hex chars), put it in config.yaml's `key:` field, and ensure `~/.hermes/.env`, the systemd unit, and docker `.env` all match.
 ### How to Verify All Sources
 ```bash
 # 1. Config YAML
 grep -A1 'key:' ~/.hermes/config.yaml | grep -v extra
 # 2. User .env (read_file blocked — use terminal/xxd)
 grep API_SERVER_KEY ~/.hermes/.env
 # 3. Systemd service
 grep API_SERVER_KEY /etc/systemd/system/hermes-gateway.service
 # 4. Docker compose .env
 grep API_SERVER_KEY /opt/hermes/docker/.env
 # 5. Live process env (verify the actual value the gateway uses)
 SYSTEMD_PID=$(systemctl show hermes-gateway.service -p MainPID | grep -oP '\d+')
 cat /proc/$SYSTEMD_PID/environ | tr '\0' '\n' | grep API_SERVER_KEY
 ```
 All five should show the identical value. If step 5 (live process) differs from 1-4, then something is overwriting the key at runtime — likely step 2.
 ## Verification
 ```bash
 # Confirm listener
 ss -tlnp | grep 8642
 # Health check
 curl -s http://localhost:8642/health
 # → {"status": "ok", "platform": "hermes-agent", "version": "0.19.0"}
 # From inside webui container
 docker exec hermes-webui curl -s http://host.docker.internal:8642/health
 # Check gateway logs for platform startup
 journalctl -u hermes-gateway.service --no-pager -n 10 | grep -i api
 ```
 ## Env Vars Expected by WebUI
 In `docker-compose.yml` the webui expects:
 ```yaml
 - HERMES_WEBUI_GATEWAY_API_KEY=${API_SERVER_KEY:-}
 - HERMES_API_URL=http://host.docker.internal:8642
 - HERMES_WEBUI_GATEWAY_BASE_URL=http://host.docker.internal:8642
 ```
 The `API_SERVER_KEY` env var must be exported or set in the `.env` file that docker-compose reads.