mirror of
https://gitverse.ru/kpa39l/rag-pipeline-docker.git
synced 2026-09-29 09:15:11 +00:00
estorozhenko
108 lines
3.8 KiB
Markdown
108 lines
3.8 KiB
Markdown
# Memory OS Deployment — Session Details
|
|
|
|
## Environment
|
|
|
|
- Host: Linux, no Docker Desktop
|
|
- Ollama: Docker container on `ollama_default` network, port 11434
|
|
- Worker: Docker Compose stack with Qdrant + Redis + ARQ worker
|
|
- Obsidian vault: `/opt/hermes/obsidian-vault/`
|
|
- Wiki path: `/opt/hermes/vault/wiki/raw/obsidian/`
|
|
- State file: `~/.hermes/wiki_ingest_state.json` (tracks ingested files by hash)
|
|
- Sync state: `~/.hermes/obsidian_sync_state.json`
|
|
|
|
## Docker Compose Path
|
|
|
|
`/opt/hermes/memory-os/docker/docker-compose.yml`
|
|
|
|
## .env File
|
|
|
|
`/opt/hermes/memory-os/docker/.env` — contains:
|
|
- `OLLAMA_BASE_URL=http://ollama:11434`
|
|
- `OLLAMA_MODEL=qwen3-8b-64k`
|
|
- `OLLAMA_EMBEDDING_MODEL=nomic-embed-text:latest`
|
|
- `EMBEDDING_API_BASE=http://ollama:11434/v1`
|
|
- `EMBEDDING_DIMS=768`
|
|
- `REDIS_PASSWORD`, `QDRANT_API_KEY`, `OPENROUTER_API_KEY`
|
|
|
|
## Error: Reflection Failing
|
|
|
|
**Symptom:** `j_failed=90` on worker, all from `cron:process_reflection`
|
|
|
|
**Error:**
|
|
```
|
|
httpx.ConnectError: [Errno -2] Name or service not known
|
|
```
|
|
The worker was trying `http://host.docker.internal:11434/api/generate`
|
|
|
|
**Root cause:** `OLLAMA_BASE_URL` defaulted to `http://host.docker.internal:11434` in `services/llm.py`:
|
|
```python
|
|
OLLAMA_BASE_URL = os.environ.get("OLLAMA_BASE_URL", "http://host.docker.internal:11434")
|
|
```
|
|
This variable was not listed in `docker-compose.yml` under `worker.environment`, so the container fell back to the hardcoded default.
|
|
|
|
**Fix:**
|
|
1. Added `OLLAMA_BASE_URL: ${OLLAMA_BASE_URL:-http://ollama:11434}` and `OLLAMA_MODEL: ${OLLAMA_MODEL:-qwen3-8b-64k}` to `docker-compose.yml`
|
|
2. Added `OLLAMA_MODEL=qwen3-8b-64k` to `.env`
|
|
3. Connected worker to `ollama_default` external network in compose
|
|
4. `docker compose up -d --force-recreate worker` to rebuild
|
|
|
|
## Network Layout
|
|
|
|
```
|
|
worker (memory-os_default: 172.24.0.x, ollama_default: 172.18.0.3)
|
|
→ ollama (ollama_default: 172.18.0.5) via DNS
|
|
→ qdrant (memory-os_default) via DNS
|
|
→ redis (memory-os_default) via DNS
|
|
search-api (memory-os_default: 172.24.0.x, ollama_default: 172.18.0.x)
|
|
→ ollama (ollama_default) via DNS
|
|
→ qdrant (memory-os_default) via DNS
|
|
```
|
|
|
|
## Qdrant Collection
|
|
|
|
- Name: `knowledge_base`
|
|
- Points: 683
|
|
- Dense vector: 768 dims, Cosine distance
|
|
- Sparse vector: BM25 (on_disk=True)
|
|
- Embedding model: `nomic-embed-text:latest`
|
|
|
|
## Search API
|
|
|
|
**Path:** `/opt/hermes/memory-os/search_api/main.py`
|
|
**Port:** localhost:8000
|
|
**Docker image:** `docker-search-api` (built from `search_api/Dockerfile`)
|
|
|
|
**Files:**
|
|
```
|
|
search_api/
|
|
├── Dockerfile
|
|
├── requirements.txt # fastapi, uvicorn, httpx, pydantic
|
|
└── main.py # FastAPI app with POST /search and GET /health
|
|
```
|
|
|
|
**Endpoints:**
|
|
- `GET /health` → `{"status": "ok", "qdrant": true, "ollama": true}`
|
|
- `POST /search` → body: `{"query": "search text", "top_k": 3}` → `{"query": "...", "results": [...], "total": N}`
|
|
|
|
## Cron Jobs
|
|
|
|
### sync+ingest (LLM-driven, every 10m)
|
|
- Name: `memory-os obsidian sync + ingest`
|
|
- Schedule: every 10 minutes
|
|
- Prompt: Runs sync script + ingest pipeline
|
|
- Tools: terminal only
|
|
|
|
### micro-reflection trigger (no_agent, every 5m, silent)
|
|
- Name: `memory-os micro-reflection trigger`
|
|
- Schedule: `*/5 * * * *`
|
|
- Script: `docker exec docker-worker-1 python3 /app/scripts/reflection_trigger.py`
|
|
- `no_agent: true` — no LLM tokens, just runs the script
|
|
- `deliver: local` — silent, no Telegram notifications
|
|
- Script path must be mounted into container: `- ../scripts:/app/scripts:ro` in docker-compose.yml
|
|
|
|
## Scripts
|
|
|
|
- `scripts/test_qdrant_search.py` — standalone test: takes --query, gets embedding, searches Qdrant, prints top-5
|
|
- `scripts/sync_obsidian_to_wiki.py` — copies .md from Obsidian vault to wiki path, tracks state
|
|
- `scripts/wiki_continuous_ingest.py` — detects new/changed files, enqueues to ARQ worker
|
|
- `scripts/reflection_trigger.py` — checks idle, respects budget, enqueues micro-reflection |