Initial commit: Hermes skill memory-os

references/verify-threshold-bug.md

@@ -0,0 +1,90 @@
+# Icarus Threshold Bug — 2026-07-16 (v2)
+
+## The Problem
+
+`icarus/hooks.py` calls `_search_qdrant(query, top_k=2, threshold=0.55)` from `pre_llm_call()`.
+This passes `score_threshold=0.55` to `search_with_fallback()` in `context_enhancer.py`.
+
+RRF fusion (hybrid dense+sparse) returns scores 0.33–0.50 even for good matches.
+At 0.55, every query is filtered out. The search cascade falls to `level="none"`,
+then hits SQLite fallback (`[CE-FALLBACK] SQLite search failed: no such table: lineage`),
+and finally returns an empty list. Icarus injects nothing.
+
+This is fundamentally different from dense-only search, which returns COSINE scores
+0.90+ for the same queries. The threshold bug was invisible because:
+- `_search_qdrant()` is fail-open (returns `[]` on any error)
+- The plugin loads and runs without crashing
+- "Runs without crashing" ≠ "returns useful results"
+
+## Root Cause: RRF vs Dense Score Regimes
+
+RRF (Reciprocal Rank Fusion) normalises scores from two independent retrievers
+(dense and sparse) into a shared 0–1 range via `1/(k + rank)`. This inherently
+produces clustered scores around 0.33–0.50 regardless of the underlying semantic
+similarity. This is **not a bug in RRF** — it's how RRF works.
+
+Dense-only search returns raw COSINE similarity (0.90+ for good matches).
+
+The same query at different thresholds:
+
+```
+Query: "XRay VPS VPN"
+
+=== Hybrid (RRF), threshold 0.55 ===
+  Level: none, Results: 0
+  Falls through to SQLite → [CE-FALLBACK] All fallback levels exhausted.
+
+=== Dense-only, threshold 0.35 ===
+  Level: dense-only, Results: 3
+  [0.9938] Пароль Юлии Зозули
+  [0.9078] Nagios
+  [0.9017] Клавиатуры
+
+=== Hybrid (RRF), threshold 0.35 ===
+  Level: hybrid, Results: 2
+  [0.5000] Инструкция как безопасно расширить кластер Garage до 3+ нод (v2.1)
+  [0.5000] Пароль Юлии Зозули
+
+=== Hybrid (RRF), threshold 0.30 ===
+  Level: hybrid, Results: 4
+  [0.5000] Garage кластер
+  [0.5000] Пароль Юлии Зозули
+  [0.3333] Двухфакторная VPN
+  [0.3333] Nagios
+```
+
+At 0.35, RRF still filters the 0.33 results (2 out of 4 are lost).
+At 0.30, all 4 are returned.
+
+## The Fix
+
+Lower `threshold` in `_search_qdrant()` in `/opt/hermes/.hermes/plugins/icarus/hooks.py`
+from 0.55 to 0.30.
+
+```python
+# Line 722 — NOW:
+qdrant_results = _search_qdrant(user_message, top_k=2, threshold=0.30)
+```
+
+The comment above it was also updated to explain the RRF vs dense score gap.
+
+## Verification
+
+After fix, all queries return hybrid results within 6ms Qdrant time:
+
+```
+XRay VPS VPN              → hybrid, 4 results
+hermes plugin icarus       → hybrid, 3 results
+telegram                   → hybrid, 4 results
+wireguard                  → hybrid, 4 results
+```
+
+No SQLite fallback noise. No `level="none"`.
+
+## Key Insight
+
+When debugging an empty `[qdrant]` block in Icarus context injection:
+1. First check if `_search_qdrant()` even runs (no import error → PYTHONPATH fix)
+2. Then check if it returns results (threshold too high for RRF)
+3. These are SEPARATE bugs — the first was fixed 2026-07-14 (PYTHONPATH),
+   the second on 2026-07-16 (threshold 0.55→0.30)