mirror of
https://gitverse.ru/kpa39l/memory-os.git
synced 2026-09-29 09:35:05 +00:00
estorozhenko
90 lines
3.1 KiB
Markdown
90 lines
3.1 KiB
Markdown
# Icarus Threshold Bug — 2026-07-16 (v2)
|
||
|
||
## The Problem
|
||
|
||
`icarus/hooks.py` calls `_search_qdrant(query, top_k=2, threshold=0.55)` from `pre_llm_call()`.
|
||
This passes `score_threshold=0.55` to `search_with_fallback()` in `context_enhancer.py`.
|
||
|
||
RRF fusion (hybrid dense+sparse) returns scores 0.33–0.50 even for good matches.
|
||
At 0.55, every query is filtered out. The search cascade falls to `level="none"`,
|
||
then hits SQLite fallback (`[CE-FALLBACK] SQLite search failed: no such table: lineage`),
|
||
and finally returns an empty list. Icarus injects nothing.
|
||
|
||
This is fundamentally different from dense-only search, which returns COSINE scores
|
||
0.90+ for the same queries. The threshold bug was invisible because:
|
||
- `_search_qdrant()` is fail-open (returns `[]` on any error)
|
||
- The plugin loads and runs without crashing
|
||
- "Runs without crashing" ≠ "returns useful results"
|
||
|
||
## Root Cause: RRF vs Dense Score Regimes
|
||
|
||
RRF (Reciprocal Rank Fusion) normalises scores from two independent retrievers
|
||
(dense and sparse) into a shared 0–1 range via `1/(k + rank)`. This inherently
|
||
produces clustered scores around 0.33–0.50 regardless of the underlying semantic
|
||
similarity. This is **not a bug in RRF** — it's how RRF works.
|
||
|
||
Dense-only search returns raw COSINE similarity (0.90+ for good matches).
|
||
|
||
The same query at different thresholds:
|
||
|
||
```
|
||
Query: "XRay VPS VPN"
|
||
|
||
=== Hybrid (RRF), threshold 0.55 ===
|
||
Level: none, Results: 0
|
||
Falls through to SQLite → [CE-FALLBACK] All fallback levels exhausted.
|
||
|
||
=== Dense-only, threshold 0.35 ===
|
||
Level: dense-only, Results: 3
|
||
[0.9938] Пароль Юлии Зозули
|
||
[0.9078] Nagios
|
||
[0.9017] Клавиатуры
|
||
|
||
=== Hybrid (RRF), threshold 0.35 ===
|
||
Level: hybrid, Results: 2
|
||
[0.5000] Инструкция как безопасно расширить кластер Garage до 3+ нод (v2.1)
|
||
[0.5000] Пароль Юлии Зозули
|
||
|
||
=== Hybrid (RRF), threshold 0.30 ===
|
||
Level: hybrid, Results: 4
|
||
[0.5000] Garage кластер
|
||
[0.5000] Пароль Юлии Зозули
|
||
[0.3333] Двухфакторная VPN
|
||
[0.3333] Nagios
|
||
```
|
||
|
||
At 0.35, RRF still filters the 0.33 results (2 out of 4 are lost).
|
||
At 0.30, all 4 are returned.
|
||
|
||
## The Fix
|
||
|
||
Lower `threshold` in `_search_qdrant()` in `/opt/hermes/.hermes/plugins/icarus/hooks.py`
|
||
from 0.55 to 0.30.
|
||
|
||
```python
|
||
# Line 722 — NOW:
|
||
qdrant_results = _search_qdrant(user_message, top_k=2, threshold=0.30)
|
||
```
|
||
|
||
The comment above it was also updated to explain the RRF vs dense score gap.
|
||
|
||
## Verification
|
||
|
||
After fix, all queries return hybrid results within 6ms Qdrant time:
|
||
|
||
```
|
||
XRay VPS VPN → hybrid, 4 results
|
||
hermes plugin icarus → hybrid, 3 results
|
||
telegram → hybrid, 4 results
|
||
wireguard → hybrid, 4 results
|
||
```
|
||
|
||
No SQLite fallback noise. No `level="none"`.
|
||
|
||
## Key Insight
|
||
|
||
When debugging an empty `[qdrant]` block in Icarus context injection:
|
||
1. First check if `_search_qdrant()` even runs (no import error → PYTHONPATH fix)
|
||
2. Then check if it returns results (threshold too high for RRF)
|
||
3. These are SEPARATE bugs — the first was fixed 2026-07-14 (PYTHONPATH),
|
||
the second on 2026-07-16 (threshold 0.55→0.30) |