Icarus Threshold Bug — 2026-07-16 (v2)

The Problem

icarus/hooks.py calls _search_qdrant(query, top_k=2, threshold=0.55) from pre_llm_call(). This passes score_threshold=0.55 to search_with_fallback() in context_enhancer.py.

RRF fusion (hybrid dense+sparse) returns scores 0.33–0.50 even for good matches. At 0.55, every query is filtered out. The search cascade falls to level="none", then hits SQLite fallback ([CE-FALLBACK] SQLite search failed: no such table: lineage), and finally returns an empty list. Icarus injects nothing.

This is fundamentally different from dense-only search, which returns COSINE scores 0.90+ for the same queries. The threshold bug was invisible because:

_search_qdrant() is fail-open (returns [] on any error)
The plugin loads and runs without crashing
"Runs without crashing" ≠ "returns useful results"

Root Cause: RRF vs Dense Score Regimes

RRF (Reciprocal Rank Fusion) normalises scores from two independent retrievers (dense and sparse) into a shared 0–1 range via 1/(k + rank). This inherently produces clustered scores around 0.33–0.50 regardless of the underlying semantic similarity. This is not a bug in RRF — it's how RRF works.

Dense-only search returns raw COSINE similarity (0.90+ for good matches).

The same query at different thresholds:

Query: "XRay VPS VPN"

=== Hybrid (RRF), threshold 0.55 ===
  Level: none, Results: 0
  Falls through to SQLite → [CE-FALLBACK] All fallback levels exhausted.

=== Dense-only, threshold 0.35 ===
  Level: dense-only, Results: 3
  [0.9938] Пароль Юлии Зозули
  [0.9078] Nagios
  [0.9017] Клавиатуры

=== Hybrid (RRF), threshold 0.35 ===
  Level: hybrid, Results: 2
  [0.5000] Инструкция как безопасно расширить кластер Garage до 3+ нод (v2.1)
  [0.5000] Пароль Юлии Зозули

=== Hybrid (RRF), threshold 0.30 ===
  Level: hybrid, Results: 4
  [0.5000] Garage кластер
  [0.5000] Пароль Юлии Зозули
  [0.3333] Двухфакторная VPN
  [0.3333] Nagios

At 0.35, RRF still filters the 0.33 results (2 out of 4 are lost). At 0.30, all 4 are returned.

The Fix

Lower threshold in _search_qdrant() in /opt/hermes/.hermes/plugins/icarus/hooks.py from 0.55 to 0.30.

# Line 722 — NOW:
qdrant_results = _search_qdrant(user_message, top_k=2, threshold=0.30)

The comment above it was also updated to explain the RRF vs dense score gap.

Verification

After fix, all queries return hybrid results within 6ms Qdrant time:

XRay VPS VPN              → hybrid, 4 results
hermes plugin icarus       → hybrid, 3 results
telegram                   → hybrid, 4 results
wireguard                  → hybrid, 4 results

No SQLite fallback noise. No level="none".

Key Insight

When debugging an empty [qdrant] block in Icarus context injection:

First check if _search_qdrant() even runs (no import error → PYTHONPATH fix)
Then check if it returns results (threshold too high for RRF)
These are SEPARATE bugs — the first was fixed 2026-07-14 (PYTHONPATH), the second on 2026-07-16 (threshold 0.55→0.30)

3.1 KiB Raw Permalink Blame History Unescape Escape