Repository navigation
Hybrid search and the similar-notes advisory return weak vector matches #1718
Description
Found during the v0.24.0 acceptance test.
Default (hybrid) search returns unrelated notes for a gibberish query. In a 3-note project, search_notes(query="zzqxnonexistent") returned two notes with vector scores of 0.58 and 0.57, just above semantic_min_similarity (0.55, bge-small-en-v1.5). Text mode returns nothing. Since #1696 semantic search is always on, so this is the default experience. v0.23.2 had the same defaults, so it is not a regression.
The similar-notes advisory on write_note shows the same problem. In a project with one other, unrelated note, it suggested that note as a possible duplicate.
Investigate: for hybrid, require some lexical support or a higher vector floor when FTS finds nothing. For the advisory, use a higher threshold. Base the numbers on measurements; see the #1259 experiment.
Activity
claude commentedon Oct 10, 2026
Claude finished @phernandez's task in 19s —— View job
Triage
Type: bug (labeled
bug). Hybrid search and thewrite_notesimilar-notes advisory return unrelated notes. It is not a regression, since v0.23.2 had the same defaults. Since feat(core): remove semantic_search_enabled; semantic search is always on #1696 it is the default experience.Component: MCP tools (search and
write_noteadvisory) plus the search and semantic layer. No Cloud involvement, so the component label isnone.Priority: Medium. The result is wrong but there is no data loss. It shows up in a basic default search path, and most visibly in small projects.
Complexity: Medium. The fix needs measurement, not just a code change. Two separate thresholds are involved:
The numbers should come from the feat: upsert_note tool — semantic dedup before write using existing embeddings #1259 experiment. Measuring them against
bge-small-en-v1.5on gibberish queries and on true near-duplicates would be the first step.Duplicates
I found no exact duplicate. #1685 (closed) is related: "Default hybrid search and grep return unrelated notes instead of no results". It is worth checking whether it covered the same ground, or whether this issue is the follow-up for the vector-floor case.
Triage only. I made no code changes.
fix(mcp): apply the advisory floor only on the measured default model
fix(mcp): recognize every spelling of the default embedding model
refactor(mcp): decide the advisory floor by the persisted embedding i…
fix(mcp): give the similar-notes advisory its own similarity floor
fix(mcp): apply the advisory floor only on the measured default model
fix(mcp): recognize every spelling of the default embedding model
refactor(mcp): decide the advisory floor by the persisted embedding i…
docs(core): rewrap the #1718 changelog entry
fix(mcp): build the measured embedding identity without env overrides
FBISiri commentedon Oct 13, 2026
Saw #1722 land for the advisory half — gating the 0.70 floor on the exact embedding identity, and falling back to
semantic_min_similarityelsewhere, is a careful way to keep a model-specific number from leaking onto text-embedding-3-small users.A question on the hybrid half, before anyone picks it up. The issue lists two options for the case where FTS finds nothing: (a) require some lexical support, or (b) raise the vector floor. Reading
search_reader.py, vector-only rows currently pass straight throughfuse()with their raw cosine, and the floor is applied once on the vector leg before fusion. So the two options fail differently:zzqxnonexistent, but it also drops the queries where vector-only recall is the point: paraphrases that share no tokens with the note, and CJK queries — unspaced CJK compounds already get zero FTS rows (that's whattests/api/v2/test_cjk_search_guidance.pyfrom CJK: unspaced multi-word queries miss silently (no token split, relaxed fallback not triggered) #1533 covers), so hybrid is effectively vector-only there. Under (a), those return nothing in the default mode, even thoughsearch_type="vector"would still find them._resolve_rows, it tightens pure vector mode too, not just the FTS-empty case.So: is the plan for (b) to reuse the same identity gating as
similar_notes_min_similarity? Or is a relative criterion on the table, e.g. when FTS is empty, keep vector hits only if they clear the floor by some margin, or only if the top hit separates from the rest? A relative criterion would sidestep per-model calibration. I'm asking because we hit the same drift in our own memory store's dedup and ended up with per-type absolute thresholds, and I'm not sure that was the right call either.Happy to run the gibberish / paraphrase / CJK probe set against bge-small and post the score distributions here if that would help pick the number.
phernandez commentedon Oct 23, 2026
Another data point from live testing of main at 0276799 (2026-10-22): in a ~15-note project,
search_notes(query="zxqvnonexistentterm")(hybrid) returned 3 unrelated notes at 0.554-0.574; text mode returned none. The similar-notes advisory onwrite_notesuggested an empty-body note (Empty Note) as similar to a long unrelated test report. Empty or near-empty notes probably embed close to everything; the advisory could skip them, like it skips generated notes (91d83d6).FBISiri commentedon Oct 23, 2026
Thanks, that's a useful reading. 0.554–0.574 for
zxqvnonexistenttermsits just above the 0.55 floor, and text mode returning nothing means FTS contributed zero rows. So all three hits are vector-only rows that went throughfuse()on raw cosine with no lexical support. That's the case the (a)/(b) split above is about, and it shows the margin over the floor is only about 0.02. That points away from an absolute floor bump and toward either (a) or a relative check for the FTS-empty case.The empty-body advisory hit looks like a separate mechanism. A note with almost no body embeds near the generic center of the space, so it scores "similar" to everything. Skipping empty/near-empty notes the way 91d83d6 skips generated notes seems right to me.
I'd like to take the hybrid half: gating vector-only rows when FTS returns nothing. Before writing anything, which direction do you prefer? Lexical support required, or a relative criterion (margin over the floor, or top-hit separation)? I can post the gibberish / paraphrase / CJK score distributions on bge-small first if that would help decide.