Embeddings alone failed on part numbers: what hybrid search fixed for us
Published · Updated
Pure vector search scored 61% on queries containing SKUs and error codes. Adding BM25 alongside pgvector lifted retrieval accuracy to 89% without touching the embedding model.
We shipped a semantic search feature for an industrial client earlier this year. The corpus was technical documentation: manuals, maintenance logs, spec sheets, about 40,000 chunks indexed in pgvector. Demo queries worked beautifully. Then real users arrived and typed things like 'E47 fault on HX-2200' and got back poetic but useless passages about general fault handling. This article is about why that happened and the unglamorous fix.
The failure mode nobody demos
Dense embeddings are excellent at intent and terrible at rare tokens. A part number like HX-2200 appears a handful of times in the corpus, so the model never learned a meaningful representation for it. The vector for 'E47 fault on HX-2200' ends up close to generic troubleshooting text, because that is what the embedding model understands. We measured it: across 200 real user queries, the 61 containing SKUs, error codes or model numbers had a top-5 hit rate of 61%. Natural-language questions sat at 88%.
Why reranking alone did not save us
Our first instinct was to add a cross-encoder reranker on top of the vector results. It helped on natural-language queries but did almost nothing for part-number queries, for a simple reason: the right chunk never made it into the candidate set. A reranker can only reorder what retrieval hands it. If the document containing 'HX-2200' ranks 4,000th by vector distance, no reranker will ever see it. Retrieval recall was the bottleneck, not ranking quality.
The fix: boring lexical search in the same database
We added PostgreSQL full-text search alongside pgvector and fused the two result sets with reciprocal rank fusion, k=60, no learned weights. No new infrastructure, no Elasticsearch cluster, one extra index. The lexical leg catches exact tokens: part numbers, error codes, proper nouns. The vector leg catches paraphrases and intent. RRF merges them without needing score normalisation, which matters because ts_rank and cosine distance live on completely different scales.
Numbers after 30 days in production
We reran the same 200-query evaluation set. Overall top-5 hit rate went from 80% to 89%. The part-number subset jumped from 61% to 93%. Natural-language queries improved slightly, 88% to 91%, because some of them also contained a model name that lexical search anchored. Latency cost was about 12 milliseconds per query for the extra index lookup, well inside our 300 ms budget. Index size grew by roughly 15%.
Where fusion still struggles
Two honest caveats. First, typos in part numbers defeat both legs: 'HX-220' does not match lexically and means nothing semantically. We added a trigram index for fuzzy matching on the metadata fields, which recovered about half of those cases. Second, RRF weights matter more than the literature suggests on skewed corpora. We ended up boosting the lexical leg for queries shorter than six tokens, a heuristic we arrived at by testing, not theory.
What we would tell our past selves
Start hybrid from day one on any technical corpus. Pure vector search demos well and fails quietly on exactly the queries your most expert users ask. The whole fix was two SQL indexes and forty lines of fusion logic, and it outperformed a week of embedding model experiments. Measure retrieval recall on real queries before reaching for a reranker or a bigger model; the bottleneck is usually earlier in the pipeline than you think.
Working on a project where these methods apply?