Blazing fast open-source Semantic Search-as-you-type with Rust

Faster than embedding inference.

The benefits of having a fast semantic search are obvious: synonyms, typos, and multi-language queries out of the box. But semantic search requires computing embeddings. Can it be fast enough to be used in the search-as-you-type scenario?

The main challenge of the search-as-you-type scenario is to deliver results within the time of a single keystroke, ideally within the blink of an eye. And the load of the search-as-you-type method is much higher than that of the regular search.

Made possible with Rust and a few optimization tricks.

\- Qdrant as a vector search engine

\- ONNX inference in #Rust

\- Embeddings cache & lookup

\- Parallel & Batch requests

\- Hybrid search with full-text filtering + vector re-scoring

​

Details https://qdrant.tech/articles/search-as-you-type/
Code repo https://github.com/qdrant/page-search