Vector Search Engine Built From Scratch

The problem vector search solves

Semantic search, recommendations and retrieval-augmented generation all reduce to the same question: given a query vector, which of millions of stored vectors are closest? Comparing against every vector is exact and too slow. Every vector database therefore trades a little accuracy for a lot of speed, using an index that only looks at a promising subset.

Many developers now pick a vector database and copy a configuration from a blog post without understanding that trade-off. When recall quietly drops after a filter is added, or memory use explodes, they cannot say why. Building the engine yourself is the most direct way to understand it, and it produces a rare kind of portfolio project.

Who uses the result? Primarily you and your examiners as a learning artefact; secondarily, developers who want a small embeddable index for a desktop app or an edge device, and lecturers who want a readable reference implementation.

What already exists and the gap

• Faiss, from Meta, is a C++ library with Python bindings covering many index types, including IVF and product quantisation.
• hnswlib is a compact, widely used HNSW implementation.
• Qdrant, Weaviate and Milvus are vector databases with filtering, persistence and APIs.
• pgvector adds vector search to PostgreSQL.
• ANN-Benchmarks is a public benchmark comparing these libraries on standard datasets.

These are excellent and you will not beat them. The gap is understanding: a small, readable engine with a documented benchmark that explains, with measurements, why the production libraries are faster and where filtering breaks approximate search.

What you build

1. Exact search first: brute force over in-memory vectors, used as ground truth for every later measurement.
2. Persistence: vectors and metadata stored on disk, loaded on restart.
3. Approximate index: HNSW from the original paper, a layered graph where each node keeps a short list of neighbours, with build parameters (M, efConstruction) and search parameter (efSearch).
4. Filtering: queries such as "nearest 10 products where category is shoes and price is under a limit".
5. Benchmarks: on a standard dataset with published ground truth, at several parameter settings.
6. Python bindings so you can benchmark from a notebook and compare against Faiss or hnswlib.

The technically interesting part

Filtering is the hard part. If you search first and filter afterwards, a selective filter can leave you with two results instead of ten. If you filter first and search exactly, you lose the speed. Production systems use strategies such as filtering during graph traversal or switching to brute force when the filter is very selective. Implement at least two strategies, measure recall and speed at different filter selectivities, and show your supervisor where each one breaks. That is ground most tutorials skip.

How to evaluate it

• Recall at 10 against exact ground truth, at several efSearch values.
• Queries per second on one core and, if implemented, several.
• Build time and memory per million vectors.
• Recall against queries per second curve for your engine and a reference library on the same chart and hardware.
• Filter experiment: recall and speed at filter selectivities from 50 percent down to 0.1 percent, for each strategy.
• Durability: kill the process during insert and confirm the index loads and returns correct results.

Suggested plan

1. Weeks 1 to 2: read the HNSW paper, set up datasets and the exact baseline.
2. Weeks 3 to 4: persistence and the benchmark harness.
3. Weeks 5 to 8: HNSW implementation and tuning.
4. Weeks 9 to 10: filtering strategies.
5. Weeks 11 to 12: Python bindings and comparison with the reference library.
6. Weeks 13 to 15: stretch goal and write-up.

Path to a product

Do not pitch this as a startup; the market has strong open source options. The realistic outcomes are a portfolio centrepiece that gets interviews at search, database and AI infrastructure companies, an open source educational repository that others learn from, and a small embeddable library for niche cases such as offline desktop search. Contributing a fix or benchmark back to an existing project afterwards is an excellent next step.

Scope warning

Do not build a distributed cluster, a REST server with authentication or an embedding model. Use pre-computed benchmark vectors. Losing to the reference is fine: explaining precisely why, with measurements, is a better result than a vague claim of comparable performance.

Learn what you need on Sefism

• AI engineering roadmap : where vector search fits in retrieval and RAG systems.
• AI engineering cheat sheet : embeddings, similarity and retrieval terms.
• DSA cheat sheet : graphs, heaps and priority queues used in HNSW search.
• CMU 15-445/645: Intro to Database Systems : storage, indexing and buffer management.
• Google Research Blog : readable posts on approximate nearest neighbour search research.

Get help with this idea

The risk is an index that works but has no honest benchmark. Tauseef can help you plan the build order, choose datasets and design comparisons examiners will respect. If you are new here, create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session