Embeddings and similarity

What a vector is in this world, where it comes from and the three ways of measuring how close two vectors are.

Embedding

Also called vector embedding, dense vector

What: A fixed-length list of floating-point numbers that a model produces for a piece of text, an image or other data, placed so that similar meanings land close together in the vector space.
Why: It turns "find things that mean the same" into a geometry problem: search, recommendations, de-duplication, clustering and retrieval for LLMs all become nearest-neighbour lookups.

Python

from openai import OpenAI

client = OpenAI()
resp = client.embeddings.create(
    model="text-embedding-3-small",
    input=["refund policy", "how do I get my money back?"],
)
vectors = [d.embedding for d in resp.data]
print(len(vectors[0]))

Watch out

Comparing vectors made by two different models: their spaces are unrelated, so the distances mean nothing.

Choosing an embedding model

Also called encoder, bi-encoder

What: The model that maps input to vectors. Options range from hosted APIs to open models you run yourself, and they differ in quality, languages, maximum input length, dimensions and price.
Why: Retrieval quality is capped by the model: no index or tuning can find a match the embedding never placed nearby.
Example: A support bot for German and English customers needs a multilingual model; a code search tool does better with a model trained on code.

Watch out

Picking a model from a public leaderboard alone instead of testing it on a sample of your own queries and documents.

Dimensions

Also called vector size, dimensionality

What: The number of values in each vector, fixed by the model, commonly between 384 and 3072. Some models let you request a shorter vector with a small loss in quality.
Why: Dimensions drive storage, memory and query cost linearly, so a smaller vector can make an index fit in RAM.
Example: 1 million vectors of 1536 float32 values take about 6 GB before any index overhead; at 512 dimensions it is about 2 GB.

Watch out

Declaring a column or collection with the wrong dimension count: inserts fail, or a later model change needs a new column.

Cosine similarity

Also called cosine distance

What: The cosine of the angle between two vectors, from -1 (opposite) to 1 (same direction). Cosine distance is 1 minus that. It ignores vector length and compares direction only.
Why: The default for most text embedding models, because meaning is carried by direction rather than magnitude.

Python

import numpy as np

def cosine(a: np.ndarray, b: np.ndarray) -> float:
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))

Watch out

Mixing up similarity and distance: higher similarity is better, lower distance is better.

Dot product

Also called inner product, IP, MIPS

What: The sum of the element-wise products of two vectors. It grows with both the angle match and the vectors' lengths.
Why: It is the cheapest metric to compute, and some models are trained for it so that length encodes something such as popularity or confidence.

Watch out

Using dot product on unnormalised vectors from a model trained for cosine: long vectors win regardless of meaning.

Euclidean distance

Also called L2 distance

What: The straight-line distance between two points: the square root of the summed squared differences. Zero means identical.
Why: Natural for image and geometric features, and the metric many index algorithms were first designed around.

Watch out

Assuming L2 and cosine rank results differently on normalised vectors: they give the same order, so pick the cheaper one.

Normalisation

Also called unit vectors, L2 normalisation

What: Scaling a vector to length 1 by dividing it by its norm. For unit vectors, cosine similarity equals the dot product, and L2 distance ranks results the same way.
Why: Normalising once at write time lets the store use the fast dot product while keeping cosine behaviour.

Python

import numpy as np

def normalise(v: np.ndarray) -> np.ndarray:
    return v / np.linalg.norm(v)

Watch out

Normalising stored vectors but not query vectors, or the other way round.

Semantic vs lexical search

Also called keyword search, full-text search

What: Semantic search compares meaning through embeddings. Lexical search matches the actual words, weighted by how rare they are.
Why: Semantic search finds "cancel my plan" for "end subscription"; lexical search finds exact codes, names and error strings that embeddings blur.
Example: A query for ERR_CONN_RESET is best served by keywords; "why does my connection keep dropping" is best served by vectors.

Go deeper