Introduction to Information Retrieval
The classic textbook on how search engines work, by Manning, Raghavan and Schutze, with the full text free to read online. Inverted indexes, tokenisation, scoring, tf-idf and evaluation explained from first principles.
Book
Other
Cambridge University Press (free online edition)
Several weeks; read chapters as needed
Advanced
Free
What the book is
Introduction to Information Retrieval, by Christopher D. Manning, Prabhakar Raghavan and Hinrich Schutze, was published by Cambridge University Press in 2008 and grew out of courses taught at Stanford and other universities. The companion website hosts the book free to read online and as PDFs, along with slides and exercises. It explains, from a computer science point of view, how a search engine finds and ranks documents.
What it covers
- The inverted index: how a search engine maps each term to a list of documents, and how queries walk those lists.
- Tokenisation and normalisation: splitting text into terms, case folding, stemming and stop words, which explain why "running" can match "run".
- Index construction and compression: building indexes over collections too large for memory.
- Scoring and the vector space model: tf-idf, cosine similarity and why rare words count for more.
- Evaluation: precision, recall and ranked evaluation, so you can tell whether a change made search better.
- Later chapters: probabilistic models, classification, clustering and web search topics such as link analysis.
Is it worth reading?
Not cover to cover for most app builders. But if you are adding search to an app with Elasticsearch, OpenSearch, Meilisearch or PostgreSQL full-text search, chapters 1, 2 and 6 explain what those tools are actually doing. Relevance tuning stops being guesswork once you understand term frequency, document frequency and normalisation. Modern engines mostly use BM25, a refinement of the ideas in chapter 6 and 11, and modern semantic search builds on the same vector space intuition.
Is it out of date?
The fundamentals are not. Neural ranking and embeddings arrived after it was written, so pair it with a current vector search guide for that part.
Prerequisites and time
Comfortable programming and some basic probability and linear algebra for the later chapters. Plan an evening per chapter.
Free or paid?
The online edition on the companion site is free to read. The printed book is sold by Cambridge University Press.
Best for: students and developers who want to understand search properly, and final year students building a search or retrieval project.
Ready to start?
Opens on Cambridge University Press (free online edition) in a new tab.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
Follow along
New writing, resources and project ideas land here first.