--- title: Measuring recall — RAG, lesson 7 description: Twenty real questions, the chunk that answers each, and the share found in the top five: recall at 5. Under 80%, fix retrieval, not the prompt. url: https://instibrain.ai/guides/rag/measuring-recall updated: 2026-11-07 site: https://instibrain.ai/llms.txt --- lesson 7 of 8 · rag # Measuring recall. Measure before you tune: twenty real questions, the chunk that answers each, and the share found in the top five. Under 80%, fix retrieval, not the prompt. ## new words **Test set** Twenty questions people asked, each with the chunk that answers it, marked by a person. It is kept and rerun after every change. **Recall at 5** The share of questions whose marked chunk is among the five retrieved. **Reciprocal rank** 1 for a chunk found first, 1/2 for second, 1/3 for third, 0 beyond the top five. The average over the questions is the MRR. ## the lesson A pipeline has four steps and any of them can fail. Without a measure, every failure looks like the model's, and the prompt gets rewritten for nothing. The measure is cheap: twenty questions, marked once, rerun in a minute. ``` five questions, the rank of the marked chunk: 1, 2, 7, 1, not found recall at 5 = found in the top five / questions = 3 / 5 = 60% MRR = (1 + 1/2 + 0 + 1 + 0) / 5 = 2.5 / 5 = 0.50 ``` | The number says | Fix | | Recall under 80%: the chunk is not in the top five | Retrieval: the cuts, lesson 3; hybrid search, lesson 4; the reranker, lesson 5 | | Recall fine, MRR low: it is there, ranked fourth or fifth | The reranker, lesson 5 | | Recall fine, the answer wrong | The prompt and the rule, lesson 6 | | The marked chunk no longer exists | The job, lesson 2: the index is stale | Precision is the other half: of the five pasted, how many were needed. Three of five is fine; one of five means four chunks of noise in every prompt, paid for and read. Both numbers move when the chunk size moves, so the test set is rerun after every change to the cuts. Twenty questions, where the marked chunk landed. An example at the 80% line. ## try it Type this and press Enter, with a topic your company has pages about: ``` Write five questions a new colleague would ask about . For each, name the page that answers it, or say that no page does. ``` ## you should see - Five questions, each with a page's path or "no page". That is a test set's first five lines. - The share with a page is your recall before any index: what the brain answers today. --- instibrain is made by WebWork AI, Inc. Everyone uses the same models · the difference is what yours knows.