--- title: RAG — retrieval beside your brain, in eight lessons description: Retrieval beside your brain, in eight lessons: what to index, the ingestion job, cutting by structure, hybrid search, the reranker, the prompt, the measure. url: https://instibrain.ai/guides/rag updated: 2026-11-07 site: https://instibrain.ai/llms.txt --- # Retrieval beside your brain, in eight short lessons. AI basics said what a vector, an embedding and a chunk are. These lessons run a retrieval pipeline beside the brain: what goes in it, the job that builds it, the cuts, hybrid search, the reranker, the prompt, the measure that says which step failed, and where your pages sit. Lesson 1: What goes in the index → AI basics comes first Every number is computed on the page or is a list price. Lesson 3 has a live toy that cuts a page. the lessons · the words under it ## Eight lessons, from the files to the loop back to a page. The first two decide what is indexed and how the job runs. Lessons 3 to 6 are the four steps, each with its number. Lesson 7 measures them, and lesson 8 puts the pages beside the index. - 1What goes in the index A reviewed position is a page, read whole; thousands of raw files nobody reviewed are the index. - 2Ingestion as a job A job reads the files, cuts, embeds and stores, with each chunk's source and who may read it; it runs again on change. - 3Chunking by structure Cut on headings and paragraphs, carry the section's title, repeat one sentence as overlap; the toy cuts a page live. - 4Hybrid search Keyword search beside vector search, the two rankings merged, so a code and a meaning are both found. - 5The reranker A second model reads the question with each of the top chunks and reorders them; the best five go in. - 6The prompt and the citation rule Passages first, numbered with their source, the question last, and the rule: answer only from them, or say so. - 7Measuring recall Twenty real questions, the chunk that answers each, and the share found in the top five; under 80%, fix retrieval. - 8Where instibrain sits Pages read whole beside the index, and the loop that turns an answer found in the files into a page. Start lesson 1 → what you learned ## The numbers to hold. Each is computed in its lesson, from the counts given there and Claude's list prices. | Number | What it is | | 200,000 tokens | 400 pages at 500 tokens: a fifth of the window, read whole. | | 50,000,000 tokens | 5,000 documents of 20 pages: fifty windows, so an index. | | $5, then $0.001 | The first embedding run at $0.10 per million, then one changed 20-page document. | | 150 to 600 words | One chunk, 200 to 800 tokens, cut on structure. | | 1 / (60 + rank) | Rank fusion: a chunk's score in each list, added up. | | 26,000 tokens | What a reranker reads for 50 chunks of 500 and a 20-token question. | | $0.0053 + $0.0015 | One question on Sonnet 5.5: 2,630 tokens in, 150 out. | | 80% | Recall at 5 on twenty questions, the line under which retrieval is fixed first. | the series before How a model works Inside the chat model: one pass from your text to the next token, attention, layers and weights, training, running, and the kinds of model. the pages themselves The brain What to write down, and what an AI does with a page. every series Guides The four kinds, what you need, and how a lesson works. --- instibrain is made by WebWork AI, Inc. Everyone uses the same models · the difference is what yours knows.