lesson 7 of 8 · ai basics

RAG: one question, end to end.

RAG is four steps at question time: embed the question, find the nearest chunks, paste them into the prompt, let the model answer from them with a citation.

new words

Retrieval: Finding the chunks: lesson 5 against the index of lesson 6.
Augmented: The prompt is enlarged with the chunks found.
Generation: The chat model writes the answer. It never saw the index, only the pasted chunks.
Grounding, citation: The answer points at the chunk it came from, so a reader can check.
Hybrid search: The vector score plus a keyword score, added together. It catches codes, names and numbers that vectors blur.
Reranker: A second, slower model that reads the question with each of the top 20 to 50 chunks and reorders them. The best 5 go into the prompt.

the lesson

The prompt that goes to the model, with the chunks first and the question last:

System: Answer only from the passages below. Cite the passage number.
If the passages do not answer the question, say so.

[1] contracts/acme-2025.pdf, p. 4
"Either party may terminate this agreement with 60 days' written notice."
[2] contracts/acme-2025.pdf, p. 1
"This agreement runs for 24 months from 1 March 2025."
[3] handbook/suppliers.md
"A supplier bill is settled one month after it arrives."

Question: What notice does the Acme contract require?

The answer: "60 days, in writing [1]." The model added nothing; it read. Acme is an example name.

Per question	Figure
Chunks pasted	5 to 20
Context added, at 500 tokens each	2,500 to 10,000 tokens
Cost of that context on Sonnet 5.5, $2 per million	0.5 to 2 cents
Time to embed, search and rerank, before the model starts	0.3 to 1.5 s

Where it fails, and the fix for each:

Symptom	Cause	Fix
Right file, half a fact	Chunks too small, or cut mid-clause	Lesson 6: cut on structure, add overlap
A question in other words finds nothing	Keyword search alone	Embeddings, then hybrid search
A code or a name is missed	Vectors alone	Hybrid search
The right chunk exists, ranked 30th	The embedding is coarse	A reranker over the top 50
The answer is from last year's version	The index was not rebuilt	Reingest on file change, nightly at least
The model answers from memory	A weak instruction	"Only from the passages", citations required

Measure before you tune. Take 20 real questions and mark the chunk that answers each. Recall at 5 is the share of questions whose chunk is in the top 5. Under 80%, fix retrieval; the prompt is not the problem.

Where instibrain sits: a page holds a position your team reviewed, and your AI reads it whole, in its context, before it works. Nothing is cut or ranked. RAG is for the thousands of raw files nobody reviewed one by one.

One question through the four steps. Retrieval finds; the chat model only reads what was found.

try it

Type a question. The prompt under it is rebuilt from the sentences of lesson 5 that score above zero, the top three at most.

Your question

System: Answer only from the passages below. Cite the passage number.
If the passages do not answer the question, say so.

[1] finance/payments.md  (cosine 0.58)
"Invoices are paid within 30 days of the date we receive them."
[2] finance/expenses.md  (cosine 0.29)
"Expense claims need a receipt and are paid with the next salary."

Question: When are invoices paid?

you should see

The retrieved sentences numbered and placed before your question, each with its source file.
A question that shares no word with any sentence gets no passage, and the model must say it cannot answer. That is the whole of RAG; everything else tunes the four steps.