---
title: What goes in the index — RAG, lesson 1
description: A reviewed position is a page, read whole in the window; the thousands of raw files nobody reviewed, contracts, tickets, mail, notes, are what an index is for.
url: https://instibrain.ai/guides/rag/what-goes-in-the-index
updated: 2026-11-07
site: https://instibrain.ai/llms.txt
---

lesson 1 of 8 · rag

# What goes in the index.

A reviewed position is a page, read whole. The thousands of raw files nobody reviewed one by one are what an index is for.

## new words

**Page**
One thing your team knows, reviewed, current, in a space with its readers. Your AI reads it whole, before it works.

**Raw file**
A document nobody reviewed into a page: a contract, a ticket, a mail, a meeting's notes. Thousands of them, and more each day.

**Index**
The table of chunks and vectors that a question is searched against. AI basics, lesson 6.

## the lesson

Two stores, two jobs. A page says what is true now, for the people who read its space, and it is small: a brain of 400 pages at 500 tokens each is 200,000 tokens, a fifth of a current Claude model's window. Your AI reads a page whole; nothing is cut, nothing is ranked.

A firm's files are another size. 5,000 documents of 20 pages are 50 million tokens: fifty windows. Nobody reviewed each one, and nobody will. That is what an index is for: the question finds the five chunks that matter, and the rest stays on disk.

| It is | It goes |
| A rule, a price, a decision, a way of working | On a page |
| What a new colleague must know on day one | On a page |
| A signed contract, a closed ticket, a thread of mail | In the index |
| Three years of meeting notes | In the index |
| An answer found in the index that the team asks for again | On a page, from then on |

The test for a file: would a reviewed page replace it? A contract's notice period belongs on the client's page; the contract itself stays a file, indexed, for the day someone needs clause 14. The three levers says why: context is read whole and cited by path; retrieval is cited by chunk and fresh at the next reindex.

Two stores by size. The counts are an example, computed at 500 tokens a page.

## try it

Type this and press Enter:

```
List the kinds of documents my company keeps that are not pages: contracts, tickets, mail, notes, files. For each kind, say whether one reviewed page could replace it, or whether it needs an index.
```

## you should see

- A short table: each kind, and "a page" or "an index" with one reason.
- Most kinds need an index; what the team asks about again and again gets a page.

---

instibrain is made by WebWork AI, Inc. Everyone uses the same models · the difference is what yours knows.