What goes in the index.
A reviewed position is a page, read whole. The thousands of raw files nobody reviewed one by one are what an index is for.
new words
- Page
- One thing your team knows, reviewed, current, in a space with its readers. Your AI reads it whole, before it works.
- Raw file
- A document nobody reviewed into a page: a contract, a ticket, a mail, a meeting's notes. Thousands of them, and more each day.
- Index
- The table of chunks and vectors that a question is searched against. AI basics, lesson 6.
the lesson
Two stores, two jobs. A page says what is true now, for the people who read its space, and it is small: a brain of 400 pages at 500 tokens each is 200,000 tokens, a fifth of a current Claude model's window. Your AI reads a page whole; nothing is cut, nothing is ranked.
A firm's files are another size. 5,000 documents of 20 pages are 50 million tokens: fifty windows. Nobody reviewed each one, and nobody will. That is what an index is for: the question finds the five chunks that matter, and the rest stays on disk.
| It is | It goes |
|---|---|
| A rule, a price, a decision, a way of working | On a page |
| What a new colleague must know on day one | On a page |
| A signed contract, a closed ticket, a thread of mail | In the index |
| Three years of meeting notes | In the index |
| An answer found in the index that the team asks for again | On a page, from then on |
The test for a file: would a reviewed page replace it? A contract's notice period belongs on the client's page; the contract itself stays a file, indexed, for the day someone needs clause 14. The three levers says why: context is read whole and cited by path; retrieval is cited by chunk and fresh at the next reindex.
try it
Type this and press Enter:
List the kinds of documents my company keeps that are not pages: contracts, tickets, mail, notes, files. For each kind, say whether one reviewed page could replace it, or whether it needs an index.you should see
- A short table: each kind, and "a page" or "an index" with one reason.
- Most kinds need an index; what the team asks about again and again gets a page.