Fine-tuning versus context.
Training teaches a model how to answer. A page in its context teaches it what is true today, with a source. Facts belong on pages.
new words
- Fine-tuning
- Any further training of a finished model on your examples. It changes the weights, and teaches format and tone more than facts.
- Pretraining
- Training from nothing on a huge body of text, to predict the next token. It fixes the knowledge cutoff.
- Knowledge cutoff
- The date after which the model saw nothing. A newer fact needs retrieval or context.
- Instruction file
- A file an AI reads at the start of a session, such as CLAUDE.md. The simplest durable context.
the lesson
Nine methods change what a model knows or how it acts. Six change the weights, two fetch at question time, one tells it at the start.
| Method | Lever | What it does |
|---|---|---|
| Pretraining | weights | Trains from nothing on trillions of tokens, How a model works, lesson 7. Everything, frozen at the cutoff date. |
| Continued pretraining | weights | More pretraining on one field's text. Vocabulary and style; facts without a source. |
| Supervised fine-tuning | weights | Example answers teach format, tone and the shape of a task. Not facts. |
| Preference tuning | weights | Better-versus-worse pairs teach judgment. |
| LoRA, adapters | weights | Trains a small add-on while the base model stays frozen. Fine-tuning at a fraction of the cost. |
| Distillation | weights | A big model writes answers; a small one learns to copy them. |
| RAG | retrieval | Chunk, embed, index. Fetches the nearest chunks into the prompt, cited. Lesson 7. |
| Tools, MCP | retrieval | The model looks things up and acts, live, through structured calls. |
| Context | context | Instruction files, pages and skills, read at the start of every session. |
Deeper is costlier, slower to redo, and staler. A fine-tuned model knows nothing written after its training run, cannot say where a fact came from, and carries your data inside a vendor's weights. A page is fresh on save, is its own source, and stays in your space.
So a fact goes on a page. A way of answering goes in an instruction file first, and into a fine-tune only when a file cannot carry it. Judgment comes from a better model. Your Claude reads your pages before it works; nothing about your company is trained into it, and the same model answers another company from its own pages.
try it
Type this and press Enter:
Read the CLAUDE.md you loaded at the start and tell me one rule from it. Did that rule change anything inside the model, or only what you were told?you should see
- One rule, quoted from the file.
- Only what it was told: the rule sits in its context for this session, the model's weights are the same for everyone, and a change to the file changes the rule at the next start.