lesson 7 of 9 · how a model works

Training, in four stages.

Training starts from random numbers and nudges every weight after each guess of the next token. Four stages turn that into the chat model you use.

new words

Loss: One number for how wrong a guess was. Training makes it smaller.
Backpropagation: The way the model works out, for every weight, which direction makes the loss smaller.
Gradient descent: Moving every weight a tiny step in that direction, after every batch of text.
Base model: A model after pretraining alone. It continues text; it does not answer questions.

the lesson

People design the shape of lesson 5 and the rule. Nobody sets the numbers: they start random. The model reads a piece of text, guesses each next token, and measures how wrong it was. Backpropagation finds which way each weight should move, and gradient descent moves all of them a tiny step. Then the next piece of text.

Stage	Learns from	Makes
Pretraining	Trillions of tokens of text: guess the next one	A base model, which continues text
Supervised fine-tuning	A prompt and an ideal answer, written by people	An instruct model, which answers
Preference training	Two answers; a person, or a model following written principles, picks the better	A chat model, helpful and declining harm
Reasoning training	Problems whose answer can be checked; the try that gets it right is rewarded	A reasoning model, which works it out first

Pretraining is the big one. Meta reports that Llama 3 read over 15 trillion tokens. A common rule of thumb counts the work as 6 × weights × tokens:

6 × 8,030,000,000 × 15,000,000,000,000
= 722,700,000,000,000,000,000,000
≈ 7.2 × 10²³ operations

Meta reports 1.3 million hours of graphics-card time for Llama 3 8B and 6.4 million for 70B. 1.3 million hours on one card is 148 years; on 16,000 cards at once, 3.4 days.

"We will train our own model on our documents" can only mean stage 2: no company has trillions of tokens of its own. Stage 2 teaches a way of answering. A document is not a prompt with an ideal answer, so its facts come out vague, go stale on the first change, and name no source. Facts belong on pages the model reads: AI basics, lesson 8.

The pretraining loop. It runs until the text is read; the three later stages run the same loop on smaller, chosen examples.

try it

Type this and press Enter:

We want to train our own model on our company's documents. Which of the four training stages would that be, and what would it teach the model?

you should see

Stage 2, fine-tuning: pretraining needs trillions of tokens no company has.
It teaches format and tone, not reliable facts. The facts belong in pages the model reads, which change on save and name their source.