Chapter 4- How LLMs Learn

Chapter Goal

In the previous chapters, we built the processing side of a Large Language Model.

Chapter 1 showed how human-readable text becomes tokens:

Text
↓
Tokenizer
↓
Tokens
↓
Token IDs

Chapter 2 showed how language can be represented numerically:

Text
↓
Numerical Representation
↓
Embeddings
↓
Vectors
↓
Semantic Relationships

Chapter 3 showed how a Transformer can use Attention to turn initial token representations into context-sensitive representations:

Tokens
↓
Numerical Representations
↓
Attention
↓
Transformer Layers
↓
Contextual Representations

But there is still a major unanswered question.

Where did the model's useful parameters come from?

Why does the model behave differently for:

The capital of France is ...

than for:

The capital of Japan is ...

Why can it recognize patterns involving:

doctor → patient
Paris → France
restaurant → food
lawyer → client

And why do its internal numerical transformations produce useful results instead of random numbers?

The answer is training.

In this chapter, we will study how an LLM learns its parameters from data.

By the end of the chapter, you should understand:

• what training means;
• what training data is;
• what a learning objective is;
• how next-token prediction works during pre-training;
• what prediction error means;
• how parameter updates improve the model;
• what pre-training is;
• what a base model is;
• what a learned representation is;
• the difference between training and inference;
• why model parameters normally remain fixed during ordinary inference;
• what kinds of patterns an LLM can learn;
• how frequency, co-occurrence, context, consistency, and data quality can shape learned associations;
• and why learning language patterns is not the same as storing a perfect database of facts.

The central learning loop of this chapter is:

Training Data
↓
Prediction
↓
Compare with Correct Target
↓
Prediction Error
↓
Parameter Update
↓
Repeat

1. The Missing Piece: Where Do the Parameters Come From?

In Chapter 3, we introduced parameters.

A parameter is a numerical value inside the model that influences its calculations.

A simplified model might contain values such as:

0.213
-0.842
1.107
0.036
...

A real Large Language Model can contain billions of such values.

These parameters participate in the enormous number of calculations performed inside the model.

Consider:

The capital of France is ...

Eventually, the model may produce something conceptually like:

Paris   → high probability
London  → lower probability
Berlin  → lower probability
Tokyo   → very low probability

Many learned parameters contributed to the calculations that produced these predictions.

But the model was not manually created with billions of carefully chosen numbers.

Nobody sits down and writes:

Parameter 1 = 0.382
Parameter 2 = -0.194
Parameter 3 = 0.772
...

until the model knows language.

Instead, the parameters are learned through training.

2. What Is Training?

Training is the process of adjusting a model's parameters so that the model becomes better at a defined task.

At a very high level:

Model makes prediction
↓
Prediction is evaluated
↓
Model receives information about its error
↓
Parameters are adjusted
↓
Model tries again

This happens repeatedly.

Not once.

Not a thousand times.

But across enormous quantities of training examples.

Input
↓
Prediction
↓
Error
↓
Parameter Update
↓
New Input
↓
Prediction
↓
Error
↓
Parameter Update
↓
...

The model therefore improves gradually through many small numerical adjustments.

Training changes the model itself by changing its learned parameters.

3. Training Data

The model needs examples from which it can learn.

These examples are called training data.

For a language model, training data can include very large amounts of text.

Depending on the model and training process, sources may include things such as:

• web pages;
• books;
• articles;
• reference material;
• code;
• conversations;
• documents;
• and other text datasets.

The exact training data differs between models.

We should therefore avoid assuming that every LLM was trained on exactly the same information.

Large Collection of Language
↓
Training Examples
↓
Model Learns Statistical Patterns

Consider a tiny fictional training dataset:

Paris is the capital of France.

France is a country in Europe.

The Eiffel Tower is located in Paris.

Tokyo is the capital of Japan.

Japan is a country in Asia.

Even this tiny set contains several patterns and relationships:

Paris ↔ France
Tokyo ↔ Japan
capital ↔ country
Paris ↔ Eiffel Tower

Now imagine training across an enormous and diverse collection of language.

The number of possible patterns becomes extremely large.

Training Data Is Not Simply Copied into the Model

A common misunderstanding is:

An LLM is just a giant database containing copies of its training documents.

That is not a good mental model.

Training does not normally work like:

Document
↓
Store document inside parameter database
↓
Retrieve exact document later

Instead, training uses data to adjust a very large network of numerical parameters.

Training Text
↓
Model Processing
↓
Prediction Error
↓
Parameter Changes

The resulting model contains learned numerical structure.

This structure can capture:

• language patterns;
• conceptual relationships;
• entity associations;
• grammatical structure;
• factual associations;
• and reasoning patterns.

This does not mean memorization never occurs.

Models can sometimes memorize particular sequences or information.

Training
≠
simply storing every document

4. The Model Needs a Learning Objective

If we want a model to learn, we need some way to define:

What does “better” mean?

This is the role of the learning objective.

A learning objective defines what the model is being trained to improve.

For many modern autoregressive language models, the core pre-training objective is conceptually simple:

Predict the next token given the previous tokens.

For example:

The capital of France is ___

The correct continuation in the training text may be:

Paris

Or:

The dog ran across the ___

might continue with:

street

Or:

She opened her umbrella because it started to ___

might continue with:

rain

The training process rewards the model for assigning more probability to the correct next token.

5. Next-Token Prediction

Suppose the training text contains:

The capital of France is Paris.

At one training step, the model may receive:

The capital of France is

and must predict the next token.

Suppose the model produces this simplified distribution:

London  → 35%
Paris   → 25%
Berlin  → 15%
Rome    → 10%
Other   → 15%

These values are illustrative.

The actual correct next token in the training data is:

Paris

The model did not completely fail.

It gave Paris some probability.

But it gave London more.

The training system therefore has evidence that the model should change.

Conceptually, the goal is to move from something like:

London  → 35%
Paris   → 25%

toward something more like:

Paris   → 90%
London  → 2%

for this kind of context.

One Sentence Creates Many Training Opportunities

Consider:

The capital of France is Paris.

The model does not necessarily learn only from the final prediction:

The capital of France is → Paris

The sequence can provide multiple next-token prediction opportunities.

Simplified:

The → capital

The capital → of

The capital of → France

The capital of France → is

The capital of France is → Paris

Real tokenization may split the text differently.

The important idea is that language sequences can provide many prediction targets.

Across enormous text collections, this creates a massive number of learning examples.

6. Prediction Error and Parameter Updates

Prediction Error

The model's prediction is compared with the actual next token in the training data.

Model Prediction
+
Correct Token
↓
Prediction Error

Suppose:

Input:
The capital of France is

Correct next token:
Paris

But the model strongly predicted:

London

The model's prediction was poor.

The training system calculates a numerical measure of how poor the prediction was.

This is commonly called loss.

We do not need the detailed mathematics of the loss function in this course.

Good Prediction
→ lower error

Bad Prediction
→ higher error

What Does the Error Actually Do?

The prediction error does not simply tell the model:

Wrong.

Training needs something more useful.

It needs information that can eventually answer:

Which parameters should change, and in what direction, so that predictions improve?

The final prediction was produced through many calculations:

Input
↓
Layer
↓
Layer
↓
Attention
↓
More Layers
↓
Final Representation
↓
Next-Token Prediction

Parameters participated throughout these calculations.

Training works backward from the prediction error to determine how small changes to those parameters would affect the error.

Prediction Error
↓
Determine how parameters contributed
↓
Calculate useful adjustments
↓
Update parameters

The detailed mathematical process involves concepts such as gradients and backpropagation.

We do not need to study those mathematically here.

The error provides a training signal that allows the system to determine how the parameters should be adjusted.

Parameter Update

A parameter update is a change to one or more model parameters during training.

Suppose one parameter currently has the value:

0.4821

After a training update, it might become:

0.4824

Another parameter might change from:

-0.7318

to:

-0.7312

These individual changes can be extremely small.

But the model may contain billions of parameters, and training performs enormous numbers of updates.

Before Training Step

Parameter A = 0.4821
Parameter B = -0.7318
Parameter C = 1.0427

After Training Step

Parameter A = 0.4824
Parameter B = -0.7312
Parameter C = 1.0421

The model is now slightly different.

Its future calculations may therefore be slightly different.

Tiny Parameter Changes
×
Huge Number of Training Steps
↓
Major Learned Capabilities

Parameter Updates Are Distributed

It would usually be wrong to imagine:

Parameter 481,927
=
France → Paris

A factual association such as:

France ↔ Paris

is generally not stored in one neatly identifiable parameter.

Instead, model behavior emerges from distributed interactions among many parameters.

Many Parameters
+
Many Layers
+
Many Learned Representations
+
Current Context
↓
Model Prediction

This is one reason LLM knowledge does not behave exactly like records in a traditional database.

7. The Training Loop

We can now describe one simplified training cycle.

Suppose the training text contains:

The capital of France is Paris.

Step 1 — Prepare the Input

Text
↓
Tokens
↓
Token IDs

Step 2 — Process the Context

The model processes:

The capital of France is

through Transformer layers.

Token Representations
↓
Attention
↓
Transformer Layers
↓
Contextual Representations

Step 3 — Predict the Next Token

London  → 35%
Paris   → 25%
Berlin  → 15%
...

Step 4 — Compare with the Correct Token

Paris

Step 5 — Calculate Prediction Error

Prediction
+
Correct Target
↓
Loss

Step 6 — Determine Parameter Adjustments

The training system determines how the model's parameters should change to reduce the error.

Step 7 — Update the Parameters

Old Parameters
↓
Small Adjustments
↓
Updated Parameters

Step 8 — Continue Training

Next Example
↓
Prediction
↓
Error
↓
Update
↓
Next Example
↓
...

This repeated process is the foundation of learning.

8. Why Next-Token Prediction Is More Powerful Than It Sounds

At first, next-token prediction may sound trivial.

It may seem like the model is simply learning:

Which word usually comes next?

But predicting the next token accurately across complex language can require much richer structure.

Consider:

The capital of France is ___

Useful prediction requires a factual association:

France ↔ Paris

Consider:

The mouse is next to the keyboard, so I moved it to the other side of the desk.

Prediction can depend on:

• context;
• word meaning;
• references;
• relationships;
• and sentence structure.

Consider:

If John is older than Mary, and Mary is older than Sam, then John is ___ than Sam.

A strong continuation may require a learned reasoning pattern.

So although the learning objective is:

predict next token

becoming very good at that objective across diverse text encourages the model to learn much richer internal structure.

9. What Does an LLM Learn?

Language Patterns

One of the things an LLM learns is language patterns.

She is walking.

He is running.

The company is growing.

The model can learn recurring structures involving:

• grammar;
• syntax;
• word order;
• phrasing;
• style;
• and sentence structure.

Consider:

The doctors are ...

versus:

The doctor is ...

Repeated exposure to language allows the model to learn statistical patterns involving grammatical agreement.

No person needs to manually program every grammatical rule into the model.

Concepts

The model can also develop useful internal representations associated with concepts.

doctor
hospital
patient
medicine
treatment
nurse

These frequently appear in related contexts.

Across training data, the model encounters many relationships among them.

Similarly:

restaurant
menu
food
chef
reservation
dinner

form another broad semantic region.

This connects back to Chapter 2.

Embeddings and other numerical representations can reflect these learned semantic relationships.

Entities

Training data contains references to specific real-world or fictional things.

These are entities.

Examples include:

Paris
France
Microsoft
Albert Einstein
Amazon
The Eiffel Tower

Suppose the model repeatedly encounters patterns such as:

Paris is in France.

Paris is the capital of France.

The Eiffel Tower is located in Paris.

France is a European country.

Through training, the model may develop useful associations involving:

Paris
France
capital
Europe
Eiffel Tower

This does not mean the model has constructed a perfectly explicit database entry saying:

Entity: Paris
Type: City
Country: France
Capital: Yes

The learned representation is more distributed.

Relationships

Language constantly expresses relationships.

Paris → capital of → France

doctor → treats → patient

author → writes → book

company → produces → product

restaurant → located in → city

A language model can learn statistical structure reflecting many such relationships.

Recognizing these relationships improves prediction.

Relationships expressed repeatedly in language can influence the model's learned parameters and representations.

Statistical Associations

Not every learned association should be interpreted as a precise fact.

Some are better understood as statistical associations.

rain ↔ umbrella

hospital ↔ doctor

restaurant ↔ menu

airport ↔ flight

beach ↔ ocean

These concepts frequently appear in related contexts.

For example:

It started raining, so she opened her ___

The model may strongly favor:

umbrella

because of learned statistical relationships in language.

This is not necessarily a fact lookup.

It is a learned pattern.

Factual Associations

Some learned patterns correspond to factual relationships.

France ↔ Paris

Japan ↔ Tokyo

Earth ↔ Sun

water ↔ H₂O

These can be called factual associations.

Training data may repeatedly express these relationships in different forms:

Paris is the capital of France.

France's capital is Paris.

The French capital, Paris, ...

Paris, the capital city of France, ...

Repeated exposure strengthens the model's ability to predict language consistent with that association.

learned factual association
≠
guaranteed factual correctness

Reasoning Patterns

Training data can also contain recurring reasoning patterns.

All mammals breathe air.

Whales are mammals.

Therefore, whales breathe air.

Or:

If the shop closes at 8 PM
and it is now 9 PM,
the shop is closed.

Or:

John is taller than Mary.
Mary is taller than Sam.
Therefore, John is taller than Sam.

The model can encounter enormous numbers of examples containing:

• comparison;
• cause and effect;
• deduction;
• classification;
• analogy;
• and multi-step explanation.

Training can therefore create representations and parameter configurations that support reasoning-like behavior.

This does not mean there is necessarily one component called a Reasoning Module.

The capability emerges from the interaction of learned parameters, representations, Transformer layers, and context.

10. Learned Representations

A learned representation is a numerical representation whose useful structure emerges through training.

Before training, numerical values inside a model are not yet organized to represent language usefully.

Before Training

Numbers
↓
little useful language structure

After training:

Learned Parameters
↓
Useful Numerical Representations
↓
Patterns involving meaning, context, entities, relationships, etc.

This connects directly to Chapter 2.

When we discussed embeddings, we said that useful semantic relationships are represented numerically.

Now we can answer:

Where did those useful numerical relationships come from?

They are learned through training.

11. Pre-Training and Base Models

Pre-Training

The large-scale initial learning stage for an LLM is commonly called pre-training.

During pre-training, a model is trained on very large datasets to learn broad patterns of language.

Large Training Dataset
↓
Next-Token Prediction
↓
Prediction Error
↓
Parameter Updates
↓
Repeat at Massive Scale
↓
Pre-Trained Model

Why is it called pre-training rather than simply training?

Because modern AI assistants often go through additional training stages afterward.

The initial stage teaches broad language modeling capability.

Later stages can shape behavior such as:

• instruction following;
• assistant behavior;
• helpfulness;
• format preferences;
• and safety behavior.

Those later processes belong primarily to Chapter 6.

Pre-training is the large-scale foundational training stage that produces broad language-model capabilities.

The Base Model

The result of broad pre-training is commonly called a base model.

Model Architecture
+
Large-Scale Pre-Training
↓
Base Model

The base model has learned a large amount of language structure.

It may know how to continue text impressively.

But a base model is not automatically a polished conversational assistant.

Base Model
≠
Finished Assistant

Later post-training stages help turn general language modeling capability into assistant behavior.

Pre-Training Does Not Mean Perfect Knowledge

A large pre-trained model may learn an enormous number of useful associations.

But pre-training has limitations.

Training data may contain:

• incorrect information;
• outdated information;
• contradictions;
• biases;
• missing information;
• rare information;
• and ambiguous information.

Large Training Dataset
≠
Perfect Knowledge

Learned Association
≠
Verified Fact

These limitations eventually motivate systems involving retrieval, RAG, web search, evidence, grounding, and citations.

12. Training vs. Inference

This is one of the most important distinctions in the entire textbook.

Training and inference are different processes.

Training

Input
↓
Prediction
↓
Prediction Error
↓
Parameter Update

The model's parameters change.

Inference

Inference is what happens when an already-trained model processes new input and produces output.

Suppose you ask:

What is the capital of France?

The model processes your input and generates a response.

User Input
↓
Existing Learned Parameters
↓
Transformer Processing
↓
Output

During normal inference:

no ordinary training loop
no prediction target from a training dataset
no standard parameter update

The model uses what it has already learned.

Training
→ change the model

Inference
→ use the model

Static Parameters During Normal Inference

Suppose the model processes:

The bank approved my loan.

Its internal representation of bank may become financial.

Now it processes:

The fisherman sat on the river bank.

The contextual representation of bank changes.

But this does not mean the model's billions of learned parameters have been retrained.

Same Parameters
+
Different Context
↓
Different Contextual Representations

Dynamic Representation
≠
Parameter Update

Why a Conversation Can Change Behavior Without Training the Model

Suppose you tell an assistant:

For the rest of this conversation, when I say "Project X",
I mean our Toronto restaurant dataset.

Later you ask:

What should we improve in Project X?

The model can use the information from the current context.

Its behavior has changed.

But that does not necessarily mean its core model parameters changed.

New Context
↓
Different Contextual Representations
↓
Different Output

This is in-context processing, not ordinary parameter training.

A model can adapt its output to new information in the current context without permanently learning that information into its parameters.

Training Is Slow Learning; Context Is Immediate Information

Training
→ modifies long-term learned parameters

Context
→ provides temporary information for current inference

A useful analogy is:

Training
= changing the machine

Context
= giving the machine new material to process

Suppose a model was trained several months ago.

Today you provide it with a newly published document.

If that document is inserted into the context, the model can reason over it even though the information was not part of its original training.

That is one of the foundations of retrieval-augmented systems.

13. Generalization and Memorization

Generalization

Training would be much less useful if the model could only reproduce examples it had seen exactly before.

A major capability is generalization.

Generalization means using learned patterns successfully on new examples that were not encountered in exactly the same form during training.

Suppose a model has learned many examples involving patterns like:

X is larger than Y.
Y is larger than Z.
Therefore X is larger than Z.

Now it receives:

Building A is taller than Building B.
Building B is taller than Building C.
Which building is tallest?

If the model applies the underlying pattern correctly to this new example, that is an example of generalization.

Many Training Examples
↓
Learn General Pattern
↓
Apply Pattern to New Input

Memorization

Models can also memorize information.

Memorization means retaining specific patterns or sequences from training data closely enough that they can later be reproduced or strongly influence output.

For example, a frequently repeated phrase or famous passage may be strongly represented.

So model behavior can involve a mixture of:

memorization
+
statistical learning
+
generalization
+
contextual computation

LLMs only memorize everything. ❌

LLMs never memorize anything. ❌

The reality is more complex.

14. What Shapes Learned Associations?

When an LLM learns an association between concepts, entities, words, or relationships, several properties of the training data and training process can influence how strongly and reliably that association is learned.

Important factors include:

• frequency;
• co-occurrence;
• context diversity;
• consistency;
• data quality;
• training procedure;
• and model capacity.

Frequency

Frequency refers to how often a pattern or relationship appears in the training data.

Suppose a model encounters:

Paris → capital of France

many times and in multiple linguistic forms:

Paris is the capital of France.

France's capital is Paris.

The French capital is Paris.

Paris, the capital city of France, ...

Repeated exposure can strengthen the learned association.

However:

More mentions
≠
guaranteed stronger or correct knowledge

Frequency is one factor, not the complete explanation.

Co-Occurrence

Co-occurrence means that two or more pieces of information repeatedly appear together or within related contexts.

For example:

doctor ↔ hospital

Paris ↔ France

restaurant ↔ menu

rain ↔ umbrella

If doctor repeatedly occurs in contexts containing words such as:

hospital
patient
treatment
medicine
nurse

the training process receives repeated evidence that these concepts are related.

Co-occurrence does not require two words to be immediately next to each other.

They may appear within the same sentence, passage, topic, or recurring contextual pattern.

For example:

The doctor examined the patient at the hospital.

Patients were treated by doctors in the emergency department.

The hospital hired several new physicians.

Across many examples, related concepts appear in overlapping contexts.

This helps the model develop learned associations among them.

But co-occurrence alone does not prove a specific relationship.

For example:

coffee ↔ laptop

might frequently co-occur in articles about working in cafés.

That does not mean:

coffee is a type of laptop ❌

The surrounding context and the relationships expressed in the text matter.

Co-occurrence provides statistical evidence of association, but context helps determine what the association means.

Context Diversity

A relationship appearing in many different contexts can provide richer learning signals than the exact same sentence being repeated.

Compare:

Paris is the capital of France.
Paris is the capital of France.
Paris is the capital of France.

with:

Paris is the capital of France.

France's government is based in Paris.

Paris is the country's capital city.

The French capital attracts millions of visitors.

Paris, located in France, serves as the national capital.

The second group expresses related information through different linguistic and contextual structures.

This can provide broader evidence for the relationship.

Consistency

Consistency refers to whether training examples repeatedly support the same relationship or contain conflicting signals.

Suppose many documents say:

Company X was founded in 2012.

but other documents say:

Company X was founded in 2013.

The model receives competing signals.

The training process does not automatically perform perfect fact verification and decide which source is authoritative.

Conflicting information can therefore produce weaker, mixed, or inconsistent learned associations.

Data Quality

The quality of the training examples also matters.

Training data can contain:

• accurate information;
• incorrect information;
• clear explanations;
• ambiguous wording;
• high-quality sources;
• low-quality sources;
• duplicated material;
• and contradictory claims.

The model learns from the data it receives; it does not automatically know which sentence should be trusted simply because it appears in training data.

Training Procedure

What the model learns also depends on how training is performed.

Factors such as the objective, data selection, optimization process, and later training stages influence the final model.

Two models exposed to similar information can still develop different behavior because their training processes are different.

Model Capacity

Model capacity refers broadly to how much complex structure a model is capable of representing and learning.

A larger or more capable model may be able to represent patterns that a much smaller model cannot capture as effectively.

However:

Larger model
≠
automatically correct model

Capacity enables learning; it does not guarantee accuracy.

Putting the Factors Together

A useful conceptual model is:

Frequency
+
Co-Occurrence
+
Context Diversity
+
Consistency
+
Data Quality
+
Training Procedure
+
Model Capacity
↓
Shape Learned Associations

This should not be interpreted as a universal formula.

These are factors that help explain how statistical patterns in training data can influence what the model eventually learns.

15. Training Cutoff

A model's parameters reflect information available during its training process.

If an event happens after the model's training data period, the model cannot have learned that event through its original pre-training.

Training Data
ends at Time T

New Event
happens at Time T + 1

Original Parameters
cannot contain that event
from pre-training

This is often described using the idea of a training cutoff or knowledge cutoff.

Later, retrieval systems can supply newer information at inference time.

We will study that properly in Chapter 13.

One Model, Many Learned Associations

A useful way to imagine the scale of training is:

language
entities
relationships
grammar
style
facts
reasoning patterns
code patterns
world descriptions

all contributing to one enormous optimization process.

The parameters are not normally divided into clean sections such as:

Grammar Parameters
France Parameters
Restaurant Parameters
Reasoning Parameters

Instead, representations and capabilities are distributed.

The same parameter can participate in many different computations.

Likewise, one behavior can depend on many parameters.

16. A Complete Simplified Training Example

Let us connect everything with one training example.

The training text is:

The capital of France is Paris.

Step 1 — Text Becomes Tokens

Text
↓
Tokenizer
↓
Tokens
↓
Token IDs

Simplified:

The
capital
of
France
is
Paris
.

Real tokenization may differ.

Step 2 — Tokens Become Numerical Representations

Token IDs
↓
Initial Numerical Representations

Step 3 — Transformer Processing

The context:

The capital of France is

is processed through Transformer layers.

Representations
↓
Self-Attention
↓
Updated Representations
↓
More Transformer Layers
↓
Contextual Representations

Step 4 — The Model Predicts the Next Token

London  → 40%
Paris   → 30%
Berlin  → 15%
Rome    → 10%
Other   → 5%

These numbers are illustrative.

Step 5 — The Training System Knows the Correct Target

Paris

Step 6 — Prediction Error Is Calculated

Predicted Distribution
+
Correct Target
↓
Loss

Step 7 — Parameter Contributions Are Evaluated

The training system determines how small parameter changes could reduce the error.

Step 8 — Parameters Are Updated

Parameter A
0.5123 → 0.5127

Parameter B
-0.3811 → -0.3808

Parameter C
0.0245 → 0.0241

These values are purely illustrative.

Step 9 — Training Continues

Prediction
↓
Error
↓
Parameter Adjustment
↓
Prediction
↓
Error
↓
Parameter Adjustment
↓
...

Over enormous numbers of updates, the model gradually becomes much better at modeling language.

17. From Training to Learned Behavior

After large-scale training, the model's parameters have been shaped by an enormous number of prediction tasks.

Now consider the trained model during inference:

The capital of France is ...

It may produce:

Paris

with high probability.

Why?

Not because someone manually coded:

if input contains "capital of France":
    output "Paris"

Instead:

Training Data
↓
Enormous Number of Prediction Signals
↓
Parameter Optimization
↓
Learned Representations and Associations
↓
Current Context Processing
↓
Paris becomes likely

The behavior was learned, not manually programmed as a fixed rule.

What Exactly Has the Model Learned?

These categories overlap.

They are not necessarily stored in separate parts of the model.

They represent useful ways for us to describe the patterns learned through training.

18. Training Does Not Guarantee Truth

This distinction deserves special emphasis.

The fundamental training objective we have studied is:

predict likely next token

It is not:

prove that every statement is factually true

A model can therefore generate something that is:

• grammatically correct;
• plausible;
• confident-sounding;
• statistically likely;

and still be factually wrong.

Probability of a token
≠
Probability that a factual claim is true

We will study this distinction in detail in Chapter 5.

It eventually leads to important concepts such as:

• uncertainty;
• hallucination;
• grounding;
• evidence;
• retrieval;
• and citations.

19. Common Misunderstandings

Misunderstanding 1: Training means uploading facts into a database

Incorrect. Training changes numerical parameters through optimization.

Misunderstanding 2: One parameter equals one fact

Incorrect. Knowledge and capabilities are generally distributed across many parameters and computations.

Misunderstanding 3: The model is manually taught every grammar rule

Incorrect. Many patterns emerge from learning across large amounts of language.

Misunderstanding 4: Next-token prediction only learns which words often appear next to each other

Incorrect. Accurate prediction across complex language benefits from learning concepts, relationships, context, grammar, factual associations, and reasoning patterns.

Misunderstanding 5: Training data is stored exactly inside the model

Incorrect as a general mental model. Training primarily modifies parameters, although models can also memorize some specific patterns.

Misunderstanding 6: Every time I send a message to an LLM, it retrains itself

Incorrect for ordinary inference. The model normally processes the new context using existing parameters.

Misunderstanding 7: If the model changes its answer after I provide more context, its parameters changed

Incorrect. Its contextual representations and input changed. Its core parameters can remain the same.

Misunderstanding 8: A base model is the same as a finished assistant

Incorrect. A base model has broad pre-trained language capability. Additional post-training can shape instruction-following and assistant behavior.

Misunderstanding 9: If a model saw something many times, it must always answer correctly

Incorrect. Frequency can influence learning, but co-occurrence, context, consistency, data quality, conflicting information, model limitations, and other factors also matter.

Misunderstanding 10: Frequent co-occurrence proves two concepts have a specific relationship

Incorrect. Co-occurrence provides evidence of association, but the surrounding context helps determine the nature of that relationship.

Misunderstanding 11: A model that predicts a factual statement confidently must know that the statement is true

Incorrect. Next-token probability and factual truth are different things.

20. How the Concepts Connect

We can now connect Chapters 1 through 4.

Chapter 1:

Human Text
↓
Tokenization
↓
Tokens
↓
Token IDs

Chapter 2:

Text
↓
Numerical Representation
↓
Embeddings
↓
Vectors
↓
Semantic Relationships

Chapter 3:

Token Representations
↓
Attention
↓
Q / K / V
↓
Transformer Layers
↓
Contextual Representations

Chapter 4 adds the learning process:

Training Data
↓
Next-Token Prediction
↓
Prediction Error
↓
Parameter Update
↓
Repeated Optimization
↓
Learned Parameters

Now combine them:

Training Text
↓
Tokenization
↓
Tokens
↓
Token IDs
↓
Numerical Representations
↓
Transformer Processing
↓
Contextual Representations
↓
Next-Token Prediction
↓
Compare with Correct Token
↓
Prediction Error
↓
Parameter Update
↓
Repeat

After training:

Learned Parameters
+
New Input
↓
Inference
↓
Transformer Processing
↓
Next-Token Predictions
↓
Generated Output

Mental Model

Imagine a gigantic machine containing billions of adjustable numerical controls.

At first, those controls are not arranged in a useful way.

The machine is repeatedly shown language.

For each example:

read context
↓
predict next token
↓
compare prediction with actual token
↓
measure error
↓
adjust many controls slightly

It repeats this process again and again.

Gradually, the configuration of those controls becomes useful for recognizing and generating language patterns.

The machine begins to capture relationships such as:

Paris ↔ France

rain ↔ umbrella

doctor ↔ patient

question ↔ answer

cause ↔ effect

Not because someone manually entered each relationship.

But because configurations that better modeled these patterns produced better predictions during training.

The analogy has limits.

Real LLM training involves high-dimensional representations, matrix operations, Transformer layers, optimization algorithms, and enormous computation.

But the core mental model is correct:

Training repeatedly compares predictions with real data and changes the model's parameters so that future predictions improve.

And one additional distinction should now be permanent in your mental model:

Training
→ changes parameters

Inference
→ uses parameters

Chapter 4 Recap

Training

The process of adjusting a model's parameters so that it becomes better at its learning objective.

Training Data

The examples used during the model's learning process.

Parameter

A learned numerical value that influences the model's computations.

Parameter Update

A numerical adjustment made to model parameters during training.

Learning Objective

The criterion that defines what the model is trying to improve during training.

Next-Token Prediction

Predicting the next token based on the preceding context.

Prediction Error

A numerical measure of how poorly the model's prediction matched the training target. This is commonly represented through a loss.

Pre-Training

The large-scale foundational training stage in which the model learns broad language patterns from large datasets.

Base Model

A general language model produced through broad pre-training before later assistant-oriented post-training.

Learned Representation

A numerical representation whose useful structure has emerged through training.

Language Patterns

Recurring structures involving grammar, syntax, phrasing, word order, and other linguistic behavior.

Concepts

Learned semantic structures involving recurring ideas and categories.

Entities

Specific people, organizations, locations, products, objects, or other identifiable things represented in language.

Relationships

Learned statistical structure connecting concepts and entities.

Statistical Associations

Recurring relationships and correlations learned from patterns in training data.

Co-Occurrence

The repeated appearance of concepts, entities, or terms together or within related contexts. Co-occurrence can provide evidence of association, but does not by itself define the meaning of the relationship.

Context Diversity

The variety of contexts in which a relationship appears. Diverse contexts can provide richer evidence about a relationship than simple repetition of the same wording.

Consistency

The degree to which training examples support the same relationship rather than providing conflicting signals.

Factual Associations

Learned patterns corresponding to factual relationships expressed in training data. A learned factual association is not a guarantee of factual correctness.

Reasoning Patterns

Learned structures that can support comparison, deduction, cause-and-effect reasoning, classification, and other structured behavior.

Inference

Using an already-trained model to process new input and generate output.

Static Parameters During Ordinary Inference

During ordinary inference, the model's learned parameters generally remain unchanged while its internal contextual representations can change dynamically.

Generalization

Applying learned patterns successfully to new examples that were not seen in exactly the same form during training.

Memorization

Retaining specific information or sequences from training closely enough to reproduce or strongly reflect them.

The complete dependency through Chapter 4 is now:

Human Text
↓
Tokenization
↓
Tokens
↓
Token IDs
↓
Numerical Representations
↓
Embeddings
↓
Transformer Layers
↓
Attention
↓
Contextual Representations
↓
Next-Token Prediction
↓
Prediction Error
↓
Parameter Updates
↓
Repeated Training
↓
Learned Language Model

We now understand how the model learns its parameters.

But another major question remains.

Once training is finished and the parameters are fixed, how does the model actually produce an answer token by token?

How does it choose among possible next tokens?

Why can the same prompt produce different answers?

What does temperature do?

And why can a model produce a highly plausible answer that is completely wrong?

Those questions lead directly into Chapter 5:

How LLMs Generate Answers.

The Single Most Important Idea from Chapter 4

An LLM learns by repeatedly predicting the next token, measuring how wrong that prediction was, and adjusting a huge network of numerical parameters so that future predictions improve. Through this process, the model develops learned representations and patterns involving language, concepts, entities, relationships, factual associations, and reasoning.