Chapter 5- How LLMs Generate Answers

Chapter Goal

In Chapter 4, we learned how a language model is trained.

At a simplified level:

Training Data
↓
Predict Next Token
↓
Compare with Correct Token
↓
Prediction Error
↓
Parameter Update
↓
Repeat

Through this process, the model develops learned parameters and representations.

After training, however, something different happens.

The model is no longer being shown a correct answer and updating its parameters.

Instead, a user provides an input, and the trained model must generate an answer.

For example:

User:
What is the capital of France?

The model may respond:

The capital of France is Paris.

But how does that actually happen?

Does the model create the entire sentence at once?

Does it search through a list of complete answers?

Does it know in advance exactly what sentence it will produce?

Why can the same model sometimes give different answers to the same question?

And how can a model generate an answer that sounds extremely confident but is factually wrong?

This chapter explains the generation process.

By the end of this chapter, you should understand:

how next-token prediction is used during inference;
what a probability distribution over tokens is;
what candidate tokens are;
what token probability means;
how an answer is generated one token at a time;
what sampling means;
the difference between deterministic and probabilistic generation;
what temperature does;
why the same prompt can produce different outputs;
what model uncertainty means;
why token probability is not the same as factual confidence;
what calibration means at a conceptual level;
what hallucination is;
what unsupported generation is;
what fabrication means;
and why plausible language can still be incorrect.

The central generation loop is:

Current Context
↓
Predict Probability of Next Tokens
↓
Select One Token
↓
Add Token to Context
↓
Predict Again
↓
Repeat

This loop is the foundation of language generation.

1. Training Is Finished — Now What?

During training, the model repeatedly predicted tokens and received feedback.

During ordinary inference, that feedback loop is absent.

The trained parameters are already available.

Now the model receives new input.

For example:

What is the capital of France?

The model processes this input using:

tokens
learned parameters
Transformer layers
Attention
contextual representations

and eventually produces a prediction for what token should come next.

The important point is:

The model does not normally generate the entire answer in one step.

It generates the answer incrementally.

Input
↓
Predict Token 1
↓
Add Token 1 to Context
↓
Predict Token 2
↓
Add Token 2 to Context
↓
Predict Token 3
↓
...

This is called autoregressive generation.

2. Autoregressive Generation

An autoregressive language model generates new tokens based on the tokens that already exist in the context.

Suppose the user asks:

What is the capital of France?

The model might first predict:

The

Now the context becomes:

What is the capital of France?

The

The model predicts again.

Perhaps:

capital

Now:

What is the capital of France?

The capital

The model predicts again:

of

Then:

France

Then:

is

Then:

Paris

Then perhaps punctuation:

So the answer emerges incrementally:

The
↓
The capital
↓
The capital of
↓
The capital of France
↓
The capital of France is
↓
The capital of France is Paris
↓
The capital of France is Paris.

At every step, the already-generated output becomes part of the context for the next prediction.

3. The Model Does Not Choose from Complete Answers

A useful misconception to remove is this:

Question
↓
Database of Complete Answers
↓
Choose One Answer

That is not the normal generation process.

The model is instead repeatedly making next-token predictions.

Context
↓
Possible Next Tokens
↓
Choose One
↓
Updated Context
↓
Possible Next Tokens
↓
Choose One
↓
...

The full answer emerges from these repeated local decisions.

This is why one early token can influence everything that comes afterward.

If the model chooses a different word near the beginning, the rest of the answer may follow a different path.

4. Probability Distribution

At each generation step, the model does not normally produce only one possible token.

Instead, it assigns scores to many possible tokens.

Those scores are transformed into a probability distribution.

A probability distribution assigns probabilities across possible outcomes.

For example, suppose the current context is:

The capital of France is

The model might produce an illustrative distribution like:

Paris      → 92%
located    → 2%
the        → 1.5%
in         → 1%
Lyon       → 0.5%
London     → 0.2%
other      → remaining probability

These numbers are purely illustrative.

The important idea is:

Candidate Tokens
+
Probabilities
=
Probability Distribution

Across all possible next tokens, the probabilities approximately sum to:

100%

or mathematically:

1.0

5. Candidate Tokens

A candidate token is a token that could potentially be selected as the next output token.

Suppose the context is:

She opened the umbrella because it started to

Possible candidate tokens might include:

rain
snow
fall
get
pour
...

Each receives some probability.

Illustratively:

rain   → 64%
pour   → 12%
snow   → 4%
fall   → 3%
get    → 2%
...

The model then needs a mechanism for selecting one candidate.

That selection process is part of generation.

6. Token Probability

A token probability represents how strongly the model predicts a token as the next token given the current context.

Conceptually:

P(next token | current context)

You do not need the mathematical notation.

The important idea is:

Token probability is conditional on the current context.

For example:

The capital of France is

might give Paris a very high probability.

But:

I traveled from Paris to

creates a different context.

Now possible continuations could include:

London
Berlin
Rome
Lyon
...

Probability of token
depends on
current context

The token itself does not have one permanent probability.

7. Token Probabilities Change After Every Generated Token

Suppose the model begins with:

Artificial intelligence is

It predicts one token.

Imagine it selects:

changing

Now the context becomes:

Artificial intelligence is changing

The next probability distribution is different.

Possible continuations might now include:

how
the
rapidly
many
...

If it selects:

how

the context becomes:

Artificial intelligence is changing how

Now the next predictions change again.

Context A
↓
Probability Distribution A
↓
Selected Token
↓
Context B
↓
Probability Distribution B
↓
Selected Token
↓
Context C
...

Generation is therefore a dynamic process.

Every selected token changes the future context.

8. Generation as a Path Through Possibilities

One useful mental model is to imagine a branching tree.

Start with:

The restaurant was

Possible next tokens:

excellent
closed
busy
located
serving
...

If the model chooses:

excellent

new possibilities appear:

The restaurant was excellent and ...
The restaurant was excellent because ...
The restaurant was excellent, especially ...

If it instead chooses:

closed

a completely different path develops:

The restaurant was closed when ...
The restaurant was closed because ...
The restaurant was closed for ...

Current Context
          ↓
     ┌────┼────┐
     ↓    ↓    ↓
 Token A Token B Token C
     ↓
 new possibilities

The model does not normally explore every complete branch before deciding.

It repeatedly chooses the next token and continues from there.

9. How Is the Next Token Selected?

Suppose the model predicts:

Paris   → 70%
Lyon    → 10%
London  → 5%
Rome    → 3%
Other   → 12%

There are different ways to choose the next token.

One simple approach is:

Always choose the token with the highest probability.

In this example:

Paris

would always be selected.

But generation systems can also use sampling.

10. What Is Sampling?

Sampling means selecting a token from a probability distribution rather than automatically selecting only the highest-probability token.

Suppose:

Paris   → 70%
Lyon    → 15%
London  → 8%
Rome    → 4%
Other   → 3%

With sampling, Paris is still much more likely to be selected.

But another candidate may occasionally be chosen.

A useful analogy is a weighted lottery.

Imagine 100 tickets:

70 Paris tickets
15 Lyon tickets
8 London tickets
4 Rome tickets
3 other tickets

One ticket is selected.

Paris is most likely.

But it is not absolutely guaranteed.

This analogy is simplified, but it captures the basic principle.

11. Deterministic vs. Probabilistic Generation

Deterministic Generation

In a simplified deterministic strategy, the system consistently chooses the highest-probability token.

Paris   → 70%
Lyon    → 15%
London  → 8%

Choose:

Paris

If everything else remains identical, the same choices can produce the same output.

Highest Probability Token
↓
Selected

Probabilistic Generation

In probabilistic generation, the distribution influences selection, but the highest-probability token is not necessarily chosen every time.

Probability Distribution
↓
Sampling
↓
Selected Token

This introduces variability.

As a result:

Same Prompt
+
Same Model

can sometimes produce:

Different Output A
Different Output B
Different Output C

especially when sampling is enabled.

12. Why Randomness Can Be Useful

At first, randomness may sound undesirable.

Why not always choose the highest-probability token?

Because always selecting the most likely token can sometimes produce text that is:

repetitive
overly predictable
rigid
less diverse

Controlled probabilistic selection can make generation more flexible.

This is particularly useful for tasks such as:

creative writing
brainstorming
alternative wording
dialogue
idea generation

However, more randomness can also increase the chance of unusual or undesirable outputs.

Generation therefore involves a trade-off between:

predictability
and
diversity

One important control over that trade-off is temperature.

13. Temperature

Temperature is a generation setting that changes the shape of the model's token probability distribution before token selection.

At a conceptual level:

Lower Temperature
→ probability distribution becomes sharper

Higher Temperature
→ probability distribution becomes flatter

A sharper distribution strongly favors the already-high-probability tokens.

A flatter distribution gives lower-probability alternatives more opportunity to be selected.

Low Temperature

Suppose the original distribution is:

Paris   → 70%
Lyon    → 15%
London  → 8%
Rome    → 4%
Other   → 3%

At a lower temperature, the effective distribution may become conceptually more like:

Paris   → very dominant
Lyon    → much lower
London  → very low
Rome    → extremely low

The exact numbers depend on the implementation.

Lower Temperature
↓
More concentration on likely tokens
↓
More predictable output

Low-temperature generation often tends to be more:

stable
conservative
repeatable
focused

But low temperature does not guarantee factual correctness.

High Temperature

At a higher temperature, the probability distribution becomes less concentrated.

Original:

Paris   → 70%
Lyon    → 15%
London  → 8%
Rome    → 4%

Higher Temperature:

Paris   → less dominant
Lyon    → relatively more competitive
London  → relatively more competitive
Rome    → relatively more competitive

This means alternative tokens become more likely to be selected.

The resulting text may become:

more varied
more creative
less predictable
more surprising

but potentially also:

less stable
less coherent
more error-prone

Temperature Does Not Change What the Model Learned

Changing temperature does not retrain the model.

It does not alter the model's learned parameters.

Same Model
+
Same Learned Parameters
+
Different Generation Setting
↓
Different Token Selection Behavior

Training
→ changes parameters

Temperature
→ changes generation behavior

Temperature Is Not Creativity Stored Inside the Model

Another common misunderstanding is:

Temperature = creativity level inside the model

That is too simplistic.

Temperature changes the relative likelihood of candidate tokens during generation.

Higher temperature can produce more varied output because lower-probability candidates become more likely.

Higher temperature
≠
more intelligent

Higher temperature
≠
more knowledgeable

Higher temperature
≠
better reasoning

It primarily changes the randomness and diversity of token selection.

14. A Complete Simplified Generation Example

Suppose the user asks:

What is the capital of France?

Step 1 — Tokenize the Input

User Text
↓
Tokenizer
↓
Tokens
↓
Token IDs

Step 2 — Build Numerical Representations

Token IDs
↓
Numerical Representations

Step 3 — Transformer Processing

Representations
↓
Attention
↓
Transformer Layers
↓
Contextual Representations

Step 4 — Predict the Next Token

The model produces a probability distribution.

Illustratively:

The      → 38%
Paris    → 22%
France   → 6%
It       → 4%
...

Suppose:

The

is selected.

Step 5 — Update the Context

What is the capital of France?

The

becomes the context.

Step 6 — Predict Again

Possible next tokens:

capital  → 76%
city     → 8%
answer   → 3%
...

Suppose:

capital

is selected.

Step 7 — Repeat

The capital
↓
The capital of
↓
The capital of France
↓
The capital of France is
↓
The capital of France is Paris
↓
The capital of France is Paris.

The entire answer emerges one token at a time.

15. Generation Depends on the Entire Available Context

When predicting the next token, the model is not necessarily considering only the immediately previous word.

Because of Transformer processing and Attention, its prediction can depend on relationships across the available context.

Suppose the context contains:

Earlier in this conversation, the user defined Project Falcon
as the company's Toronto expansion plan.

...

What are the main risks for Project Falcon?

The model can use the earlier definition while generating the answer.

Available Context
↓
Transformer Processing
↓
Contextual Representations
↓
Next-Token Distribution

This connects directly to Chapters 1 and 3.

The context window determines what information is available.

Attention helps the model use relationships within that information.

Generation then produces tokens based on the resulting representations.

Generated Tokens Become New Context

Suppose the model begins an answer with:

There are three main reasons:

Those generated tokens are now part of the context.

The next prediction is conditioned on:

User Input
+
There are three main reasons:

If the model then generates:

First,

that becomes part of the next context too.

Generated Output
becomes
Future Input Context

Predict
↓
Generate
↓
Add to Context
↓
Predict Again

An early generation choice can therefore influence the structure of the rest of the answer.

16. Why the Same Prompt Can Produce Different Answers

Suppose you ask twice:

Give me three names for an AI search startup.

The model may produce different names each time.

This can happen because generation is often probabilistic.

At various token positions, multiple candidates may have reasonable probability.

Search...
Answer...
Signal...
Vector...
Query...

Small differences in early token selection can produce increasingly different paths.

Same Prompt
↓
Similar Probability Distribution
↓
Different Sampled Token
↓
Different Updated Context
↓
Different Future Probabilities
↓
Different Answer

This is why output variability can compound over multiple generated tokens.

17. Model Confidence and Uncertainty

People often say:

The model seems confident.

But we need to define this carefully.

An LLM does not necessarily have a single human-like feeling of confidence.

There are several different concepts that can be confused.

Token Probability

Model Uncertainty

Factual Confidence

These are not automatically the same thing.

Token Probability vs. Confidence

Suppose the context is:

The capital of France is

and the model assigns:

Paris → 95%

This means:

Given the current context and the model's learned distribution, Paris is a very likely next token.

It does not directly mean:

The model has scientifically verified with 95% confidence that Paris is the capital of France.

High Token Probability
≠
Verified Factual Confidence

Token probability concerns:

What token is likely next?

Factual confidence would concern:

How likely is the underlying factual claim to be true?

Those are not equivalent.

Why Token Probability Is Not Factual Probability

Consider a fictional claim:

The inventor of the HyperFlux engine was

Suppose the model has learned a strong linguistic pattern associated with the fictional-sounding name:

Dr. Marcus Ellison

The model might assign a high probability to that continuation.

But perhaps:

HyperFlux engine

does not exist at all.

The generated continuation can still be linguistically plausible.

High probability of next token

may indicate:

this continuation fits the model's learned language patterns

not necessarily:

this real-world claim has been verified

Model Uncertainty

Model uncertainty refers broadly to situations where the model's predictions are less decisive or where multiple continuations remain plausible.

Consider:

My favorite color is

Without more context, many continuations may be plausible:

blue
green
red
black
purple
...

The probability distribution may be relatively spread out.

Compare that with:

2 + 2 =

where one continuation may dominate strongly.

Highly concentrated distribution
→ less next-token uncertainty

Broad distribution
→ more next-token uncertainty

But even this should not be confused with perfect knowledge of factual uncertainty.

A narrow distribution can still occur around an incorrect continuation.

Linguistic Certainty vs. Factual Certainty

Consider two cases.

Case A

The capital of France is

The model may strongly favor:

Paris

and the claim is also factually correct.

Case B

Imagine the model has learned incorrect or conflicting information:

The founder of Company X was

It may strongly favor a particular name.

The linguistic distribution may be concentrated.

But the underlying claim may still be wrong.

Certain-looking generation
≠
certain factual truth

Factual Confidence

Factual confidence would ideally describe how strongly we should believe that a factual claim is correct.

For example:

Claim:
Paris is the capital of France.

A factual confidence system would ideally evaluate evidence supporting that claim.

But ordinary next-token prediction does not automatically provide that evaluation.

This is why later systems introduce mechanisms involving:

retrieval
external sources
evidence
grounding
claim verification
citations

These systems try to answer a different question:

Not only:
"What continuation is likely?"

But also:
"What evidence supports this claim?"

18. Calibration

A model is calibrated when its expressed or measured confidence corresponds reasonably well to how often it is actually correct.

A simplified example:

Suppose a system says:

I am 80% confident

on 100 comparable questions.

If it is well calibrated, we might hope it is correct approximately:

80 out of 100 times

when making predictions at that confidence level.

Stated / Estimated Confidence
≈
Observed Accuracy

This is calibration.

Poor calibration might look like:

Model acts 95% confident
but
is correct only 60% of the time

Calibration is a deeper evaluation topic, but the conceptual distinction matters now.

A model can sound confident without being calibrated.

Verbal Confidence Is Not Necessarily Numerical Confidence

Suppose an assistant says:

I am absolutely certain that...

The phrase:

absolutely certain

is itself generated language.

It should not automatically be interpreted as a precise measurement of internal confidence.

Likewise:

Probably...

or:

I think...

may be generated because such wording fits the context.

Confidence-sounding language
≠
reliable confidence measurement

19. Hallucination, Unsupported Generation, and Fabrication

What Is Hallucination?

A hallucination is generated content that presents unsupported, fabricated, or incorrect information as if it were valid.

Examples may include:

invented facts
nonexistent sources
incorrect dates
fabricated quotations
made-up people
false relationships
incorrect explanations

Suppose a user asks:

Who founded a fictional company called Northstar Quantum Labs?

If no reliable information exists, but the model generates:

Northstar Quantum Labs was founded in 2017 by Daniel Reeves.

without evidence, that would be an example of hallucinated information.

The sentence may look completely natural.

That is exactly why hallucination is important.

Hallucination Is Not Random Gibberish

A hallucination does not usually look like:

purple bicycle quantum banana 482

That would simply be incoherent.

The more difficult problem is that hallucinations can be:

fluent
specific
grammatical
confident
plausible
well-structured

For example:

The company was founded in Toronto in 2018 by software engineer Michael Reeves.

This sounds realistic.

But every detail could be invented.

Plausibility
≠
Truth

Unsupported Generation

Unsupported generation occurs when the model produces a claim without sufficient support from the information available to it.

Suppose the provided context says:

Company A launched a new product in 2025.

The model generates:

The launch increased company revenue by 35%.

If the context contains no revenue information and there is no external evidence available, the claim is unsupported.

It may happen because:

"product launch"
+
"revenue increase"

is a plausible learned association.

Plausible Association
≠
Evidence

Fabrication

Fabrication is a stronger form of unsupported generation in which the model invents specific information.

Examples:

invented person
invented statistic
invented study
invented source
invented date
invented quotation
invented event

For example:

A 2024 Stanford study found that 87% of consumers prefer AI-generated search results.

If no such study exists, the model has fabricated:

institution
date
statistic
claim

The specificity can make the statement seem more credible even though it is false.

Hallucination Can Arise from the Generation Objective

Why can hallucinations happen at all?

Return to the model's fundamental task:

Given the current context,
predict a plausible next token.

Suppose a user asks about something for which the model has weak, incomplete, or conflicting information.

The model still needs to produce the next token.

It may generate a sequence that fits learned patterns.

Incomplete Knowledge
+
Plausible Language Patterns
↓
Likely-Sounding Continuation

The continuation may sound excellent even when it is unsupported.

This is one structural reason hallucination is possible.

The Model Is Always Continuing the Sequence

Suppose the prompt is:

The CEO of the fictional company Velora Systems is

The model's immediate generation task is not:

Search reality and prove who the CEO is.

Its underlying language-model task is closer to:

Given these tokens,
what token should come next?

A learned pattern may strongly suggest:

a person's name

So the model may produce one.

David Chen

Then:

David Chen, who joined the company in...

Now each invented token creates new context supporting further invented detail.

20. Hallucination Cascades

Suppose the first unsupported generation is:

Velora Systems was founded by Sarah Mitchell.

Now that sentence becomes part of the generation context.

The next tokens may build on it:

Sarah Mitchell founded Velora Systems in 2018.

Then:

The company initially focused on enterprise analytics.

Then:

It raised $12 million in Series A funding.

The model may create an internally coherent story.

Unsupported Token Sequence
↓
Becomes New Context
↓
Supports More Plausible Continuation
↓
More Unsupported Details

This is one reason an answer can become increasingly detailed while moving further away from verified reality.

Why Specificity Can Be Dangerous

Users often associate specificity with authority.

Compare:

The company was founded several years ago.

with:

The company was founded on March 14, 2018, by Daniel Mercer in Boston.

The second sounds more authoritative.

But an LLM can generate:

dates
names
percentages
locations
citations
quotations

because these formats occur frequently in language.

Specific Detail
≠
Verified Detail

Why Fluent Writing Can Hide Errors

Language models are specifically trained to model language well.

As a result, an incorrect answer can still have:

good grammar
logical transitions
professional tone
clear structure
appropriate vocabulary

For example:

The study was conducted across 14 European countries and included 8,200 participants. Researchers found a 37% improvement in...

Everything about the writing may appear professional.

But if the underlying study is fictional, fluency provides no factual protection.

Fluency is evidence of language-generation ability, not evidence of factual accuracy.

Hallucination vs. Error

Not every wrong answer needs to be described in exactly the same way.

A model may make:

calculation errors
reasoning errors
misinterpretation errors
outdated claims
unsupported claims
fabricated claims

The term hallucination is often used broadly for generated factual content that is unsupported or incorrect.

But when analyzing systems carefully, it is useful to distinguish the failure mode.

Correct source
+
incorrect interpretation

is different from:

completely invented source

Later chapters on evidence and grounding will make these distinctions much more precise.

Why the Model Does Not Simply Say “I Don't Know”

A natural question is:

If the model is uncertain, why does it not always say “I don't know”?

Because the base language-model objective is not inherently:

Answer only when verified.

It is fundamentally based on predicting continuations.

Patterns like:

Question
↓
Direct Answer

occur frequently in training and post-training data.

So a model can be pushed toward providing an answer even when its knowledge is weak.

Later assistant training can improve behaviors such as:

expressing uncertainty
refusing unsupported claims
asking for clarification
using external tools

But these behaviors are additional layers on top of the core generation mechanism.

21. Generated Answers Can Combine Learned Knowledge and Current Context

An answer can be influenced by both:

learned parameters

and:

current context

Suppose the model has general learned knowledge about restaurants.

Then the user provides:

Northstar Bistro is located in Toronto.
It serves Persian food.
It is open until midnight.

The model can combine:

Learned General Patterns
+
Provided Context
↓
Generated Answer

For example:

Northstar Bistro is a Persian restaurant in Toronto that stays open until midnight.

This answer relies strongly on supplied context.

Later, when we study RAG, retrieved documents will play a similar role.

Context Can Override Learned Associations

Suppose the model has learned that a fictional company usually operates in London.

But the current context explicitly says:

For this exercise, assume that the company moved its headquarters to Toronto in 2026.

When answering:

Where is the company headquartered?

the current context may cause the model to answer:

Toronto

even if its learned parameters contain older associations.

Learned Parameters
+
Current Context
↓
Generation

Context Does Not Automatically Guarantee Correctness

Providing context improves the information available to the model.

But it does not automatically guarantee that the generated answer will perfectly reflect that context.

The model may still:

ignore information
misinterpret information
combine information incorrectly
invent unsupported details

Information in Context
≠
Guaranteed Faithful Output

Later chapters will introduce faithfulness and groundedness to evaluate this issue directly.

Generation and Information Competition

Chapter 1 introduced Information Competition.

Suppose the context contains a large amount of information:

Document A
Document B
Document C
Instructions
Conversation History
Retrieved Passages
Generated Tokens

All of this may compete for influence during processing.

The next-token prediction depends on how the model represents and uses that context.

So generation quality can be affected by:

what information is present
where it appears
how relevant it is
how much competing information exists
how clearly relationships are expressed

This is one reason context construction becomes important in RAG systems.

22. Generation vs. Retrieval and Verification

A Simplified Hallucination Example

Suppose a user asks:

Who founded Meridian Quantum Analytics?

Assume the model has weak or no reliable factual knowledge about this organization.

Its learned patterns may include:

"Who founded [company]?"
↓
"[Person] founded [company] in [year]."

Generation may proceed:

Meridian
↓
Meridian Quantum Analytics
↓
Meridian Quantum Analytics was
↓
Meridian Quantum Analytics was founded
↓
Meridian Quantum Analytics was founded by
↓
Meridian Quantum Analytics was founded by Daniel
↓
Meridian Quantum Analytics was founded by Daniel Harris
↓
...

Each token can be locally plausible.

But the full claim may be entirely unsupported.

Locally plausible token sequence
≠
Globally factual answer

Token-Level Generation vs. Claim-Level Truth

LLMs generate at the token level.

Humans usually evaluate answers at the claim level.

For example, the generated sentence:

Company X was founded in 2018 by Sarah Lee.

contains multiple factual claims:

Company X exists.
Company X was founded.
It was founded in 2018.
Sarah Lee was the founder.

The model generated this as a sequence of tokens.

But evaluating its truth requires examining the claims.

Token Generation
↓
Sentence
↓
Claims
↓
Evidence
↓
Grounding
↓
Citation

That is why token probability alone cannot solve factual evaluation.

Generation Is Not Retrieval

Generation
≠
Retrieval

Generation asks:

What token should come next?

Retrieval asks:

What existing document or passage should be found?

A language model can generate an answer from its learned parameters without retrieving any external document.

A retrieval system can search for documents without generating prose.

Later, RAG combines these two processes:

Retrieve Information
↓
Provide Information to LLM
↓
Generate Answer

Generation Is Not Verification

Generation
≠
Verification

Producing:

Paris is the capital of France.

and verifying:

Is Paris actually the capital of France?

are different operations.

A model can produce a statement because it is a likely continuation.

Verification requires evidence or a reliable checking mechanism.

This distinction will become central when we study:

evidence
claims
entailment
grounding
citations

23. The Complete Generation Pipeline

We can now combine the chapter into one simplified pipeline.

A user provides:

What is the capital of France?

The system performs:

User Text
↓
Tokenization
↓
Token IDs
↓
Numerical Representations
↓
Transformer Processing
↓
Contextual Representations
↓
Next-Token Probability Distribution
↓
Candidate Token Selection
↓
Generated Token
↓
Add Token to Context
↓
Transformer Processing Again
↓
New Probability Distribution
↓
Next Token
↓
Repeat
↓
Final Answer

If sampling is involved:

Probability Distribution
↓
Temperature / Generation Settings
↓
Sampling
↓
Selected Token

The final answer is therefore the result of repeated token-level generation.

24. Common Misunderstandings

Misunderstanding 1: The model generates the entire answer at once

Incorrect. Autoregressive LLMs generate output incrementally, typically token by token.

Misunderstanding 2: The model chooses an answer from a database of complete answers

Incorrect. It repeatedly predicts the next token.

Misunderstanding 3: Token probability means probability that a claim is true

Incorrect. Token probability describes how likely a token is as the next continuation under the model.

Misunderstanding 4: The highest-probability token must always be selected

Incorrect. Sampling systems can select other candidate tokens according to their probabilities.

Misunderstanding 5: Different answers mean the model's parameters changed

Incorrect. Probabilistic token selection can produce different outputs using the same model parameters.

Misunderstanding 6: Temperature changes what the model knows

Incorrect. Temperature modifies token-selection behavior during generation, not the model's learned parameters.

Misunderstanding 7: Higher temperature means greater intelligence

Incorrect. Higher temperature generally increases variability in token selection.

Misunderstanding 8: Low temperature guarantees factual correctness

Incorrect. A model can consistently produce the same incorrect answer.

Misunderstanding 9: A confident-sounding answer means the model is confident internally

Incorrect. Confidence language is itself generated text.

Misunderstanding 10: A high-probability continuation must be factually correct

Incorrect. Linguistic probability and factual correctness are different concepts.

Misunderstanding 11: Hallucinations are always obvious nonsense

Incorrect. Hallucinations can be fluent, detailed, plausible, and professionally written.

Misunderstanding 12: Specific numbers, names, and dates prove an answer is reliable

Incorrect. Specific details can also be fabricated.

Misunderstanding 13: Information inside the context is guaranteed to appear correctly in the answer

Incorrect. The model can still misunderstand, ignore, or misuse available information.

Misunderstanding 14: Generation is the same as retrieval

Incorrect. Generation creates token sequences. Retrieval finds existing information.

Misunderstanding 15: Generation automatically verifies factual claims

Incorrect. Verification requires additional evidence or checking mechanisms.

25. How the Concepts Connect

We can now extend the dependency built across the previous chapters.

Chapter 1:

Human Text
↓
Tokenization
↓
Tokens
↓
Context

Chapter 2:

Text
↓
Numerical Representations
↓
Embeddings
↓
Semantic Relationships

Chapter 3:

Token Representations
↓
Attention
↓
Transformer Layers
↓
Contextual Representations

Chapter 4:

Training Data
↓
Next-Token Prediction
↓
Prediction Error
↓
Parameter Updates
↓
Learned Model

Chapter 5 now explains what the trained model does during generation:

Current Context
↓
Transformer Processing
↓
Next-Token Probability Distribution
↓
Candidate Tokens
↓
Token Selection
↓
Generated Token
↓
Updated Context
↓
Repeat

And generation settings can modify the selection process:

Probability Distribution
↓
Temperature
↓
Sampling
↓
Selected Token

The final answer emerges from repeated prediction:

Token
↓
Token
↓
Token
↓
Token
↓
...
↓
Answer

But factual quality introduces another layer:

Likely Token
≠
Verified Fact

Therefore:

Fluent Generation
≠
Grounded Generation

This distinction prepares us for later chapters on retrieval, evidence, grounding, and citations.

Mental Model

Imagine the model standing at the beginning of a path.

It sees the current context:

The capital of France is

In front of it are several possible next steps:

Paris
London
Lyon
Rome
...

Each path has a different weight.

The model selects one.

Suppose it selects:

Paris

Now it has moved forward:

The capital of France is Paris

A new set of possible paths appears:

.
,
and
which
...

The model chooses again.

This repeats:

Context
↓
Possible Next Tokens
↓
Probability Distribution
↓
Select Token
↓
New Context
↓
New Possibilities
↓
Select Again

The model is therefore not writing the full answer in one moment.

It is constructing the answer step by step.

Training has shaped the landscape of probabilities.

Generation navigates that landscape.

Sampling determines how strictly the system follows the most likely path.

Temperature changes how strongly the most likely paths are favored.

But there is an important limitation.

The landscape represents:

what language the model expects

not necessarily:

what reality proves to be true

That is why a path can be:

smooth
natural
specific
convincing

and still lead to a false claim.

The simplest reliable mental model is:

An LLM generates an answer by repeatedly predicting a probability distribution over possible next tokens, selecting one token, adding it to the context, and repeating the process. These probabilities reflect learned language patterns—not direct guarantees of factual truth.

Chapter 5 Recap

Autoregressive Generation

Generating output incrementally, where previously generated tokens become part of the context for future token predictions.

Predict
↓
Generate
↓
Add to Context
↓
Predict Again

Probability Distribution

A distribution assigning probabilities to possible next tokens.

Token A → probability
Token B → probability
Token C → probability
...

Candidate Token

A token that could potentially be selected as the next generated token.

Token Probability

The model's estimated probability that a particular token should come next given the current context.

Token Probability
=
likelihood of next-token continuation

It is not the same as factual probability.

Generation

The process of producing output tokens from a trained model.

Sampling

Selecting a token probabilistically from the model's next-token distribution rather than always selecting only the highest-probability token.

Deterministic Generation

A generation strategy that consistently selects according to a fixed rule, such as choosing the highest-probability candidate.

Probabilistic Generation

Generation in which token selection involves probability-based sampling and can therefore produce different outputs.

Temperature

A generation setting that changes how concentrated or spread out the candidate-token probability distribution is.

Lower Temperature
→ more concentrated
→ more predictable

Higher Temperature
→ flatter distribution
→ more varied

Temperature does not retrain the model.

Model Uncertainty

The degree to which the model's predictive distribution is concentrated or spread across possible continuations.

Model Confidence

A broad term that must be interpreted carefully. There is no single simple confidence value that automatically represents factual certainty.

Factual Confidence

How strongly a factual claim should be believed to be true. Ordinary next-token probabilities do not directly provide factual confidence.

Calibration

The degree to which estimated confidence corresponds to observed correctness.

Confidence
≈
Actual Accuracy

in a well-calibrated system.

Hallucination

Generated information that is unsupported, fabricated, or incorrect while often appearing plausible and fluent.

Unsupported Generation

A claim generated without sufficient support from the available evidence or context.

Fabrication

The invention of specific information such as:

names
dates
statistics
sources
quotations
events

without valid support.

Plausible but Incorrect Output

Text can follow strong linguistic patterns while expressing a false claim.

Plausibility
≠
Truth

Token Probability vs. Factual Truth

The critical distinction of this chapter is:

Probability that a token comes next
≠
Probability that the underlying claim is true

The full dependency through Chapter 5 is now:

Human Text
↓
Tokenization
↓
Tokens
↓
Numerical Representations
↓
Transformer
↓
Attention
↓
Contextual Representations
↓
Training
↓
Learned Parameters
↓
Inference
↓
Next-Token Probability Distribution
↓
Candidate Tokens
↓
Sampling / Selection
↓
Generated Token
↓
Updated Context
↓
Repeated Generation
↓
Final Answer

And the reliability problem is:

Generated Answer
↓
May Be Fluent
↓
May Be Plausible
↓
May Still Be Unsupported or Incorrect

We now understand how a trained LLM produces an answer.

But another important question remains.

A raw language model is fundamentally trained to continue sequences.

How does it become something that:

follows instructions
acts like an assistant
respects system-level instructions
responds to user requests
produces structured output

That leads directly into Chapter 6:

Turning a Base Model into an Assistant.

The Single Most Important Idea from Chapter 5

An LLM generates text one token at a time by predicting a probability distribution over possible next tokens and repeatedly selecting from that distribution. A high-probability continuation means the token fits the model's learned patterns well—it does not mean the factual claim being generated has been verified as true.