Chapter 15- Preparing Information for Retrieval

Chapter Goal

In Chapter 14, we learned the basic RAG architecture:

User Query
↓
Retriever
↓
Relevant Documents / Passages
↓
Retrieved Context
↓
Generator
↓
Answer

But this architecture assumes something important:

The retriever already has a searchable collection of information.

A company may have:

PDFs
web pages
support articles
product manuals
policies
research reports

But a retriever cannot simply treat every raw file as an ideal search unit.

The information must first be prepared.

A basic preparation pipeline looks like this:

Document
↓
Parsing
↓
Chunking
↓
Embedding
↓
Indexing
↓
Searchable Collection

This chapter explains that preparation process.

By the end of the chapter, you should understand:

what a Document is in a retrieval system;
why Document Structure matters;
what Parsing does;
what a Passage is;
what a Chunk is;
why Chunking is necessary;
what Chunk Size means;
why chunks sometimes overlap;
what Chunk Overlap does;
what Metadata is;
what Document Metadata is;
how Metadata Filtering works;
what a Chunk Embedding is;
how an Embedding Model participates in indexing;
what a Vector Database does;
what a Search Index is;
and how raw documents become searchable units for retrieval.

The central pipeline is:

Raw Document
↓
Parsing
↓
Structured Content
↓
Chunking
↓
Chunks
↓
Embedding / Index Representation
↓
Indexing
↓
Searchable Retrieval System

15.1 Document Processing

1. What Is a Document?

In Information Retrieval, a Document is an information unit stored in the searchable collection.

It does not have to mean a Microsoft Word file.

A document could be:

web page
PDF
support article
news article
product page
company policy
research paper
manual

For example:

Document:
Enterprise Refund Policy.pdf

may contain several pages covering:

eligibility
refund periods
exceptions
payment methods
cancellation

The entire file is one document, but different parts may answer different queries.

2. Why Raw Documents Need Processing

Suppose we have a 60-page PDF.

A user asks:

How long do Enterprise customers have to request a refund?

Only one paragraph may contain the answer.

If the retrieval system treats the entire 60-page PDF as one indivisible searchable object, the relevant information becomes mixed with a large amount of unrelated material.

Large Document
├── Introduction
├── Billing
├── Refund Policy ← relevant
├── Security
├── Account Management
└── Legal Terms

The system therefore needs to transform the document into more useful retrieval units.

This is document processing.

3. Document Structure

Documents are not usually flat blocks of text.

They contain structure.

Title

Heading 1
  Paragraph
  Paragraph

Heading 2
  Paragraph
  List

Heading 3
  Table

Common structural elements include:

title
headings
sections
paragraphs
lists
tables
captions
links

Structure can help preserve the meaning of information.

For example:

Heading:
Refund Eligibility

Paragraph:
Annual subscriptions can be refunded within 14 days.

The heading provides useful context for the paragraph.

4. Why Document Structure Matters for Retrieval

Consider this text:

Standard Plan
Cancellation period: 30 days

Enterprise Plan
Cancellation period: 14 days

If document structure is preserved, the relationship is clear:

Standard Plan
→ 30 days

Enterprise Plan
→ 14 days

But poorly processed text might become:

Cancellation period: 30 days
Cancellation period: 14 days

Now important context has disappeared.

Useful relationships can include:

heading → paragraph

table row → value

section → content

The goal is not merely to extract words.

It is to preserve enough structure for later retrieval to remain meaningful.

5. What Is Parsing?

Parsing is the process of reading a source document and extracting usable content and structure from it.

Raw File
↓
Parser
↓
Usable Content

For example:

PDF File
↓
Parsing
↓
Title
Headings
Paragraphs
Tables
Text

Or:

Web Page
↓
Parsing
↓
Main Article Content
Headings
Paragraphs
Metadata

Parsing converts the source into something the retrieval pipeline can work with.

6. Parsing Is Not Retrieval

Parsing and retrieval perform different jobs.

Parsing
→ prepares information

Retrieval
→ finds information for a query

For example:

PDF
↓
Parsing
↓
Extracted Content
↓
Indexing
↓
Later User Query
↓
Retrieval

Parsing happens while preparing the information source.

Retrieval happens later when someone searches it.

7. Poor Parsing Can Damage Retrieval

Imagine a table:

Plan        Refund Window
Standard    30 days
Enterprise  14 days

A good parser preserves:

Standard → 30 days
Enterprise → 14 days

A poor parser might extract:

Standard
Enterprise
30 days
14 days

The information exists, but its relationships have been damaged.

Retrieval quality can therefore depend on processing quality long before a query arrives.

8. What Is a Passage?

A Passage is a smaller section of a document containing a coherent piece of information.

For example:

Document:
Enterprise Billing Policy

Passage:
Annual Enterprise subscriptions may be refunded
within 14 days of the original purchase.

The document may contain thousands of words.

The passage contains the information relevant to a particular question.

Document
↓
Sections / Passages
↓
Specific Information

9. Document Retrieval vs. Passage Retrieval

Suppose a user asks:

Can I cancel my subscription after 14 days?

Document-level retrieval might return:

Enterprise Billing Policy.pdf

Passage-level retrieval might return:

Cancellation requests submitted after the 14-day
refund period are not automatically eligible for refund.

Document Retrieval
→ Which document is useful?

Passage Retrieval
→ Which part of the document is useful?

15.2 Chunking

10. What Is a Chunk?

A Chunk is a unit of content created during preprocessing so that it can be represented, indexed, and retrieved.

Suppose a document contains:

3,000 words

The system might divide it into:

Chunk 1
Chunk 2
Chunk 3
Chunk 4
...

Document
↓
Chunking
↓
Chunk A
Chunk B
Chunk C
Chunk D

11. Passage vs. Chunk

These terms are related, but useful to distinguish.

A Passage is conceptually:

a meaningful section of information

A Chunk is more specifically:

a unit created by the preprocessing pipeline
for retrieval or indexing

Chunks can be created using:

characters
words
tokens
paragraph boundaries
section boundaries

Passage
→ semantic/document concept

Chunk
→ processing/retrieval unit

In practice, the terms are sometimes used loosely or interchangeably.

12. What Is Chunking?

Chunking is the process of dividing a document into smaller units that can be indexed and retrieved.

Document
↓
Chunking
↓
Chunk 1
Chunk 2
Chunk 3
...

For example:

50-page Product Manual
↓
Chunking
↓
Installation Chunk
Configuration Chunk
Billing Chunk
Troubleshooting Chunk
Security Chunk
...

The goal is to create retrieval units that are small enough to be specific but large enough to preserve useful meaning.

13. Why Chunking Is Necessary

Suppose a long document contains ten unrelated subjects.

If we create one embedding for the whole document:

Entire Document
↓
One Embedding

that representation must summarize many different meanings at once.

Chunking allows:

Topic A
→ Chunk A

Topic B
→ Chunk B

Topic C
→ Chunk C

Now a query about Topic B can directly retrieve:

Chunk B

instead of relying on the representation of the entire document.

14. Chunk Size

Chunk Size describes how much content is placed inside one chunk.

For example:

Chunk A
≈ 100 tokens

or:

Chunk B
≈ 500 tokens

There is no universally correct chunk size.

Smaller Chunk
→ more specific

Larger Chunk
→ more surrounding context

15. What Happens When Chunks Are Too Large?

Suppose a chunk contains:

refund policy
password reset
account deletion
payment methods
security settings

A query asks only about:

refund policy

The chunk contains the answer but also large amounts of unrelated information.

Possible consequences include:

less focused embedding
weaker passage-level relevance
more irrelevant context
higher token usage

16. What Happens When Chunks Are Too Small?

Imagine the source says:

Enterprise customers are eligible for a refund
when cancellation occurs within 14 days
of the original annual-plan purchase.

Poor chunking might create:

Chunk 1:
Enterprise customers are eligible for a refund

Chunk 2:
when cancellation occurs within 14 days

Chunk 3:
of the original annual-plan purchase

The relationship between:

Enterprise customer
refund
14 days
annual plan

has been fragmented.

17. The Chunk-Size Trade-Off

Too Large
→ more context
→ less specificity
→ more noise

Too Small
→ more specificity
→ less context
→ fragmented meaning

A useful chunk tries to preserve a coherent information unit:

Specific Enough
+
Contextual Enough

18. Chunk Boundaries Matter

Chunking should not be thought of only as:

every N tokens

Meaningful boundaries can matter too.

Possible boundaries include:

paragraph end
section heading
subheading
list
table
topic transition

For example:

H2: Refund Policy

Paragraph 1
Paragraph 2
Paragraph 3

may form a more coherent chunk than cutting the text exactly halfway through Paragraph 2.

19. What Is Chunk Overlap?

Chunk Overlap repeats some content between neighboring chunks.

Without overlap:

Chunk 1:
Tokens 1–300

Chunk 2:
Tokens 301–600

With overlap:

Chunk 1:
Tokens 1–300

Chunk 2:
Tokens 251–550

Tokens:

251–300

appear in both chunks.

20. Why Use Chunk Overlap?

Imagine the important statement begins at the end of one chunk:

Enterprise customers may receive a refund...

and finishes in the next:

...within 14 days of the original purchase.

Without overlap, neither chunk may contain the complete statement.

Chunk Boundary
↓
Potential Meaning Loss

Overlap
↓
Preserve Some Boundary Context

21. More Overlap Is Not Always Better

Overlap duplicates information.

Chunk 1
AAAA BBBB

Chunk 2
BBBB CCCC

Chunk 3
CCCC DDDD

More overlap can preserve context, but it also creates:

more stored content
more embeddings
more duplicate retrieval candidates
more indexing cost

Its purpose is:

reduce information loss at chunk boundaries

not simply duplicate as much content as possible.

15.3 Metadata

22. What Is Metadata?

Metadata is information that describes the document or chunk rather than being the main content itself.

For example:

Title:
Enterprise Refund Policy

Author:
Finance Department

Published:
2026-08-15

Language:
English

Document Type:
Policy

Content
→ what the document says

Metadata
→ information about the document

23. Common Metadata Fields

Metadata can include:

title
URL
author
publication date
updated date
language
category
document type
source
location
product
department
access level

There is no universal required metadata schema.

The useful fields depend on the retrieval task.

24. Document Metadata

Document Metadata describes the source document.

Document:
Enterprise Refund Policy

Metadata:
document_id = 8421
department = Finance
language = English
updated_at = 2026-08-15
document_type = Policy

When that document is divided into chunks, the chunks can inherit relevant metadata.

Chunk 17

text:
Annual Enterprise subscriptions...

document_id:
8421

department:
Finance

document_type:
Policy

This keeps the chunk connected to its source.

25. Why Metadata Matters

The text itself answers:

What does this passage say?

Metadata can answer:

Where did it come from?

When was it updated?

What language is it?

What category does it belong to?

Which product does it describe?

This information can improve filtering, ranking, access control, or source tracking.

26. Metadata Filtering

Metadata Filtering restricts retrieval using metadata conditions.

Suppose the query is:

refund policy for Enterprise customers

The vector search might find semantically related chunks across many products.

But the system could apply:

product = Enterprise

as a metadata filter.

Semantic Search
+
Metadata Constraint
↓
More Appropriate Candidates

27. Metadata Filtering Example

Suppose the collection contains:

Chunk A
product = Standard
topic = refunds

Chunk B
product = Enterprise
topic = refunds

Chunk C
product = Business
topic = refunds

The query requires:

Enterprise

A metadata filter can restrict retrieval to:

product = Enterprise

leaving:

Chunk B

28. Metadata Is Not the Same as Content

Suppose a chunk says:

Customers can request refunds within 14 days.

and metadata says:

product = Enterprise

The meaning available to the system comes from both:

Content:
14-day refund rule

Metadata:
Enterprise product

But they play different roles.

Metadata usually provides structured descriptive information.

The chunk text contains the information being retrieved and read.

15.4 Indexing

29. What Is Indexing in This Pipeline?

After documents have been processed and divided into useful chunks, those chunks need to become searchable.

That is the role of Indexing.

Processed Chunks
↓
Represent Them for Search
↓
Store in Searchable Structure
↓
Index

Later:

User Query
↓
Search Index
↓
Relevant Chunks

30. Chunk Embedding

For dense retrieval, each chunk can be passed through an Embedding Model.

Chunk
↓
Embedding Model
↓
Chunk Embedding

For example:

Text Chunk
↓
Embedding Model
↓
[0.17, -0.42, 0.08, ...]

That vector represents the chunk in the embedding space.

This is a Chunk Embedding.

31. Why Embed Chunks?

Recall dense retrieval from Chapter 9.

The system compares:

Query Embedding

with:

Chunk Embeddings

Document
↓
Chunks
↓
Chunk Embeddings
↓
Vector Index

Later:

User Query
↓
Query Embedding
↓
Similarity Search
↓
Relevant Chunks

32. One Document Can Produce Many Embeddings

Suppose:

Document
= 5,000 tokens

After chunking:

Chunk 1
Chunk 2
Chunk 3
...
Chunk 15

Each chunk can receive its own embedding:

Chunk 1 → Vector 1
Chunk 2 → Vector 2
Chunk 3 → Vector 3
...

Instead of:

One Document
→ One Broad Representation

we get:

One Document
→ Multiple Focused Retrieval Representations

33. Embedding Model

The Embedding Model converts each chunk into a dense vector representation.

Preparation:

Chunk
↓
Embedding Model
↓
Chunk Embedding

Query time:

Query
↓
Same Compatible Embedding Model
↓
Query Embedding

Then:

Query Embedding
↔
Chunk Embeddings

can be compared using similarity search.

The query and indexed chunks must use compatible representations in the same embedding space.

34. Vector Database

A Vector Database is a system designed to store and search vector representations, often together with their associated content and metadata.

A simplified stored record might contain:

Chunk ID

Chunk Text

Embedding Vector

Metadata

For example:

chunk_id: 17

text:
Annual subscriptions may be refunded within 14 days.

embedding:
[0.17, -0.42, ...]

metadata:
document_type = policy
product = enterprise

35. Vector Database Is Not the Same as Embedding Model

These two components perform different jobs.

Embedding Model
→ creates vectors

Vector Database
→ stores / indexes / searches vectors

Text
↓
Embedding Model
↓
Vector
↓
Vector Database

At query time:

Query
↓
Embedding Model
↓
Query Vector
↓
Vector Database
↓
Similar Chunk Vectors

36. What Is a Search Index?

A Search Index is a searchable representation of a document collection.

It is broader than a vector database.

For lexical retrieval, the search index may use:

inverted index

For dense retrieval, it may contain:

vector index

A hybrid retrieval system may maintain both.

Prepared Content
↓
Indexing
↓
Search Index
↓
Fast Retrieval

Search Index
≠
Raw Document Collection

37. Vector Database vs. Search Index

Search Index
→ broad concept:
   searchable representation of content

Vector Database
→ system specialized in storing
   and retrieving vector representations

A retrieval architecture might contain:

Lexical Index
+
Vector Index
+
Metadata

and combine them during hybrid retrieval.

All Search Indexes
≠
Vector Databases

38. Metadata and Embeddings Can Be Stored Together

A dense retrieval record may look conceptually like:

Chunk Text
+
Chunk Embedding
+
Metadata

For example:

Text:
Enterprise subscriptions may be refunded within 14 days.

Vector:
[...]

Metadata:
product = enterprise
language = en
updated = 2026-08-15

Then retrieval can combine:

Vector Similarity
+
Metadata Filtering

For example:

Find semantically relevant chunks
WHERE
product = Enterprise
AND
language = English

39. The Complete Preparation Pipeline

Start with:

Document

Then:

Document
↓
Parsing

Parsing extracts usable content and structure.

Parsed Document
↓
Chunking

Chunking produces smaller retrieval units.

Then:

Chunks
↓
Embedding Model
↓
Chunk Embeddings

Finally:

Chunk Text
+
Chunk Embeddings
+
Metadata
↓
Indexing
↓
Searchable Index

The core pipeline is:

Document
↓
Parsing
↓
Chunking
↓
Embedding
↓
Indexing

40. What Happens Later at Query Time?

Once preparation is complete, the indexed information waits for queries.

Preparation time:

Documents
↓
Parsing
↓
Chunking
↓
Embedding
↓
Indexing

Later, query time:

User Query
↓
Retriever
↓
Search Index
↓
Relevant Chunks

Most of Chapter 15 happens:

before the user asks the question

Retrieval itself happens:

after the query arrives

41. Preparation Quality Affects Retrieval Quality

Suppose the correct information exists in a document.

That alone does not guarantee good retrieval.

The information may be damaged by:

poor parsing
bad chunk boundaries
chunks that are too large
chunks that are too small
missing metadata
incorrect embeddings
poor indexing

Good Source Information
≠
Automatically Good Retrieval

The information must also be prepared effectively.

42. A Complete Example

Suppose we have:

Enterprise_Billing_Policy.pdf

The document contains:

Section 1 — Billing
Section 2 — Renewal
Section 3 — Cancellation
Section 4 — Refunds
Section 5 — Exceptions

Step 1 — Parsing

Title
Sections
Headings
Paragraphs
Text

Step 2 — Chunking

Chunk 21:
Enterprise customers may request a refund
within 14 days of the original annual purchase.

Chunk 22:
Refund eligibility does not automatically apply
after the 14-day period.

Step 3 — Metadata

document = Enterprise Billing Policy
category = Billing
product = Enterprise
language = English

Step 4 — Embedding

Chunk 21
↓
Embedding Model
↓
Vector 21

and:

Chunk 22
↓
Embedding Model
↓
Vector 22

Step 5 — Indexing

The system stores:

Chunk Text
+
Vector
+
Metadata

inside the searchable system.

Later the user asks:

How long do I have to request an Enterprise refund?

The retriever can now find:

Chunk 21

instead of sending the entire PDF to the LLM.

43. Common Misunderstandings

Misunderstanding 1: A document can always be indexed exactly as it is

Not necessarily. Raw documents often need parsing and preprocessing first.

Misunderstanding 2: Parsing and retrieval are the same thing

Incorrect. Parsing prepares information. Retrieval searches prepared information.

Misunderstanding 3: Passage and chunk always mean exactly the same thing

Not necessarily. A passage is a meaningful section of information; a chunk is a preprocessing unit used for indexing and retrieval.

Misunderstanding 4: Smaller chunks are always better

Incorrect. Chunks that are too small can break important semantic relationships.

Misunderstanding 5: Larger chunks are always better because they contain more context

Incorrect. Large chunks can introduce unrelated information and reduce retrieval specificity.

Misunderstanding 6: Chunk overlap exists to duplicate as much information as possible

Incorrect. Its main purpose is to reduce information loss around chunk boundaries.

Misunderstanding 7: Metadata is the document content

Incorrect. Metadata describes the document or chunk.

Misunderstanding 8: Metadata Filtering is the same as semantic similarity

Incorrect. Metadata filtering applies structured conditions; semantic retrieval measures meaning-based similarity.

Misunderstanding 9: An Embedding Model stores vectors

Incorrect. The Embedding Model creates vectors. A vector index or vector database can store and search them.

Misunderstanding 10: A Vector Database and Search Index are exactly the same concept

Incorrect. A vector database specializes in vector storage and retrieval. Search index is a broader retrieval concept.

Misunderstanding 11: Indexing happens every time a user searches

Usually not. Document indexing is commonly performed before query-time retrieval.

Misunderstanding 12: If the correct fact exists somewhere on the page, retrieval will automatically find it

Incorrect. Parsing, chunking, representation, metadata, and indexing can all affect whether that information becomes retrievable.

44. How the Concepts Connect

Chapter 14 gave us:

User Query
↓
Retriever
↓
Relevant Passage
↓
Retrieved Context
↓
Generator

Chapter 15 explains where those searchable passages come from.

Before retrieval:

Raw Documents
↓
Document Processing
↓
Parsing
↓
Chunks
↓
Metadata
↓
Embeddings / Search Representation
↓
Indexing
↓
Searchable Collection

Then later:

User Query
↓
Retriever
↓
Searchable Collection
↓
Relevant Chunk
↓
RAG Context

The larger architecture is now:

SOURCE PREPARATION

Documents
↓
Parsing
↓
Chunking
↓
Embedding
↓
Indexing

        ↓

QUERY TIME

User Query
↓
Retriever
↓
Relevant Chunks
↓
LLM Context
↓
Generator
↓
Answer

We now know:

Chapter 13
→ Why retrieval is needed

Chapter 14
→ How retrieval connects to generation

Chapter 15
→ How information is prepared so retrieval can work

The next question is:

What happens to the user's query before the retrieval system searches these indexes?

That leads into:

Chapter 16 — Query Understanding

Mental Model

Imagine a large library receives 10,000 new books.

Simply placing all 10,000 books in a giant pile does not create a useful search system.

First, the library examines each book.

That is:

Document Processing

It identifies:

title
chapters
sections
content

That is similar to:

Parsing

Then the library creates useful cards for specific sections of each book.

Those smaller searchable units are similar to:

Chunks

Each card also contains descriptive information:

book title
author
category
date
language

That is:

Metadata

Then the library organizes those cards into a searchable catalog.

That is:

Indexing

Later someone asks:

Where does this book explain refund eligibility?

The librarian does not read every book from beginning to end.

They search the prepared catalog and locate the relevant section.

The AI version is:

Raw Document
↓
Parse It
↓
Divide It into Useful Chunks
↓
Represent the Chunks
↓
Attach Metadata
↓
Index Them
↓
Retrieve the Right Chunk Later

The simplest mental model is:

Parsing
→ understand the source structure

Chunking
→ create searchable units

Metadata
→ describe those units

Embedding
→ represent their meaning

Indexing
→ make them searchable

And the simplest reliable definition is:

Preparing information for retrieval means transforming raw documents into searchable units. The system parses the document, divides its content into useful chunks, preserves descriptive metadata, creates retrieval representations such as chunk embeddings, and stores them in a search index so that relevant information can later be retrieved efficiently.

Chapter 15 Recap

Document

A source information unit such as:

web page
PDF
article
manual
policy

Document Structure

The organization inside the document:

title
headings
sections
paragraphs
lists
tables

Structure helps preserve relationships between pieces of information.

Parsing

Converting a raw source into usable content and structure.

Raw Document
↓
Parsing
↓
Usable Content

Passage

A meaningful smaller section of a document.

Document
↓
Passage

Chunk

A unit created during preprocessing for indexing and retrieval.

Document
↓
Chunking
↓
Chunks

Passage vs. Chunk

Passage
→ meaningful information section

Chunk
→ preprocessing / retrieval unit

They can be similar, but they are not conceptually identical.

Chunking

The process of dividing documents into smaller retrievable units.

Document
↓
Chunking
↓
Chunk 1
Chunk 2
Chunk 3

Chunk Size

How much content is included in each chunk.

Too Large
→ more noise

Too Small
→ fragmented meaning

The goal is:

Specific Enough
+
Contextual Enough

Chunk Overlap

Repeating some content between adjacent chunks.

Chunk 1:
1–300

Chunk 2:
251–550

Purpose:

preserve context around boundaries

Metadata

Structured information describing a document or chunk.

title
author
date
language
category
location
document type

Document Metadata

Metadata associated with the original source document.

Chunks can inherit relevant document metadata.

Metadata Filtering

Using structured metadata to restrict retrieval.

Semantic Candidates
+
product = Enterprise
↓
Filtered Candidates

Embedding Model

Converts a chunk into a vector representation.

Chunk
↓
Embedding Model
↓
Chunk Embedding

Chunk Embedding

The dense vector representing a chunk in embedding space.

Chunk Text
↓
Vector

Vector Database

A system designed to store and search vector representations, often together with text and metadata.

Chunk
+
Embedding
+
Metadata
↓
Vector Database

Search Index

A searchable representation of the prepared collection.

It may contain:

lexical index
vector index
metadata

Preparation Time vs. Query Time

Preparation:

Document
↓
Parsing
↓
Chunking
↓
Embedding
↓
Indexing

Later:

Query
↓
Retrieval
↓
Relevant Chunk

The complete Chapter 15 pipeline is:

Raw Documents
↓
Document Processing
↓
Parsing
↓
Structured Content
↓
Chunking
↓
Chunks
↓
Metadata
↓
Embedding Model
↓
Chunk Embeddings
↓
Indexing
↓
Search Index / Vector Database
↓
Searchable Information

The official simplified pipeline is:

Document
→ Parsing
→ Chunking
→ Embedding
→ Indexing

Once this preparation is complete, the retrieval system has something it can efficiently search.

The next chapter asks:

How does the system interpret and transform the user's query before searching that collection?

That is:

Chapter 16 — Query Understanding

The Single Most Important Idea from Chapter 15

A retriever does not search raw knowledge in the abstract. Information must first be transformed into searchable units: documents are parsed, meaningful chunks are created, metadata is preserved, chunks are represented for retrieval, and those representations are indexed. The quality of this preparation directly affects what the retrieval system can find later.