Text chunking splits a document into units that a retrieval system can index and return. Good chunks preserve enough context to support an answer while staying small enough to retrieve precisely. There is no universal chunk size or overlap percentage that works best for every corpus.

Choose boundaries around the evidence your users need. A paragraph may be enough for a definition. A table row may need its column headers. A policy exception may need the rule immediately above it. Splitting all three in the same way can make an otherwise correct document impossible to use.

Compare the main strategies

Strategy	Useful starting point	Main cost or failure
Fixed size	A simple reproducible baseline	Cuts through sentences, tables, and conditions
Recursive boundaries	Prose with paragraphs and sentences	Weak source structure still produces awkward splits
Structure aware	Headings, code, tables, slides	Requires a parser that preserves those structures
Semantic boundaries	Topic shifts poorly reflected in formatting	Adds model calls and thresholds to tune
Parent-child retrieval	Small passages need surrounding context	Larger returned parents can consume the context budget

Semantic chunking usually compares representations of adjacent text to detect topic changes. That may help a particular collection, but additional embedding work does not guarantee better answers. Test it against a simpler baseline using the same questions, retrieval budget, and answer model.

Inspect extraction before tuning boundaries

A PDF parser can read a two-column page in the wrong order. A slide can lose its title. A table can become a stream of numbers with no units. A chunker cannot reconstruct meaning that the extraction stage discarded.

Keep page or section identifiers through the pipeline. Inspect a sample of difficult source formats, not only clean Markdown. If a table row is indexed separately, attach enough header context to interpret it. If you repeat a heading in each chunk, record that it was added as context rather than pretending it was part of the quoted passage.

For source code, use syntax and symbol boundaries where appropriate. That is a different problem from splitting prose; see the site's existing code-retrieval material before applying a generic paragraph splitter to an entire repository.

A runnable baseline for boundary experiments

This dependency-free Python function chunks by whitespace-delimited words. It is deliberately a baseline: words are not model tokens, and this function does not preserve original formatting or understand sentence meaning. It returns offsets so the original source can remain the reference.

def word_chunks(text, size=120, overlap=20):
    if size <= 0 or overlap < 0 or overlap >= size:
        raise ValueError("Require size > 0 and 0 <= overlap < size")
    words = text.split()
    chunks = []
    start = 0
    while start < len(words):
        end = min(start + size, len(words))
        chunks.append({
            "start_word": start,
            "end_word": end,
            "text": " ".join(words[start:end]),
        })
        if end == len(words):
            break
        start = end - overlap
    return chunks

example = " ".join("w" + str(i) for i in range(250))
parts = word_chunks(example)
assert [(p["start_word"], p["end_word"]) for p in parts] == [
    (0, 120), (100, 220), (200, 250)
]
assert word_chunks("") == []

The example covers all 250 source words and indexes 290 word occurrences because two boundaries repeat 20 words each. That is 16% extra occurrences in this particular input, not a universal storage-overhead estimate. Embedding dimensions, metadata, and index structure add their own costs.

Use a model-aware token splitter when you need token limits. For production citations, preserve character spans or source-native locators rather than rebuilding source text from this whitespace-normalized demo.

Test overlap against actual boundary failures

Overlap can help when an answer spans two adjacent chunks. It can also return near-duplicates that push other useful evidence out of the result set. Start with a non-overlapping or low-overlap baseline and add overlap where the fixture reveals a problem.

A useful test policy is “Refunds are available within 14 days. Enterprise annual plans require approval.” Ask about an enterprise annual refund. If the system retrieves only the first sentence, the answer may omit the condition. Check whether a structural boundary, a larger parent, or overlap resolves that failure with the least unnecessary context.

Also test negative conditions, numbered procedures, tables, and headings that repeat across products. The best size for definitions may be a poor size for a troubleshooting sequence.

Keep identity and versions on every chunk

Record the tenant or authorized scope, document ID, version, source locator, and any context needed for filtering. A revision should let you identify which old chunks no longer represent the active document. A deletion should reach its derived records too.

For a research workflow, the citation needs to identify the passage that was actually used. A chunk's similarity score is not evidence that its contents are current or authoritative.

Measure more than retrieval hits

Use a fixed set of questions and acceptable supporting spans. Compare whether the evidence is retrieved, whether it is complete, how many duplicate passages reach the model, and whether the answer uses the relevant conditions correctly.

Run comparisons under a fixed context-token budget as well as a fixed number of chunks. Ten very large chunks and ten short chunks do not represent equivalent evidence or cost. Track ingestion time, indexed volume, query latency, and answer support separately.

Anthropic's contextual retrieval experiments illustrate another option: adding brief document context to chunks before retrieval. That is distinct from changing chunk boundaries. Evaluate the two changes separately so you know which one helped.

Choose a strategy you can explain

Start with structure-aware splitting when reliable structure exists, and keep a simple baseline for comparison. Try semantic boundaries for collections where topic changes are important but headings are poor. Try parent-child retrieval when small matches repeatedly lack surrounding explanation.

Then connect the result to the rest of the RAG pipeline. Chunking cannot repair a missing permission filter, an outdated source, or an answer that ignores its evidence. The hybrid search guide covers the next retrieval decision after the chunks themselves are usable.

Once you have a chunking baseline, try retrieval with Supermemory on the same small corpus and question set. Inspect the returned passages and citations to see whether the evidence your answers need survives the full ingestion and retrieval path.

Text Chunking for RAG: Strategies, Examples, and Evaluation