THE SHORT ANSWER
Prepare RAG documents so retrieved sections contain a professionally complete statement and its necessary context. Split along meaningful headings, conditions and topics rather than character count alone. Preserve scope and source references. Suitable segmentation depends on your questions and document types, and should be tested against actual retrieval and answer failures rather than selected through a universal chunk-size recommendation.
To connect individual AI applications with reliable marketing operations, read the pillar guide AI transformation in B2B marketing: from individual prompts to reliable operations.
Identify information that must stay together
Take representative questions and mark the passages jointly required for a correct answer. A price without currency, market or exception conditions is not a complete knowledge unit. Neither is a product feature without its approval status. Inspect headings, tables and footnotes before choosing a fixed segmentation rule. A sentence may only make sense within its chapter. Document which contextual elements must survive processing. Retrieval should provide more than similar words: it should return a section whose meaning can be used reliably without silently dropping a condition that changes the business conclusion. Explore this further in the guide Writing answer sections that stand on their own.
Preserve structure during conversion
Inspect how PDFs, tables and other formats become text. Columns can become interleaved, headings can disappear and footnotes can lose their reference. Good search cannot reliably repair a foundation already damaged by extraction. Preserve appropriate page, section and document references. Table rows often need column headings and units. For strongly visual documents, consider specialised processing or a professionally maintained structured version. Label missing parts rather than treating conversion as complete merely because the technical job ended without an error. Check examples from each important layout instead of assuming one successful page represents the entire document collection.
Attach concise source context to sections
Add necessary attributes such as document title, product, market and period of applicability. Context should explain what the section concerns without introducing unsupported facts. Anthropic’s Contextual Retrieval approach examines adding contextual information before retrieval; your particular implementation still needs evaluation. Keep original text distinguishable from the added description. Otherwise an incorrect summary can become an apparent primary source. When a model generates context, check names, dates and restrictions that affect subsequent decisions. A short addition is useful only if it preserves the meaning rather than expanding an uncertain passage into a confident but unverified explanation.
Choose overlap and neighbours deliberately
Limited overlap can preserve information at boundaries but also creates similar matches and extra context. Test whether an answer needs a neighbouring condition or whether better topic boundaries resolve the issue. Provide a way to fetch adjacent sections when needed. Avoid making every text block larger as a universal repair for weak retrieval. Large sections can bury useful details among unrelated material. The decision depends on document structure and the question, so assess whether the retrieved evidence is genuinely usable rather than simply counting matches or celebrating a particular formal section length.
Inspect an explicitly fictional table case
A fictional product matrix lists features in rows and regional availability in columns. Careless text extraction returns only yes and no without the associated market headings. The team instead prepares each relevant row with its feature, market and status while retaining a reference to the original matrix. This hypothetical example exposes a structure problem that a longer answer prompt cannot solve. After the change, the team also checks footnote exceptions. A correctly preserved table row would still mislead if an applicable restriction remained outside the retrieved section and was never supplied to the answer stage.
Compare segmentations on fixed questions
Use known questions with expected evidence locations and compare segmentation methods on the same source version. First inspect whether the right information is retrieved, then whether the answer uses it correctly. This separates search problems from interpretation problems. Include conditions, footnotes and questions needing several related sections. Check citations too: readers should be able to locate the claim in the original. Record remaining limitations and the selected processing version. Repeat assessment after substantial document changes or new file formats, because earlier segmentation rules may no longer preserve the professional meaning they were designed to protect. Explore this further in the guide Sources in AI answers: verify evidence claim by claim.
INTERACTIVE / FROM READING TO ACTION
Three checkpoints for your implementation
Choose a checkpoint and track your reading progress. Checks are neither saved nor sent and do not score your business.
CHECKPOINT 01
Attach concise source context to sections
Add necessary attributes such as document title, product, market and period of applicability.
Explore in the article ↗CHECKPOINT 02
Choose overlap and neighbours deliberately
Limited overlap can preserve information at boundaries but also creates similar matches and extra context.
Explore in the article ↗CHECKPOINT 03
Inspect an explicitly fictional table case
A fictional product matrix lists features in rows and regional availability in columns.
Explore in the article ↗Frequently asked questions
What section size is optimal?
There is no universal value. Test completeness, retrievability and context against your own questions and documents.
Is more overlap always better?
No. It can preserve boundary context but also create similar matches and unnecessary volume.
Can a model add context?
Yes, as a reviewed supplementary description kept distinct from original text, rather than a new unchecked source.
How can I put “Identify information that must stay together” into practice?
Take representative questions and mark the passages jointly required for a correct answer. A price without currency, market or exception conditions is not a complete knowledge unit. Neither is a product feature without its approval status.
What matters when working on “Preserve structure during conversion”?
Inspect how PDFs, tables and other formats become text. Columns can become interleaved, headings can disappear and footnotes can lose their reference. Good search cannot reliably repair a foundation already damaged by extraction.
