Your documents are not a knowledge base yet

The pitch is seductive: upload your documents, and the agent answers questions from
them. The reality is that a folder of PDFs is raw material, not a knowledge base,
and the gap between the two is where most disappointing deployments live.

Start with what is actually in those files. A price list from two years ago that
nobody deleted. Three versions of the returns policy, one of which was never
approved. A scanned document that is an image of text with no text in it at all. A
spreadsheet whose meaning lives entirely in cell formatting. Retrieval does not know
which of these is current; it knows which is similar to the question. Ask about
refunds and the superseded policy answers, confidently, in a voice that sounds
authoritative.

Then there is the shape of the content. Documents are written to be read by a person
with the whole page in front of them. A voice answer is one or two sentences, heard
once, with no scrollback. A clause that makes sense inside a section heading becomes
misleading when read aloud on its own. Chunking that ignores document structure
routinely splits a condition from its exception — and the exception is usually the
part the caller was calling about.

Voice raises the stakes on the retrieval question too. A chat interface that returns
a slightly wrong passage lets the reader notice and re-read. A spoken answer is
committed the moment it is said. That argues for a higher confidence bar in voice
than you would accept in chat, and for an agent that says “let me put you through to
someone” rather than reaching for the nearest match.

What actually works is unglamorous: decide what is current and delete the rest; run
real OCR on scans and check the output; chunk along the document’s own structure so
a clause keeps its heading; tune retrieval against the questions callers really ask,
taken from your own transcripts rather than imagined; and set a confidence floor
below which the agent hands off instead of guessing.

The system that says “I don’t know, transferring you” is more valuable than the one
that always has an answer. Callers forgive a handover. They do not forgive being
told something untrue in a confident voice.