AI · 4 min read · Quick PDF Tools team

How to check an AI's answer against the source document

AI assistants can invent quotes, page numbers, and figures. A five-minute routine to check answers against your PDF using page citations and text search.

Chat bubble linked to an open document by a dotted line with a magnifying glass

Why AI answers need checking

AI assistants produce fluent text that sounds right. Most of the time, with a clean document, it is right. But they can also hallucinate: state a figure that isn't in the document, quote a sentence that was never written, or cite a page number that doesn't match. These mistakes look just as confident as correct answers, so you can't spot them by tone.

Vendors say this themselves. OpenAI's help pages for its Office add-ins tell users to check important numbers, formulas, citations, and claims against primary sources. The routine below makes that quick.

Step 1: Give the AI text with page markers

Checking is easiest when the AI and you are looking at the same text. Convert the PDF with PDF to text or PDF to Markdown. Both add a heading for each page, so a citation like "page 14" points to a specific place. If the PDF is a scan, run OCR PDF first, otherwise neither you nor the AI has text to search.

Step 2: Ask for exact quotes and page numbers

Add this to your question:

"For every claim, give the page number and a short exact quote (under 25 words) from the document that supports it. If the document doesn't say, reply 'not stated in the document'."

An exact quote is easy to check. A paraphrase isn't. Allowing "not stated" gives the model a way out instead of a reason to guess.

Some in-app assistants now cite their sources directly. As of late 2026, Claude for Word gives clickable section citations and Claude for Excel gives cell-level citations. Use them, but click through. A citation tells you where to look. It doesn't prove the claim.

Step 3: Search for every quote

Open the text file from PDF to text in any editor and use Find (Ctrl+F or Cmd+F):

  • Search for 4 to 6 distinctive words from the quote, not the whole sentence. Line breaks and hyphenation in PDFs can stop a full-sentence match.
  • Check the page heading above the match. Is it the page the AI cited?
  • Read the sentence before and after. A real quote taken out of context is the most common subtle error, for example "fees are refundable" when the next line says "only before 30 days".

No match at all? Treat the claim as unsupported until you find it yourself.

Step 4: Check every number separately

Numbers are where small errors do the most damage: a deadline a day off, a fee with a digit swapped, a percentage from a different table. For each figure you will act on:

  1. Search the text file for the number itself, such as 1,500 and 1500.
  2. Confirm the label next to it matches what the AI said it was.
  3. If the AI calculated something, such as a total or difference, redo the sum yourself or ask for the formula.

Step 5: Run a quick trap test

Ask one question you know the document doesn't answer, for example "What is the late fee?" on a notice that has none. A good answer says it isn't stated. If the AI invents one, be more careful with everything else in that conversation, and start a new chat with a smaller section of the document.

A worked example

You upload a 30-page internship offer and ask about the notice period. The AI says: "Either party may end the internship with 15 days' notice (page 6)." You search the text file for "days' notice" and find it on page 6: "…with 15 days' notice, except during the first month, when no notice is required." The quote was real, but the answer left out the exception. That one search took twenty seconds and changed your understanding of the contract.

How much to check

Match the effort to the stakes. A summary for revision needs a spot check of a few claims. Anything involving money, deadlines, legal terms, health, or an exam application needs every claim you act on checked against the source. When it really matters, read the original page, not just the extracted text.

Two habits make all of this faster. Keep each conversation to one document, so the AI can't mix up two sources. And when you find an error, tell the AI and ask it to recheck its other answers against the same pages. It will often find more of its own mistakes once it knows to look.

Extract PDF text to check quotes

FAQ

What is an AI hallucination?

It is when an AI states something that isn't supported by its sources, such as an invented quote, figure, or page number, and sounds just as confident as when it is correct.

Why search the text file instead of the PDF?

Both work if the PDF has a text layer. The text file from PDF to text matches what the AI read and shows a heading for each page, so citations are easy to check.

Does a citation mean the answer is correct?

No. It tells you where to look. Read the cited passage, including the sentences around it, before you rely on it.

← All posts · PDF tools home