Checking citations after generation

Once an answer has been generated with citations, you can check it before showing it to anyone: does each cited source actually contain what’s attributed to it? This is the step that turns citation from a gesture into a guarantee, and a useful amount of it costs almost nothing.

The checks form a ladder, from string comparison to a full entailment judgement. Most systems should be on the first two rungs and aren’t.

Free checks

These need no model call and catch real errors.

Do the cited source numbers exist? If five sources went in and the answer cites [7], something is wrong. Trivial to check, and it happens.

Does every factual sentence carry a citation? Count sentences and citation markers. Uncited assertions are where ungrounded content hides. You’ll get false positives on connective sentences, so treat this as a flag rather than a rejection.

If you asked for quotes, do the quotes appear in the cited chunks? The highest-value check available. Normalise whitespace and punctuation, then substring match. A quote that isn’t in its chunk is a definitively bad citation — no judgement required.

Are all sources used? Not an error, but informative. If your system routinely includes ten chunks and uses two, you’re including too many.

Together these run in milliseconds and catch fabricated source numbers, uncited claims, and invented quotes. That’s a large share of citation problems, for the cost of some string handling.

Paid checks

When you need more than the free ones.

Entailment per claim. For each cited claim, ask a model: does this passage support this statement? Yes, no, or partially. One call per claim, or one batched call.

Catches the important case the free checks miss — a citation that points at genuinely related material which doesn’t contain the specific claim. Costs a call, adds latency, and is itself a model judgement, so it’s better than nothing rather than authoritative.

Claim extraction then checking. Decompose the answer into atomic claims, then check each. More thorough, more calls, and it catches merged claims — a sentence combining two sources into an assertion neither made — which per-sentence checking can miss.

A second model. Using a different model for verification than for generation avoids correlated errors: a model asked to check its own output is inclined to approve it.

Where to run it

Synchronously, blocking the response. Users only see validated answers. Adds latency, and it’s the right choice for anything consequential.

Synchronously, but only the free checks. Milliseconds of latency, catches the mechanical errors, ships everything else. A good default.

Asynchronously, after showing the answer. No latency cost, and problems are found after the user has read the answer. Useful for measurement and monitoring, not for prevention.

On a sample. Validate a percentage of traffic thoroughly to track quality over time, plus free checks on everything. Practical compromise for high volume.

The combination most systems should run: free checks on every response synchronously, entailment checking on a sample asynchronously for monitoring, and synchronous entailment on whatever subset of questions is high-stakes.

What to do with a failure

Having detected an unsupported claim, you have to decide. Options, and when each fits:

Regenerate. Try again with feedback about what failed. Often works, since many failures are incidental. Doubles cost, and you need a retry limit and a fallback.

Strip the offending claim. Remove the unsupported sentence, ship the rest. Good when the answer is mostly fine; risky when removing a sentence changes the meaning of what remains.

Mark it. Show the answer with the unsupported claim flagged. Honest, and it puts judgement where it belongs — though it does ask the user to do work.

Refuse. Fall back to “I can’t answer this reliably from the available sources.” Safest, and the correct choice when validation fails badly rather than marginally.

Escalate. Route to a human. For high-stakes systems, this is what validation is for.

Pick per severity: a quote that doesn’t match is worse than a sentence lacking a citation, and they shouldn’t get the same treatment.

What validation cannot do

Worth being clear, because it’s easy to over-trust this layer.

It can’t catch what wasn’t retrieved. An answer perfectly supported by the sources can still be wrong, because the sources were incomplete or outdated. Validation checks internal consistency, not truth.

It can’t catch a wrong-but-supported answer. If a retrieved document is itself wrong, a faithful answer inherits the error. Garbage in, faithfully cited garbage out.

It can’t judge relevance. An answer can be fully grounded and not address the question.

It’s a model judging a model. Entailment checks have their own error rate, in both directions.

So validation raises the floor rather than guaranteeing correctness. It reliably prevents the specific failure of asserting things your sources don’t say — which is the failure most damaging to trust, because it’s the one users have been promised won’t happen.

Start here

If you have no validation today: implement the free checks. Source numbers exist, every factual sentence has a citation, quoted text appears in its chunk. An afternoon of work, no ongoing cost, and it catches the errors that are most embarrassing when a user finds them first.

Then log the failure rate. That number tells you whether the paid checks are worth adding, and you’ll have an actual basis for the decision.