When the question contains a false premise

A user asks why the policy allows fifteen carry-over days. The policy allows ten. Retrieval does its job perfectly and returns the exact paragraph that says ten. The model then explains the reasoning behind fifteen, cites that paragraph, and sounds authoritative doing it.

This failure is worse in a RAG system than in a bare chatbot, because the confabulation arrives wearing a citation.

Why the default is to go along

A question’s framing is more salient than a line buried in a retrieved chunk. “Why does X?” presupposes X, and a model tuned to be helpful answers the question it was asked rather than interrogating it. Answering why without first establishing whether is the whole failure.

Nothing upstream catches it either. The question is topically perfect, so retrieval scores are high and any abstention threshold based on those scores stays quiet — see teaching a model to say I don’t know for why score-based gating can’t see this class of problem. The context genuinely contains the relevant material. The only thing wrong is that the answer contradicts it.

Four shapes it takes

A wrong value. A number, date, name or threshold in the question that doesn’t match the source. The easiest kind to detect and the most common.

Assumed existence. “How do I submit form 27B?” when no such form exists. The model will happily invent a plausible submission process, because “how do I” invites steps and the retrieved material about other forms supplies the shape.

Assumed causation. “Why did we move to the new vendor because of the outage?” Two real facts, a relationship that no source asserts. The model supplies the connective tissue.

Assumed applicability. A rule that is real but scoped — true for full-time staff — asked as though it were universal. Not exactly false, and the correction is a qualifier rather than a contradiction.

Instruct for the check

Put it in the instruction block, before the answer format:

The question may contain incorrect assumptions. Before answering,
check the question's factual claims against the sources.

If a source contradicts an assumption, correct it first, cite the
correction, then answer the corrected question.

If the sources neither confirm nor contradict an assumption, do
not treat it as established and do not treat it as false. Say the
sources do not address it.

That third paragraph is the one people leave out, and it is what stops the feature from becoming a new failure mode. Absence in the retrieved chunks is not absence in the corpus, let alone absence in the world.

Correct, then answer

A premise check that only refuses is a downgrade. The user asked something, and there is usually a real answer nearby.

Compare “The policy does not allow fifteen days” — technically responsive, practically a dead end — with “The carry-over limit is ten days, not fifteen [2]. The stated reason is that unused days are accrued against the calendar year rather than the leave year.” The correction is unmissable, and the question the user meant to ask got answered.

For the existence case the same pattern applies with a different second half: say what does exist. “There is no form 27B in these documents. Hardware requests use the asset request in the service catalogue [1]” is a correction that solves the user’s actual problem.

Never assert non-existence from silence

This deserves its own instruction because the phrasing gap is small and the meaning gap is enormous.

“The sources I searched don’t mention a form 27B” is a claim about your index and is true. “There is no form 27B” is a claim about the world and is unsupported. Documents get missed by retrieval, excluded by permissions, dropped by a failed ingestion job, or simply live in a corpus this system doesn’t cover.

The abstention wording that works for missing answers works here too, and it should be reused rather than reinvented — the user needs to know what was searched, not just that nothing was found.

Catching it outside the instruction

A validation pass on figures. If the answer restates a number that appears nowhere in the retrieved text, either the model derived it or it accepted it from the question. Both are worth flagging, and this sits naturally alongside checking citations after generation.

A separate premise call for high-stakes questions. “Do these passages contradict any assumption in this question?” asked on its own gets a better answer than the same check asked in passing inside a long instruction block.

Watch for the correction that should have been a conflict report. If two sources disagree and one of them happens to match the question’s premise, the right behaviour is the disagreement handling in when sources disagree, not a confident correction in either direction.

Log the corrections

Corrections are one of the most useful signals a RAG system produces about the organisation it serves. A premise that keeps recurring across users means something specific: either a document is unclear enough to be misread the same way repeatedly, or an outdated version is circulating outside your index and people are reading it.

Neither is a bug in the pipeline. Both are things you can only find out by reading what your users believed before they asked.

And don’t over-correct

Tuned too aggressively, premise checking turns into pedantry — arguing with loose phrasing, restating qualifiers nobody needed, opening every answer with a correction. The line is the instruction above: correct what a source contradicts, qualify what a source scopes, and stay quiet about everything the sources simply don’t mention.