When sources disagree

Retrieval pulls back two chunks. One says the limit is five days, the other says ten. Both are from your own documents. Without instruction, the model will produce a single confident answer using one of them, and nothing in the response will indicate there was a disagreement.

That silent resolution is the worst available outcome, because it hides information the user needed. And conflicting sources are not an edge case — any corpus that has existed for a while contains superseded versions of itself.

Where conflict comes from

Knowing the cause tells you the right handling.

Superseded versions. The most common by far. An old policy and its replacement both sit in the index. Neither is wrong; one is current. Fixable with dates.

Different scope. Both correct, for different populations — one applies to full-time staff, one to contractors. Not really a conflict; a missing qualifier in the question.

Genuine inconsistency. Two current documents that contradict each other. A real problem in your corpus that a RAG system can surface but not solve.

Draft versus published. A proposal and an approved policy. The draft may be more detailed and more appealing to retrieve, which is a trap.

Extraction artefacts. One “source” is a mangled table or a mid-sentence fragment that only appears to say something different. Not a conflict at all.

Making conflict visible

The first requirement is that conflict can be noticed, which is a prompt assembly matter:

Clear source separation. Chunks concatenated into one block can’t be compared — the model reads one document that happens to say two things. Delimited and labelled, they’re two claims from two places.

Dates on every chunk. Without them, recency isn’t available as a tiebreak, no matter how you instruct. This is the single highest-value piece of metadata for this problem.

Document identity. “policy-2024.pdf” versus “policy-2026.pdf” is often enough on its own.

Authority signals, where your corpus has a hierarchy — official policy versus a wiki page versus a chat log. Lets the model prefer sensibly instead of treating all text as equal.

Instructing for it

Explicit and in the instruction block:

If the sources disagree, do not choose silently. State that they
disagree, give each position with its citation and date, and say
which appears more authoritative and why. Prefer the more recent
source, and say that you have done so.

Four requirements: don’t hide it, attribute both, reason about which wins, disclose the reasoning. The last one matters — an answer that says “per the 2026 update, ten days (the 2024 handbook said five)” lets the reader catch it if the tiebreak was wrong.

For scope conflicts, add:

If the sources apply to different cases, say which applies to which
rather than choosing one.

Detecting it mechanically

Instruction handles most of it. Where correctness matters more, you can check independently:

Ask about conflict as a separate call. Before or alongside generation: “Do these passages contain contradictory claims about the question?” A focused question gets a better answer than one asked in passing.

Answer from each source separately, then compare. More calls, and it makes disagreement structurally obvious rather than something the model has to notice. Useful for high-stakes questions.

Flag date spread. If retrieved chunks span several years, the chance of superseded material is high. Cheap heuristic worth logging.

Flag near-duplicates with differences. Two chunks that are textually similar but not identical are very often two versions of the same passage. Also cheap, and a strong signal.

Presenting it well

Once you know sources disagree, the answer has to communicate that without being useless.

Lead with the best answer, then note the conflict. “The limit is ten days, per the 2026 policy update. The 2024 handbook stated five days and appears to have been superseded.” A reader gets an answer and the caveat.

Not: “Sources disagree; some say five and some say ten.” Technically honest, practically a shrug.

Show both dates. Readers resolve superseded-version conflicts instantly once they can see the dates — faster and more reliably than any tiebreak rule.

Make both sources reachable. For a genuine inconsistency the user may need to read both and escalate.

Escalate where it matters. If your system supports consequential decisions, an unresolvable conflict is a case for a human, not for a confident synthesis.

Fix the corpus

RAG surfaces conflict; it doesn’t resolve it. If your index contains three versions of one policy, every query touching that policy has a problem, and no amount of prompt work fixes it properly.

Which makes conflict reports a maintenance signal. Log them. Recurring conflicts on the same documents point at a specific thing to clean up: an outdated file that should be removed from the index, a draft that shouldn’t have been ingested, two teams’ documents that genuinely contradict each other and need someone to decide.

That last case is worth noting — sometimes the RAG system has found a real organisational disagreement that predates it. Surfacing that is a useful thing to have done, even though it isn’t a bug you can close.