Answering in the reader's language

Your documents are in one language and the question arrives in another. Retrieval may well have worked fine — the answer is in the context. But now a translation step sits between the source and the reader, and it lands on the parts of a grounded answer that are least tolerant of paraphrase: the quotes, the identifiers and the citations.

This is a generation problem, not a retrieval one. It shows up after you have the right chunks.

Decide what picks the answer’s language

Three candidates, and you should choose rather than inherit:

The language of the question. The usual right answer, and the usual implementation is language detection on the query. Be aware that detection on short queries is unreliable, and queries full of product names, code identifiers or form numbers can be detected as almost anything.

An explicit user or tenant setting. More reliable than detection wherever you have one. Prefer it.

The language of the documents. Defensible for internal tools with one working language, and it should be a stated policy rather than an accident of the prompt being written in English.

Whatever you pick, state it in the instruction block explicitly. A model reading English documents and an English instruction block will often answer a French question in English simply because everything around it was English.

A translated quote is not a quote

If your answer format asks for supporting quotes, cross-language generation forces a decision that is easy to get silently wrong. Putting a machine translation inside quotation marks and attributing it to a document is a small forgery: the document does not contain that sentence.

The workable pattern is to keep the quote in its original language and follow it immediately with a translation that is clearly marked as one:

When quoting a source in a different language from the answer:
- Give the quotation in the source's original language.
- Follow it with a translation, marked as a translation.
- Never present a translation inside quotation marks as the
  source's own words.

Where a quote would be more noise than help, the alternative is to paraphrase in the answer’s language and drop the quotation marks entirely — see quoting versus paraphrasing for the general trade-off. What you must not do is take the third path by default, which is to translate and keep the marks.

Some strings must not be translated

Defined terms, policy names, product names, form identifiers, field labels, menu paths and job titles should survive into the answer unchanged. The reason is practical rather than stylistic: a user who takes the answer’s translated term back to the intranet search box, or looks for a translated button label on an actual screen, finds nothing.

Keep proper nouns, document titles, form and policy identifiers,
menu paths, field names and defined terms in their original
language. Give a translation in parentheses on first use only.

The same reasoning is why this matters most in procedural answers, where nearly every load-bearing string is something the reader has to match against an interface.

Citation labels stay in the source’s language

The numeric markers travel fine — [1] is [1] in any language. Section headings and document titles inside your citation labels do not, and they should not be translated, because they are what the reader will see when they open the document.

An answer that cites “the Leave Policy, section 4” for a document titled Urlaubsrichtlinie has broken the one link the citation existed to provide. Keep the original, add a gloss if it helps, and see citations that point somewhere real for why a label that doesn’t match the destination is worse than no label.

One call or two

One call. Generate directly in the target language from source-language context. Fewer calls, lower latency, and grounding stays tight because the model is looking at the source text while it writes.

Two calls. Answer in the source language, then translate. Slower and more expensive, and the second call is another place for drift to enter. What it buys is an auditable source-language answer that your existing validation can inspect.

Which is right depends entirely on the next section.

Your faithfulness check probably stops working

Most post-generation validation compares answer text against source text, and much of it leans on lexical overlap somewhere — shared terms, quoted spans, numbers appearing in both. Across languages that signal largely evaporates. A perfectly faithful French answer to an English source shares almost no strings with it.

Two ways out:

Validate before translating. The two-call pattern exists mostly for this. Check the source-language answer, then translate what passed. The translation step is still unchecked, but it is a narrower risk than unchecked generation.

Validate with a model in both languages. Workable, and worth calibrating separately per language pair rather than assuming a checker validated in one language behaves the same in another.

Either way, do not assume the checks you already have are running. This is the part that fails quietly: checking citations after generation describes a pass that can report clean results simply because it found nothing it recognised.

Mixed-language context

Retrieval will sometimes return chunks in two languages for one question. The answer is now synthesising across languages, which makes conflict detection harder — two chunks that contradict each other in different languages are much less obviously in tension, and the near-duplicate heuristics from when sources disagree won’t fire on them at all.

Label each chunk’s language in the prompt. It costs a few tokens, it lets you instruct on the quoting rules per chunk, and it gives the model something to say when the two disagree.

Test refusals per language

The last thing to check is the failure path. Abstention instructions are usually written in the corpus language and tested there. Ask an unanswerable question in another language and you may find the refusal comes back in the wrong language, or arrives as a confident answer because the instruction carried less weight.

A useful honesty touch while you are there: let the answer note which language the sources were in. A reader who can read that language now knows they can go and check.