Permissions at generation time

A RAG system indexes your organisation’s documents. Someone asks a question, retrieval finds the most relevant passage, and the answer quotes it. If that passage came from a document the asker has no right to read, your system has just performed an unauthorised disclosure — and done it fluently, with a citation.

This is the failure that turns a helpful internal tool into an incident. It’s also entirely preventable, and the prevention has to happen in a specific place.

Why it’s easy to get wrong

Vector search doesn’t care about permissions. It finds semantically similar text. A confidential document about a subject is more similar to a question about that subject than the public overview is, so restricted material is often ranked higher, not lower.

And the paths in are mundane:

Over-broad ingestion. A crawler pointed at a shared drive picks up everything readable by the service account, which is usually far more than any individual user can see. This is the most common cause.

Permissions not carried through. Documents are indexed with their text and not with who may read them. Nothing downstream can filter, because the information isn’t there.

Filtering in the wrong place. Retrieval returns everything and a later step tries to remove restricted material. Fragile, and the content has already been in the prompt.

Stale permissions. Access was revoked; the index still reflects the old state.

Leakage through synthesis. The subtle one. Even without quoting a restricted document, an answer derived from it discloses its contents. Filtering visible citations while generating from the text is not protection.

The rule

Retrieval must only ever return documents this user is permitted to read. Filter at retrieval, as part of the query, using the asking user’s identity.

Everything else is a workaround. If restricted content reaches the prompt, it can reach the answer — via quotation, paraphrase, synthesis, or the model simply mentioning that a document exists. Post-hoc removal is trying to unring a bell.

Practically, that means:

Index permission metadata alongside content. Whatever your access model uses — groups, roles, document ACLs, tenant identifiers — store it on the chunk.

Filter in the query itself, not after. Most vector stores support metadata filters; use them so restricted chunks are never candidates.

Use the asker’s identity, not the service’s. A single service account with broad access is the architecture that produces this bug.

Re-check at generation. Cheap defence in depth: before assembly, verify every selected chunk is still readable by this user. Catches revocations since indexing and bugs in the filter.

Keeping permissions current

Access changes and indexes go stale, so this is an ongoing concern rather than a setup task.

Store a reference, resolve at query time, where your permission system allows it. Slower per query and always correct — usually the right trade.

Or re-sync on a schedule, accepting a window of staleness. Then keep the window short, and know how long it is.

Handle deletions promptly. A document removed from the source system should leave the index. Deletion propagation is the part of index maintenance most likely to be quietly broken — worth testing explicitly rather than assuming.

Fail closed. If permission information is missing or unresolvable for a chunk, exclude it. The default on error must be to withhold, not to include.

Multi-tenant systems

Where one deployment serves separate customers, this stops being a quality concern and becomes the security boundary.

Tenant identifier on every chunk, and a filter that cannot be omitted. Not a parameter a caller may forget to pass — enforced in the layer that talks to the store.

Consider separate indexes per tenant where the isolation guarantee needs to be structural rather than conditional. Costs more; much harder to get wrong.

Test cross-tenant retrieval explicitly. A test that asks tenant A’s system a question whose answer only exists in tenant B’s data, and asserts nothing comes back. This test should exist and should be in the suite that blocks deploys.

What the answer should say

When permission filtering removes the material that would have answered a question, be careful about what the refusal reveals.

“I don’t have information on that” is safe. “There is a document about this that you don’t have access to” confirms the document’s existence, and existence is sometimes the sensitive part — that a project exists, that a person is under review, that an acquisition is being discussed.

Whether you can acknowledge restricted material depends on your context. In many internal settings, saying “there is material on this you’re not cleared for; contact the owner” is genuinely more useful and acceptable. In others it’s a leak. Decide it deliberately, and make it a configured behaviour rather than an accident of phrasing.

Consistency matters too: if unanswerable questions and access-denied questions produce visibly different responses, the difference itself is a signal an attacker can probe.

Shared and public deployments

Two cases worth calling out because they get missed:

Preview and demo environments built from production data. These frequently have weaker access controls and the same documents. The index is the sensitive asset, wherever it lives.

Logs and traces. Retrieved chunks in application logs are the same content in a system with different access rules. Anyone who can read the logs can read the corpus. Redact, restrict, or accept that your log store now needs the same protection as your document store.

The short version

Filter at retrieval, with the asking user’s identity, using permission data stored on the chunk. Re-check before assembly. Fail closed. Test cross-tenant isolation as a blocking test. Keep the index’s view of access fresh, and remember that logs are a copy of the corpus.

None of it is difficult. All of it has to be there before real users arrive, because the failure mode isn’t a bad answer — it’s a disclosure you can’t retract.