Internal RAG with Permissions Before Search

The problem is not only finding a relevant passage

An internal RAG system combines document search with text generation. Semantic search can find related content even when the exact words in the question do not appear in the document. In implementations based on vector indexes, files are split into passages, converted into searchable representations, and associated with metadata that can be used in filters. This solves discovery, but it does not solve authorization by itself.

The most dangerous architectural mistake is searching the entire repository and trying to remove prohibited results afterward. The cited source does not by itself establish the specific logging and intermediate-response risks described here. As an independent security recommendation, treat permission as a condition of the search, not as a cleanup step, because restricted information could otherwise enter the model context, logs, metrics, or an intermediate response.

• Identity: who is asking the question?
• Scope: which documents may this person consult now?
• Evidence: which passages support the answer?
• Decision: is there enough basis to answer, or should the system refuse?

Sources and references: [1]

Proposed flow: authorization before retrieval

The flow starts outside the model. The identity system provides a user or service identifier, groups, role, region, and any other attributes required for the decision. As an independent implementation recommendation beyond the cited source, an authorization service can convert this information into a search filter. For example, it might allow documents classified for internal use, belonging to Human Resources and the Southern region, provided that the user belongs to the relevant group.

Only after this step should the system query the index. Each document needs access-control metadata such as owner, department, classification, region, validity period, and policy identifier. The filter must be applied in the retrieval mechanism, before passages are selected for the model context. If the mechanism cannot impose this restriction, the alternative is to separate indexes by access domain or place a trusted layer in front of the search system that supplies only authorized identifiers.

After retrieval, the application should validate the returned results again. As an independent implementation recommendation beyond the cited source, this second check can detect misclassified documents, changed permissions, and integration errors. Any result without confirmed authorization should be discarded and the event recorded for technical investigation, without sending its content to the model.

• Authenticate the requester.
• Calculate permissions in the authorization service.
• Translate permissions into attribute filters or a set of authorized indexes.
• Run semantic search and, when necessary, combine it with exact-term search.
• Validate authorization for every result before building the context.

Sources and references: [1]

Evidence must stay attached to the answer

The model should not receive loose text alone. Each passage should remain associated with its source file, document identifier, section, version, validity date, and the attributes that justified its inclusion. In the final answer, the application can show a readable source reference, such as the policy name, version, and section. The purpose is not to expose the repository, but to let the user understand where the authorized statement came from.

It is also useful to separate two decisions. The first is relevance: does the passage appear to address the question? The second is sufficiency: do the available passages support a complete and current answer? A semantically close result may mention the subject without establishing the requested rule. The system should require enough evidence for each important statement, especially when documents conflict or different versions exist.

Define a structured format for the internal output, with fields such as answer, sources, gaps, detected conflict, and refusal decision. This reduces the model's freedom to invent references or mix instructions found inside documents with system rules.

• Show only sources that were used and authorized.
• Prefer the current version when validity information is available.
• Flag conflicts between policies instead of choosing silently.
• Do not treat a file name as proof that its contents support the answer.

Sources and references: [1][2]

Refusal is a normal system outcome

Refusal should occur when there is no authorized evidence, when the passages are insufficient, when the question requires an interpretation the documents do not provide, or when incompatible versions exist without a clear precedence rule. A short, specific refusal is better than a plausible statement without support. It can say that no sufficient authorized policy was found and identify the missing information, without revealing blocked documents.

The model prompt should establish that retrieved documents are data, not operational instructions. An internal document may contain phrases that try to change system behavior, request other employees' data, or trigger tool calls. Authorization rules must remain outside the model's control. Pass untrusted inputs as data and use structured formats to limit what can leave each stage. These measures reduce the risk of indirect injection, but they do not remove the need to limit access and test adversarial cases.

Do not use refusal to conceal a silent failure. Distinguish between missing documents, missing permission, low relevance, conflicting versions, and a technical error. These categories guide the user message and speed up catalog correction.

• Without an authorized passage, do not state the rule.
• Without sufficient coverage, explain the gap.
• With a conflict, present the conflict rather than choosing independently.
• With a technical failure, state that the query could not be completed.

Sources and references: [2]

Hypothetical example: querying an internal policy

Imagine a company with travel policies separated by country and job level. Someone asks, “Can I request reimbursement for lodging for a trip to Chile next month?” The system identifies the person, checks their groups, and finds that they may access travel policies for Brazil and Chile, but not executive documents. The authorization service generates a filter for the travel department, Chile, permitted classification, and documents valid on the travel date.

The search finds Chile's lodging policy and a general expense-reporting procedure. The application sends the model only the authorized passages, along with source identifiers and validity dates. The answer can explain the limit or condition described in those passages and name the policies used. It should not conclude that an expense is allowed merely because the policy mentions lodging.

Now suppose the policy covers lodging but does not clarify whether prior approval is required. The correct result is to answer only what is documented and state that the approval requirement was not found in the authorized repository. If an old and current policy contain different rules, the system should flag the conflict or apply a predefined precedence rule. This example is only a product-design illustration, not legal guidance or a normative interpretation.

• An authorized question does not automatically produce an authorized answer.
• The source must support the specific statement, not merely the general topic.
• The answer should separate the rule found, the gap, and any conflict.

How to decide whether the design is ready

Before release, build a test set containing ordinary questions, similar documents, crossed permissions, users without access, older versions, ambiguous questions, and attempts to place instructions inside files. Measure retrieval, access control, source fidelity, and refusal behavior separately. A single overall score can hide a permission leak or an answer without evidence.

Run the tests after every meaningful change to partitioning, metadata, ranking, prompts, or the model. As an independent design recommendation, record the query, the identity used for authorization, the applied filters, identifiers of returned documents, the refusal decision, and the displayed sources. Avoid recording restricted content beyond what is necessary to diagnose the system.

The evaluation should reflect actual use, including short questions, typing errors, multiple intents, and attempts to obtain information from another group. Evaluation documentation recommends defining the objective, assembling a representative set, choosing specific metrics, and repeating tests continuously instead of relying on the impression that answers look good.

• Test users with different permissions on the same question.
• Verify that revoked documents stop appearing after an update.
• Check that every important statement has an authorized source.
• Count correct refusals separately from technical errors.
• Include injection attacks inside documents and questions.

Sources and references: [3]