RAG Mental Models and Boundaries

RAG retrieves permitted evidence, selects context, and asks a model to answer from it. Trust depends on corpus, retrieval controls, and answer policy.

View full size

AWS Prescriptive Guidance: Understanding Retrieval Augmented Generation, RAG overview figureLast verified: 2026-08-25

Search returns documents for a person; RAG additionally selects context and generates. Keep direct search when source inspection is the user task.

Test permission-denied, stale, conflicting, and zero-result cases, not only fluent answers.

1. Name the user decision and allowed sources.
2. Set retrieval, grounding, citation, latency, cost, and abstention measures.
3. Create representative queries before tuning.
4. Release only improvements that preserve controls.

A RAG decision starts with a user decision, not a vector database. State the question the system may answer, the source classes it may use, the authority that remains outside the system, and the consequence of an unsupported answer.

Treat this record as a change gate. If a request needs a current transaction, entitlement check, or action, route it to an authoritative API or human workflow instead of stretching retrieval into a system of record. AWS distinguishes agentic RAG from a static retrieval step because an agent can decide whether and how to retrieve during a reasoning loop; that extra autonomy needs an equally explicit decision boundary.

Do not assign every failure to the model. AWS identifies connectors, processing, embeddings, retrieval, guardrails, orchestration, user experience, and identity as production RAG components. An actionable incident record therefore preserves the user query, permitted principal, source and chunk versions, retrieval results, answer mode, and reviewer disposition without treating a fluent answer as diagnostic evidence.

Classify three sample requests before choosing a model: “summarise this published handbook section”, “what is my remaining leave balance?”, and “compare two policy clauses and propose questions for HR”. The first is a bounded RAG candidate, the second needs an authoritative system, and the third can use retrieval only if the output is clearly advisory and cites both clauses.

1. For each request, write the user decision, permitted evidence, source owner, and a no-evidence response.
2. Create one supported query, one zero-result query, one access-denied query, and one conflicting-source query.
3. Inspect retrieved chunks before any generated answer. Confirm that every displayed citation resolves to permitted, current source material.
4. Accept the workload only when the chosen path produces an observable result for all four cases; otherwise narrow its scope or move it to an API or reviewer.

This lab deliberately avoids a universal score threshold. The useful acceptance test is whether the evidence and fallback behaviour support the specific user decision. Record disagreements between source owners as corpus governance work, not as a prompt tweak.

An evidence budget is a reviewable limit on what an answer may claim from the retrieved material. It is not a similarity-score threshold. The harness below makes the system demonstrate a safe response for supported, incomplete, conflicting, and empty retrieval results before a team optimizes prompt wording or model selection.

1. Run the same fixture through retrieval-only and answer paths, preserving returned source identities.
2. Have a reviewer compare material answer sentences with the permitted and prohibited claim lists.
3. Fail the case when the answer is fluent but omits a required qualification, source, or escalation.
4. Version the fixture with the corpus and answer policy so a source update cannot silently change the safety boundary.

AWS frames the generative AI lifecycle as a continuous process evaluated against the six Well-Architected pillars. This harness supplies evidence for that lifecycle: it turns an abstract grounding expectation into a repeatable integration and continuous-improvement check. It does not prove factual truth outside the permitted corpus.

Use a counterfactual case as well: provide a plausible answer that is not supported by the retrieved evidence and require the application to reject it. This catches a common evaluation gap where a system merely echoes the expected answer because the fixture asks only for a keyword. Keep the expected mode independent from the model provider, so a retriever or prompt change can be compared against the same evidence budget.

Use the sheet as a decision record, not a compliance badge. The Agentic AI Lens adapts the six pillars for systems that can reason, invoke tools, and maintain state; a static RAG workload can use the same questions before autonomy is added. A failed row should produce a bounded repair owner and a retest, not a generic request to “improve the model.”

Review the sheet at a real change boundary: adding a new source class, expanding an audience, changing a retrieval filter, or allowing an agent to decide whether to retrieve. Mark whether the change affects a single pillar or creates a trade-off, such as a richer context path that changes operational traceability and performance work together. The review result should name the evidence that will be re-collected after release.