Route query intent to the right retrieval signal
Dense retrieval helps with paraphrase and meaning. BM25 or lexical retrieval remains important for names, identifiers, error codes, and rare exact terms.
Start with explicit query classes: semantic question, identifier lookup, filter-bound request, and no-answer. Each class should have representative labelled queries.
Primary reference: configure OpenSearch neural, sparse, and hybrid search
OpenSearch Serverless documents neural, neural_sparse, and hybrid query capability. Store, model, collection, and Region configuration determine what a workload can actually use, so verify the current deployment rather than inferring support from a generic hybrid label.
Read the OpenSearch Serverless neural and hybrid search documentationChoose sparse and dense signals by the evidence each query needs
“Sparse” is not a synonym for BM25, dense retrieval, or hybrid. Treat each as a different candidate-generation mechanism, then measure the composition against the query classes that matter to the workload.
Retrieval signal choices: representation, useful query shape, and control boundary
| Signal | Representation | Useful query shape | Decision boundary |
|---|---|---|---|
| BM25 lexical | Observed query and document tokens plus corpus term statistics. | Exact IDs, names, error codes, and rare terms. | Vocabulary and tokenisation are explicit; it does not infer paraphrase meaning. |
| Dense | Embedding vectors produced by an embedding model. | Paraphrases and related meaning with different wording. | Model, dimensions, index, and distance metric need compatible evaluation. |
| Learned sparse | A model-produced weighted vocabulary or token-weight representation searched through sparse index structures. | Vocabulary expansion or term-like evidence where learned weighting is worth evaluating. | Model and sparse-index support are configuration-dependent, not implied by BM25. |
| Hybrid | An evaluated fusion or routing composition of candidate lists, not a fourth representation. | Mixed query classes where a fixed baseline shows complementary evidence. | Fix fusion, filters, candidate depth, reranker, and context budget before comparing runs. |
OpenSearch can support neural, neural sparse, and hybrid search, but the available path depends on the selected store, model, collection, and Region configuration. Do not infer that an Amazon Bedrock Knowledge Bases HYBRID setting automatically uses learned sparse retrieval. Verify the deployed capability and its operational constraints in current service documentation.
- Freeze the corpus snapshot, server-derived filters, candidate depth, reranker, and context budget.
- Compare BM25, learned sparse, dense, BM25 plus dense, and learned-sparse plus dense only where the selected store supports them.
- Score exact identifiers, paraphrases, synonym or domain vocabulary, denied scope, and recent changes as separate slices.
- Inspect candidates, citations, authorization outcomes, latency, and cost observations before promoting a composition.
Combine signals without weakening controls
Hybrid retrieval merges selected candidate routes, such as dense, lexical, or learned sparse. Reciprocal rank fusion is a useful baseline because it combines ranks without pretending their raw scores share a scale. Apply tenant, ACL, document-state, and freshness filters as retrieval constraints.
Controlled hybrid retrieval
Loading diagram...
Rewrite, rerank, and budget context deliberately
Query rewriting can clarify vocabulary or decompose a question, but preserve the original for audit and compare it in evaluation. Bedrock rerankers reorder retrieved textual data and can be invoked directly or with Knowledge Bases retrieval. Rerank a bounded candidate set, then allocate top-K and context tokens to evidence rather than filler.
Use ablation to make quality changes explainable
- Freeze a labelled query set and source snapshot.
- Measure BM25 lexical, learned sparse where supported, dense, and evaluated hybrid baselines.
- Add one change at a time: filter, rewrite, fusion, reranker, top-K, or context budget.
- Review gains and regressions by query class and authorization outcome.
- Promote only a versioned configuration with a rollback path.
Version retrieval configuration as a testable hypothesis
Treat result count, search mode, metadata filter, reranking, and context budget as one named configuration. A learner should be able to answer: which candidate set was searched, which evidence was allowed, what was reranked, and what finally reached generation?
Bedrock RetrieveAndGenerate accepts a retrieval configuration and can use a reranking model. The API also returns cited source chunks, which makes the configuration useful for inspection rather than only for producing a fluent answer. Keep the request record beside the corpus and prompt version; do not infer a quality improvement from an answer alone.
Illustrative retrieval configuration record
A workload-owned request shape. Values are deliberately examples, not a universal top-K, threshold, or model recommendation.
Click on an annotation to highlight it in the JSON
Make query-suite slices expose different failure modes
A single aggregate relevance score hides the query that matters most. Build a fixed suite with semantic paraphrases, exact identifiers, multi-constraint questions, permission-scoped requests, stale-document traps, and legitimate no-answer prompts. Label expected permitted sources and the expected abstention outcome before tuning.
Retrieval slices and observable validation
| Slice | Decision | Inspect |
|---|---|---|
| Exact identifier | Keep lexical signal? | Expected chunk appears before context selection |
| Paraphrase | Improve semantic recall? | Allowed source rank and citation coverage |
| Filtered request | Enforce scope? | No cross-scope chunk in candidates or citations |
| No answer | Abstain? | Low-signal context is not converted into a confident answer |
Use the same corpus snapshot and permissions when comparing slices. If a change improves a paraphrase slice but harms identifier recall or returns an unauthorized candidate, it is not a clean promotion. The appropriate response may be query routing or a narrower configuration, not a bigger context window.
Promote with an ablation worksheet and slice-level release contract
Ablation asks what changed, why it helped, and whom it harmed. Start from a frozen baseline. Toggle one control at a time: filter representation, search mode, candidate count, reranker, or context budget. Save retrieved references and final citations for sampled wins and failures, because an aggregate score cannot diagnose a misleading gain.
Minimal ablation worksheet
| Run | One change | Required decision |
|---|---|---|
| A | Dense baseline | Record slice evidence and latency distribution |
| B | Add lexical fusion | Accept only if identifier gains do not break scope |
| C | Add reranking | Inspect selected passages, not just answer fluency |
| D | Reduce context | Check citation support and no-answer behaviour |
Primary reference: inspect retrieval and cited evidence
AWS documents RetrieveAndGenerate configuration, reranking, retrieved references, and practical prompt-size boundaries. Verified 2026-08-25; model, Region, and quota support remain workload-specific.
AWS Bedrock: Query a knowledge base and generate responsesDebug fusion with rank-level evidence, not blended scores
When hybrid retrieval changes a result, preserve the rank contribution from every source list. Raw dense and lexical scores are often incomparable, so a combined score is not an explanation. A rank-level record lets a reviewer see whether an exact identifier was rescued by lexical search, a paraphrase was supplied by semantic search, or a filter removed an otherwise relevant candidate.
Fusion-debug record
An illustrative per-query audit artifact. It records candidate provenance without exposing source content in routine logs.
Click on an annotation to highlight it in the JSON
Use this record for sampled regressions, not necessarily every production query. If a forbidden candidate appears at any stage visible to the application, stop the release and trace identity-to-filter construction. If the permitted source loses rank, compare a dense-only, lexical-only, and fused replay against the same corpus snapshot before changing another control.
Constrain query rewriting so it cannot rewrite authorization or intent
A rewrite may improve retrieval vocabulary, but it also creates a second interpretation of the request. Preserve the original query, run the same server-derived filter for every rewrite, limit the number of rewrites or subqueries, and make the final evidence attributable to a particular query variant. Query decomposition is not permission expansion: a rewrite must not add a tenant, classification, or tool instruction that the caller did not have.
Rewrite safeguard and failure action
| Check | Evidence | Failure action |
|---|---|---|
| Original retained | Original and rewritten text share a case ID | Reject unreplayable run |
| Scope unchanged | Same server-built filter on every variant | Block and investigate identity path |
| Bounded fan-out | Recorded variant count and context budget | Stop expansion and return no-answer or clarification |
| Variant attribution | Selected source linked to query variant | Do not promote opaque gain |
Compare current AWS hybrid-retrieval architectures before copying one
A 2025–26 AWS architecture inventory is more useful than a single “best RAG stack”. The managed Knowledge Base launch, Knowledge Bases performance guidance, and explicit OpenSearch hybrid RAG example each place different ownership on the team. Compare them against the same corpus, identity scope, and labelled questions before promoting one.
Three AWS source architectures, three different ownership boundaries
| AWS source pattern | What AWS shows | Your design question |
|---|---|---|
| Managed Knowledge Base (June 2026 launch) | Managed connectors, ingestion, retrieval, agent integration, and performance assessment. | Can its documented configuration express the required sources, identity rules, retrieval behavior, and operations? |
| Knowledge Bases evaluation guidance (March 2025) | Iterative evaluation of retrieval settings, including hybrid search and relevance controls. | Does hybrid retrieval improve the named query class without weakening authorization, latency, or citations? |
| Bedrock plus OpenSearch hybrid RAG example | A more explicit search architecture with OpenSearch ownership. | Do custom relevance controls and integrations justify the added index, pipeline, and operational responsibility? |
Glossary: terms first introduced here
These are the technical terms introduced in this step. Later modules build on them rather than redefining them from scratch.
New vocabulary for this step
| Term | Plain-language meaning | Why it matters here |
|---|---|---|
| Learned sparse retrieval | A retrieval method that uses a model to produce weighted vocabulary or token features, then searches those features through sparse index structures. | It is distinct from BM25 lexical matching and dense embeddings, so it needs an explicit store-capability check and slice-level evaluation before fusion. |
| Lexical search | Search based on the words, identifiers, or tokens that appear in a query and source. | It is often strong for exact policy IDs, product names, error codes, and rare terms. |
| Semantic search | Search that compares embeddings to retrieve material with related meaning even when the wording differs. | It can find paraphrases, but it can miss exact identifiers that lexical search handles well. |
| Hybrid retrieval | A retrieval strategy that combines lexical and semantic candidates before selecting context. | It is a baseline to test, not an automatic quality improvement for every corpus or query class. |
| Reranker | A later-stage model that reorders a bounded candidate set using the query and candidate content together. | It can improve relevance ordering but does not authorize sources or prove an answer is grounded. |
Knowledge Check
Test your understanding with this quiz. You need to answer all questions correctly to mark this section as complete.