Embeddings and ANN Mechanics

An embedding maps content and queries into vectors. Record model, dimension, normalization policy, index metric, source version, and migration plan together.

Cosine, inner product, and Euclidean distance are not interchangeable defaults. Select the metric and normalization that the embedding and vector-store implementation expect.

Exact nearest-neighbour search establishes a reference result set. Approximate search limits work to meet latency or memory constraints, then its recall is measured against that reference on representative queries.

View full size

AWS Prescriptive Guidance: Vector database RAG flow figure (CC BY-SA 4.0)Last verified: 2026-08-26

HNSW exposes construction and search effort; IVF exposes partition and probe choices. More candidate work can improve recall but add latency or memory. Test inserts, deletes, filters, rebuilds, and concurrency too.

1. Keep an exact or high-quality reference set.
2. Measure Recall@K, latency percentiles, memory or storage, and update behaviour before and after a representation or index-compression change.
3. Re-embed and re-index as a versioned rollout; never mix incompatible dimensions in one field.
4. Retain source and chunk identifiers for debugging.

An embedding is not just a number array. It is part of a compatibility contract across the document pipeline, query path, vector field, metric, metadata mapping, and evaluation set. A new model, dimension, normalization rule, or chunking profile is a retrieval migration until proven otherwise.

AWS’s knowledge-base build guidance should be used to confirm the selected vector-store integration and field expectations for the actual configuration. Do not copy a dimension, metric, or model identifier from an unrelated store, Region, or tutorial. Record what was verified and test it with representative documents before the corpus becomes authoritative for a user workflow.

Use a small, fixed query set with relevance judgments and compare exact rankings under the metric and normalization policy you intend to deploy. The point is not to prove a universal winner; it is to catch a mismatch where query vectors, document vectors, and index scoring no longer mean the same thing.

Measure both ranking overlap and the decision the product makes from that ranking. For example, a top result may still be unacceptable if it crosses an access boundary or omits a necessary clause. Agentic AI Lens PERF03 recommends optimizing memory, context, and RAG against workload needs; this experiment makes one of those context-quality assumptions inspectable before it reaches an agent loop.

ANN is a deliberate approximation: it can reduce search work, but must be evaluated against an accepted reference on representative queries. Include filtered queries, source updates, deletes, and concurrent access because a strong unfiltered static benchmark can hide the operational retrieval behaviour users experience. Avoid a single global recall target; set a workload-specific budget with the consequence of a miss in mind.

1. Create a new compatibility record and index namespace; never overwrite the only known-good representation.
2. Backfill from versioned source chunks and verify document counts, metadata mapping, and retrieval samples.
3. Compare baseline and candidate results on the same golden set, including no-result and authorization cases.
4. Cut over through a reversible alias or application configuration only after the measured trade-off is accepted by the workload owner.
5. Monitor disagreement and retrieval regressions after cutover; restore the prior alias if the pre-agreed rollback condition is met.

An index rebuild changes more than query speed. It can change field mappings, metric behavior, filters, candidate selection, freshness, and the ability to explain a result. Write the cutover decision before backfilling so the team knows which evidence is needed to move traffic and exactly when to restore the prior representation.

AWS vector-store setup documentation identifies vector, text, and metadata field requirements for supported integrations. Verify those requirements against the exact store and configuration you operate before backfill. A dual read is valuable because it reveals a result disagreement while the prior index still supplies a recoverable baseline; it is not proof that every result should be identical across two intentionally different representations.

Keep writes and reads explicit during the comparison. A candidate that is not receiving the same source updates as the current index cannot be evaluated fairly, while a shared write path can conceal a mapping failure if it silently drops metadata. Record the corpus watermark and ingestion status for both representations with every dual-read sample. If parity cannot be established, stop the comparison rather than interpreting disagreement as an ANN-quality result.

Aggregate recall can conceal the query classes that matter most. Slice the experiment by source type, query intent, metadata filter, language or vocabulary pattern where applicable, and change age. The goal is to find a representation drift that breaks a meaningful retrieval decision before it becomes an apparent model-quality issue downstream.

Keep an exact or high-quality reference where practical, then calculate recall on each slice with human-reviewed expected source identifiers. The Agentic AI Lens asks workloads to optimize memory, context, and RAG for their needs; recall slices make “need” concrete without inventing a universal target. Treat an unexplained slice regression as a release blocker when that slice maps to a user decision the workload claims to support.

Investigate the direction of disagreement, not just its count. A candidate that returns more broadly related sources may increase apparent recall while pushing the decisive source below the context budget; a candidate that returns fewer sources may improve precision but leave a comparison question under-evidenced. Capture the reviewer’s decision consequence for each disagreement, then use that evidence to choose whether the representation, ANN controls, chunking, or downstream context policy needs to change.

1. Freeze the corpus, embedding version, filter configuration, and judged query set for the comparison window.
2. Run exact/reference and candidate ANN retrieval with the same principal and filter conditions.
3. Record source-ID overlap, missing expected evidence, extra unexpected evidence, and user-decision consequence per slice.
4. Investigate representation, chunking, metadata, and index configuration separately; roll back the candidate if the accepted slice contract is breached.