Build agentic retrieval on a Managed Amazon Bedrock Knowledge Base with CDK

Amazon launched a Managed Amazon Bedrock Knowledge Base, Type MANAGED, so retrieval is no longer a single embed-and-fetch call. The AWS Machine Learning Blog's report deploys that design as one AWS Cloud Development Kit application in Python: an agent routes across two knowledge bases, retrieves through the Amazon Bedrock AgentCore Gateway, and returns a grounded, cited answer. AgenticRetrieveStream reranks internally. One command deploys the stacks. Two Amazon CloudWatch dashboards cover seven telemetry layers, with on-demand and continuous evaluation in the same application.

This piece is for teams whose Retrieval Augmented Generation answers span two corpora, and for engineers choosing between a managed store and Amazon OpenSearch Serverless. It carries the August 2024 CDK sample forward. The Amazon Simple Storage Service path stays familiar, the vector store is gone, and roughly half of the application records what the agent did. New readers can use the design on its own.

Build agentic retrieval on a Managed Amazon: what actually changed

In August 2024, "Build an end-to-end RAG solution using Knowledge Bases for Amazon Bedrock and the AWS CDK" deployed single-shot RAG as hand-sequenced CDK stacks: an Amazon S3 corpus, an Amazon OpenSearch Serverless collection and index, a knowledge base, and a data source. Classic RAG does one retrieval and one generation. A question that spans two corpora gets half of what the model needs, which is worse than returning nothing, because the answer comes back fluent and incomplete. A single index also cannot choose the source or take a second look.

Three things changed. Type MANAGED owns ingestion, storage, indexing, and retrieval, so the OpenSearch Serverless collection is no longer created, scaled, or paid for with OpenSearch Compute Units. Amazon Bedrock AgentCore adds a Gateway that exposes each knowledge base as a Model Context Protocol tool, a Runtime for the reasoning agent, and AgenticRetrieveStream, which returns a synthesized, cited answer. Agentic retrieval runs only on the Managed Knowledge Base. One deployment also provisions two CloudWatch dashboards across seven telemetry layers, plus on-demand and continuous evaluation.

Build agentic retrieval on a Managed Amazon: how it works

The agent decides whether to retrieve, which knowledge base to consult, and whether to go back before it answers. AgenticRetrieveStream is a first-class feature: it plans across turns, runs retrieval, and returns citations, rather than an orchestration layer over raw vector search. Semantic routing picks the knowledge base, and retrieval can repeat before the grounded answer. The Gateway exposes each of two knowledge bases as a tool. The bases search and cite, the loop repeats as the question requires, and every step lands in Amazon CloudWatch.

Amazon Bedrock manages embedding and reranking with service-managed models by default, at no extra cost, and the data store auto-scales. There is no vector database to provision, scale, or patch, and no OCU capacity. On a customer-managed knowledge base the team chooses the vector store, often Amazon OpenSearch Serverless, and configures embedding and reranking. AgenticRetrieveStream and AgentCore Gateway integration are supported only on Type MANAGED. Customer-managed remains right for an existing OpenSearch Serverless collection or for data-residency, tuning, or compliance. Managed infrastructure is none.

Build agentic retrieval on a Managed Amazon: why it matters now

A single-shot pipeline can be read as a function. An agent that retrieves in a loop takes a different path on every question, and by default neither the path nor the answer quality is visible. Production needs latency, call volume, and token usage, and a continuous read on retrieval and answer quality. Two CloudWatch dashboards span seven telemetry layers, with on-demand and continuous evaluation from the same CDK application. Citations trace answers to source passages. Faithfulness and correctness scores catch drift, and those layers support responsible use.

Four densely cross-referenced stacks are why CDK fits. The Gateway needs both knowledge base ids. The agent needs the Gateway URL and ARN. The dashboards need the knowledge base ids, the Gateway ARN, and the agent runtime id. Those seven values cross stack boundaries, each a metric dimension or a connector target. The exports and the ordering script drift when a name changes. Passing a CDK construct derives the export and the dependency edge. The 2024 sample had three stacks and a mostly linear chain.

Build agentic retrieval on a Managed Amazon: who is affected

Teams whose answers span two corpora meet this failure. One retrieval returns half of what the model needs, and the reply is fluent and incomplete. The index cannot choose a knowledge base or look again after the first result. Semantic routing across two knowledge bases is the agent's loop. Hand-sequenced 2024 stacks, an S3 corpus, an OpenSearch Serverless index, and a data source, become a Python CDK application with the vector store gone. One retrieval and one generation still suit a large class of questions.

The AgentCore Gateway connector and AgenticRetrieveStream exist only on Type MANAGED, so teams that want the Gateway and the Runtime use the managed knowledge base. Teams that operate their own vector store, reuse an OpenSearch Serverless collection, or must meet data-residency, tuning, or compliance rules stay customer-managed and provision, scale, and maintain the store. Managed infrastructure is none. Do-it-yourself infrastructure is the vector database and more. The S3-to-knowledge-base path stays familiar, and roughly half the application records what the agent did.

Build agentic retrieval on a Managed Amazon: what to watch

Seven telemetry layers sit on two CloudWatch dashboards: latency, call volume, token usage, and retrieval and answer quality. On-demand and continuous evaluation come from the same CDK application. Faithfulness and correctness scores flag drift from source passages, and citations keep responses auditable. Figure 1 shows the agent deciding whether to retrieve and which knowledge base to consult, the Gateway exposing each base as a tool, the managed bases searching and citing, and the loop repeating for as long as the question needs.

Seven values cross four stacks: both knowledge base ids into the Gateway, the Gateway URL and ARN into the agent, and the knowledge base ids, Gateway ARN, and agent runtime id into the dashboards. Raw exports drift on a rename. CDK derives the dependency by passing the construct, and one command deploys the Python application. AgenticRetrieveStream runs only on the Managed Knowledge Base. A customer-managed OpenSearch Serverless collection for residency or tuning loses Gateway integration. OpenSearch Compute Units stay out because Amazon Bedrock owns storage and indexing.