Groundhog Day for AI agents: Prompt caching patterns for using OpenAI

Amazon Bedrock stores the input prompt, instructions, tools, and content, and reuses it on the next request at a fraction of the input price. AWS Machine Learning Blog's report shows the pattern for OpenAI models on Amazon Bedrock, model id us.openai.gpt-6-astra. With no internal memory between requests, the model predicts the next token only from that input, so each step resends tool definitions, instructions, every previous tool call, and every result. By turn 20 of a coding session, turns 1 through 19 are re-read and paid again.

This piece follows one agent from its first turn through new tools and helper agents. It is for developers who already have an Amazon Bedrock client, a TOOLS list, and a task, and who need that shared prefix to lower input cost while the agent stays responsive and consistent. It covers prefill, implicit and explicit marks, the 1,024-token minimum, a ttl of 30m, and the loop that appends tool results until a final answer.

Groundhog Day for AI agents: what actually changed

The report opens on the movie Groundhog Day. Bill Murray's character re-lives the same February 2, waking to the same song on the radio, in the same inn, in the same small town, for what some movie fans estimate to be 34 years. Nothing he accomplished the day before carries over unless he works it out again from scratch. An agent is in the same loop: every tool call, result, and next decision has to send the entire story so far.

A first call to the large language model is four blocks. The tool list defines every callable function and changes when a new release ships. The instructions are the developer prompt: rules, persona, policies, and examples, and they change per release. History holds the task, the model's tool calls, and the tool results, and it grows every turn. The new turn is the latest tool result or user message. Figure 1 shows that payload on every turn. Prompt caching stores the input and reuses the matching prefix.

Groundhog Day for AI agents: how it works

Reading a prompt starts with a write called prefill. During prefill the model processes every input token and builds key and value vectors, also called key-value or KV vectors. Those vectors are its working state, built before it writes the first token. Prompt caching saves that state, including the system prompt, tools, content, and history. A later request that starts with the same text loads the saved state instead of computing it again, then processes only what is new in the next payload.

Caching can be implicit or explicit, and the report recommends using both. Implicit caching lets Amazon Bedrock choose where the reusable prefix ends as the conversation grows. Explicit caching uses prompt_cache_breakpoint so tools and instructions are reused across sessions and helper agents. Both forms are not always required. Combined, a new session reuses that shared start and keeps its own cached history. Figure 2 shows turn 1 saving the prefix up to the breakpoint and turn 2 paying full price only for the new question.

Groundhog Day for AI agents: why it matters now

The first saving is static tools and instructions. Samples use the existing Amazon Bedrock client, TOOLS, and task, and replace the shortened INSTRUCTIONS string, "Your full support policies and examples...", with real policies and examples. The reusable prefix must reach 1,024 tokens to qualify. Explicit mode chooses where the saved prefix ends. Rendered order is provider context, then TOOLS, then INSTRUCTIONS, with the task after the boundary. The cache starts at the beginning, the marker picks the end, and TOOLS stay before the instructions.

The call is client.responses.create with model us.openai.gpt-6-astra, store set to False, and include set to reasoning.encrypted_content. store=False leaves response storage off while prompt caching still works. prompt_cache_key support-agent:v7:gpt-6-astra names which requests share a cache. prompt_cache_options uses mode explicit and ttl 30m. The developer role sends input_text with prompt_cache_breakpoint mode explicit, saving that block and TOOLS. The user role's task is processed but not cached. One test wrote 4,120 tokens for a roughly 4,000-token prefix and read those 4,120 tokens on later requests.

Groundhog Day for AI agents: who is affected

Developers who ship the prompt have to keep an exact match. Caching works only on exactly the same text, so instructions and tool lists must be serialized the same way every time. Model Context Protocol servers and other tool delivery need to be indexed so the list stays consistent. The date, the user's name, and a request ID go after the cached instructions. The key support-agent:v7:gpt-6-astra marks a versioned support agent on gpt-6-astra, shared by later sessions and helper agents.

The model requests tools by name and supplies arguments. The application must execute them. execute_function is a helper the developer builds to route each call to the matching Python function and return its result. TOOL_HANDLERS maps names to those functions. The loop appends the model output and every tool result to history and resends that list until a final answer. In implicit mode, Amazon Bedrock reuses the unchanged prefix and caches added content through the last tool result when earlier messages and the initial TOOLS stay unchanged.

Groundhog Day for AI agents: what to watch

Implicit mode follows the conversation as tools and content vary. ask_with_implicit_cache calls client.responses.create on us.openai.gpt-6-astra with store=False, include reasoning.encrypted_content, the same prompt_cache_key, mode implicit, ttl 30m, and tools=TOOLS. History is the input, and everything gets cached. Implicit is the default. With no explicit marker, the user task is the latest eligible message, so the breakpoint at its end covers tools, instructions, and the task. store=False keeps history in the application and does not disable caching. That fits the next turn of the same task.

A different task does not match a prefix that runs through the old task, and the implicit write does not necessarily create a separate entry of only the tools and instructions. An explicit marker makes that shorter prefix reusable across tasks. The sample sets prompt_cache_breakpoint mode explicit on the developer role, then adds the user role task, and calls ask_with_implicit_cache with request-level mode implicit. The service can also save through the latest eligible message, so the shared prefix remains for later turns, new tools, and subagents.