Skip to main contentSkip to content
Deep divePrompt caching / AI agents / Amazon Bedrock / Cost / Measurement

When a One-Hour Prompt Cache Pays Off for a Chat Agent

A chat agent's prompt cache expires between turns when people take more than five minutes to reply. A one-hour TTL took a measured session from 71.8% to 85.7% of input read from the cache and cut its cost 8%; with no pause, the same session cost 23% more.

Foxl TeamUpdated 14 min read

A hand-drawn closed book with four ribbon bookmarks of one colour hanging from its pages, each a little longer than the last: four cache breakpoints along one prompt, all with the same lifetime.
On this page
Reviewed against the shipped implementation and retained tests on October 30, 2026.

A one-hour prompt cache pays for a chat agent when the people using it often take longer than five minutes to reply. On Claude, a one-hour cache write costs twice the base input price and a five-minute write 1.25 times, so the longer entry has to save more in reads than it adds in writes. For Foxl's cloud agent, which answers people from their phones while their computer is off, it did. In a measured nine-call session the share of input read from the cache went from 71.8% to 85.7%, and the session cost 8% less. With the session's one pause taken out, the same calls cost about 23% more on the one-hour setting.

Measuring the hit rate

The hit rate is the share of a call's input tokens that the provider read from its cache instead of processing again: cache_read / (uncached input + cache_read + cache_write). A cache breakpoint is a cache_control marker on one block of the request. It asks the provider to cache everything up to that block, the prefix, for the entry's TTL (time to live), five minutes by default, and the next request that starts with the same prefix reads it instead of paying for it again. Bedrock calls breakpoints checkpoints.

Every Messages API response reports three input counters, and Anthropic's documentation gives their relationship: total_input_tokens = cache_read_input_tokens + cache_creation_input_tokens + input_tokens. So input_tokens counts only what was neither read from nor written to the cache, and dividing cache_read by input_tokens gives a different and much larger number. Bedrock's Converse API says the same of its own field: with caching on, inputTokens "represents only the non-cached input tokens".

We report two rates for every run. The rate over all calls includes the first call after the cache expired, which always writes. The rate from the second call on shows what a conversation costs once its cache is filled.

Why our agent missed its cache

Foxl's billing records for the cloud agent, from its launch to the change, cover 19 turns in 8 conversations: 35,054 tokens read from the cache, 47,173 written and 72 uncached, a hit rate of 42.6%. The agent wrote more than it read. At least 12,119 of the tokens it paid to cache (47,173 minus 35,054), about a quarter, were never read back.

Foxl's relay, the server between Foxl's apps and Bedrock, put three breakpoints with the default five-minute TTL on every Claude request: on the last tool, on the system prompt and on the last user message. Calls made close together hit; a follow-up sent 15 seconds after a reply read 3,616 tokens and wrote 44. Every large write came after a gap longer than five minutes. A message sent 26 minutes after the previous reply, and a chat opened two hours after the last one, each wrote the whole prefix again, about 3.6K tokens of tools and system prompt.

A hit also refreshes the entry at no cost ("The cache is refreshed for no additional cost each time the cached content is used"), so an agent working through a loop of tool calls seldom reaches the five-minute limit. A chat with a person reaches it between turns, which is where such a chat spends most of its time. AWS's guidance for the one-hour TTL describes this case: "when storing a long chat conversation with a user and you generally expect that user may not respond in the next 5 minutes."

The trade, in multiples of the base price

On Anthropic's price list a five-minute write costs 1.25 times the base input price, a one-hour write 2 times and a read 0.1 times, or 0.05 times on Claude Opus 5.5 and Sonnet 5.5. On those two models:

  • each prefix token that a pause would have made you write again saves about 1.2 times the base price, a 1.25 write replaced by a 0.05 read;
  • each token written while the one-hour setting is on costs 0.75 times the base price more, 2 instead of 1.25.

The one-hour TTL wins when the first sum is the larger. A chat with pauses of five to sixty minutes usually gets there, and so does a second chat started within the hour that shares the prefix. A chat that is over within five minutes, with nothing else in that hour, does not: it pays the premium on every write and never reads one back. For people answering from a phone we chose one hour. For an agent whose calls come seconds apart, five minutes is the better setting.

What we measured

We ran the cloud agent's production code through a local copy of our relay against real Bedrock, once with the relay's five-minute breakpoints and once with the agent's own one-hour ones. The scenario is one chat of four turns (a plan tracked with a tool, a follow-up, a file save, then a question about the first turn 6.5 minutes later) followed by a new chat. That is nine model calls on Sonnet 5.5, because a turn that uses a tool makes more than one. Usage was read from the stream Bedrock returned, and the cost uses Foxl's own per-token prices for Sonnet 5.5.

setupcache readcache writehit, all 9 callshit, calls 2-9cost of the run
five-minute (the relay's breakpoints)29,97911,755 (5m)71.8%78.7%$0.0538
one-hour (four breakpoints)35,8595,974 (1h)85.7%93.9%$0.0494
one-hour, plus memory files (see the last section)177,34720,108 (1h)89.8%98.6%$0.1219

Nearly all of the difference is the call after the 6.5-minute pause. With five-minute entries it wrote its 5,873-token prefix again. With one-hour entries it read 5,750 tokens and wrote 133, and Bedrock reported every write as ephemeral_1h_input_tokens, which is how we know the longer TTL was applied and not silently ignored. The one-hour run cost 15% less on input although each of its writes cost more. Uncached input was 28 tokens in every run.

Take the pause out of the same measurements and the result turns around. The five-minute setup then never rewrites its prefix, and the one-hour run costs about 23% more in total and 48% more on input.

The five-minute setup scores 71.8% here and 42.6% in production because this scenario has one gap of 6.5 minutes, while the production turns had gaps of 26 minutes and two hours. The same code scores very differently on different traffic, so compare hit rates only between runs of the same scenario.

Where the four breakpoints go

A Claude request can carry at most four breakpoints (Anthropic: "You can define up to 4 cache breakpoints"; Bedrock lists 4 for every Claude model). The cached prefix is read in the order tools, system, messages, and a change anywhere invalidates everything after it. The relay's breakpoints were all five-minute ones, so the cloud agent now sets all four itself, each with its TTL, and the relay leaves alone a request that already carries breakpoints.

One Messages request drawn as a row: tools, the shared system prompt, the person's part of the system prompt, the earlier messages, the previous user message and the newest message. Four numbered markers sit at the end of the shared system part, the end of the system prompt, the previous user message and the newest message. The first ends the part shared by every chat, the second ends this person's part, the third is where the previous call wrote, and the fourth is the write the next call reads.
Figure 1. The four breakpoints of a cloud call. Everything before the first one is the same for every conversation of every person on one model.
  1. The end of the shared part of the system prompt. We split our system prompt where the per-person memory begins, at a ## Memory heading. The tools and everything above that heading are identical for every conversation of every person on one model, so a brand new chat from anyone starts by reading them.
  2. The end of the system prompt. This adds the person's own memory and a date line. The date line gives the day, not the time, so it does not change the prefix between calls.
  3. The last cacheable block of the previous user message. This is exactly where the previous call wrote, so reading it does not depend on the API's look-back. Without it, the API starts at the newest breakpoint and looks back for an earlier entry only so far ("The lookback window is 20 blocks"), and a turn with many tool calls can push the previous write out of reach. Anthropic's API counts a run of consecutive tool_use or tool_result blocks as one position; Bedrock's page says only "approximately 20 content blocks".
  4. The last cacheable block of the newest message. This is the write the next call reads.

All four use the same TTL. Bedrock accepts two TTLs in one request only if every one-hour entry comes first ("a 1-hour cache entry must appear before any 5-minute cache entries"), and one TTL for all four keeps that true without any checking. Breakpoints go only on text, image, document, tool_use and tool_result blocks, the message content Anthropic lists as cacheable.

The one-hour TTL goes only to the Claude models that Bedrock's table lists with "Supported TTL: 5 minutes, 1 hour". Bedrock warns that sending the field to a model that supports only five minutes "can result in a ValidationException", so the other Claude models get the same four breakpoints without a ttl. GPT-5.6 and later on Bedrock use their own prompt_cache_breakpoint field and, per the same page, a "30-minute minimum TTL", so we leave them alone.

// the marker on each of the four blocks, on a model that takes it
"cache_control": { "type": "ephemeral", "ttl": "1h" }

There is also a minimum size. Bedrock caches a checkpoint only if everything before it reaches the model's minimum length: 512 tokens on the 5.5 models, and 1,024 or 4,096 on some older ones. Below the minimum, "your inference still succeeds, but your prefix isn't cached". On a model with a 4,096-token minimum, a 3.6K-token prompt leaves the tools-and-system breakpoints caching nothing until the conversation itself pushes the prefix past the minimum, so check the table for every model you serve.

Four production calls since the change

By October 27 production had served four cloud calls since the change, in four turns, too few for a hit rate worth quoting; together they read 73% of their input from the cache. The last of them shows the first breakpoint at work. It was the first call of a new chat on a different account, 26 minutes after the previous chat's last call. It read 3,542 tokens from the cache, the tools and the shared part of the system prompt that the other account's chat had written and refreshed, and wrote 108. Under five-minute entries it would have written all of them again, as the 26-minute gap in the baseline did.

What sets the ceiling

Every new user message and every reply is written to the cache once before it can be read, so no placement of breakpoints lets a call read the tokens it is adding. In the one-hour run the first call read nothing, because the earlier run's five-minute entries had expired, so all 35,859 tokens read came from calls 2 to 9: 93.9% of the roughly 38,200 input tokens they took. The other 2,300 or so were that run's new messages and replies, which makes 93.9% the ceiling for that scenario. A longer chat carries more history per new token, so its ceiling is higher.

A bigger stable prefix raises the rate without making anything cheaper. The third row of the table is the same session with the person's memory files in the prompt: four Markdown files (SOUL.md, USER.md, MEMORY.md and AGENTS.md) that Foxl's desktop app puts in its own prompt and that the phone or browser now sends along with each cloud message. One real set of these files came to about 41K characters, roughly 14K input tokens per call, and the run used a test file of the same size. Its first call wrote 17,619 tokens of the person's prompt, every later call read them, and the rate from the second call on reached 98.6%. The run cost $0.1219 against $0.0494, because the prompt is several times larger; without the cache the same nine calls would have cost $0.4187. It had the best hit rate of the three runs and the largest bill, so we report the cost next to every hit rate.

The same split on your computer, and where to see it

Since v0.9.7 the desktop app does the same for a chat on Claude through your own AWS account. Foxl's tools and instructions stay in Amazon Bedrock's cache for an hour, while each conversation's own part keeps the five minutes. The two parts get different lifetimes because the tools and the system prompt are the same for every chat and change only when you edit a file or a setting, while a conversation's part grows with every tool call and is read again seconds later. Measured once with Sonnet 5.5 on a real install, a new chat 6.5 minutes after the last one started answering after 1.7 seconds instead of 2.2, and wrote about 850 tokens to the cache instead of 57,600. The first chat after more than an hour away costs a little more, and the cost Foxl shows counts the hour's writes at twice the input price.

There is nothing to turn on. To see what the cache does for your own chats:

  1. Open Settings, then Usage, and press Usage Explorer.
  2. On Overview, In this range shows Cache share, the part of the prompt read from the cache (the hit rate defined above), beside the input, output, cache-read and cache-write totals (Figure 2).
  3. Models splits the same four counters per model (Figure 3). Pick 7 days, 30 days or All time at the top.
Usage Explorer's In this range block for 30 days: most used model Sonnet 5.5, busiest day Oct 30 at $0.216, $0.216 per active day, cache share 74% of the prompt read from cache; input 20, output 1.6K, cache read 150.3K, cache write 53.0K.
Figure 2. A test computer after two chats on Claude Sonnet 5.5 through Amazon Bedrock, on Foxl 0.9.8: 74% of the prompt came from the cache, 150.3K tokens read and 53.0K written against 20 not cached.
The Models tab: Sonnet 5.5 on Bedrock, $0.216, 6 requests, input 16, output 1.6K, cache read 150.3K, cache write 52.2K; Haiku 5.5 on Bedrock, $0.0001, 1 request, input 4, output 58, cache read 0, cache write 810.
Figure 3. The same counters per model. As the page itself notes, it shows what this computer recorded, not usage on other devices.

What we have not measured

  • Other models. We ran only Sonnet 5.5. The other Claude models that get the one-hour TTL get it on the strength of Bedrock's table and, for Opus 5.5, an earlier measurement through our relay.
  • Other scenarios. Nine calls in one pattern, none of them a turn with many tool calls, which is the case the third breakpoint is for. A chat that sits idle for more than an hour pays the first write again; one hour is the longest TTL either provider offers for Claude.
  • Production at volume. Four calls show that the mechanism works; they are too few for a rate.

To check an agent of your own, add up the three counters per call, divide reads by all input, and look at what came before each call that wrote a large prefix. A gap longer than five minutes is the one-hour TTL's case. A call made seconds earlier means the prefix changed between the two calls, or the previous write was out of the look-back's reach, which the third breakpoint prevents. An earlier post covers keeping a subagent on its parent's cache, the same prefix rule inside a single turn.

References and further reading

  1. Anthropic: Prompt cachingDocumentation
  2. Amazon Bedrock: Prompt caching for faster model inferenceDocumentation
  3. Foxl v0.9.3-beta.5 releaseRelease