Summary
Input tokens were 70% of the $58.55 we spent on Claude models in October 2026; output tokens were 13%. That split is why we keep long documents in a knowledge base instead of in the prompt.
The cost logs do not say which days ran with the knowledge base on or off, so this article does not claim a measured saving. It does three things instead:
- It explains why agents that read long documents are input-heavy, and how a knowledge base, result offloading and prompt caching each act on that.
- It shows what the October logs do show. On days without caching, input was 93–95% of the day’s cost. On 22 and 23 October, when caching was on, plain input fell to 53% and 22%.
- It lists what to measure next, so the saving can be stated as a number.
The setting
The engagement built two agents on Amazon Bedrock for a scientific-software client. The client and the delivery partner are not named here.
- The connector builder reads a vendor’s API documentation and sample responses and writes connector code. A named person approves the field mapping before any code is written. A test harness then builds, lints and tests the code, with up to three repair rounds.
- The ASM transformer turns raw instrument-software JSON into a standard scientific data format (Allotrope ASM, liquid chromatography). For the system we tested it uses an open-source conversion library, not a model.
Both agents run on Claude models through Bedrock. The configuration sets Claude Haiku 4.5 for the mapping phase, Claude Sonnet 4.5 for the code phase, and a cap of 32,000 output tokens per call.
The builder is where token cost matters. Vendor documentation is long, the agent returns to it many times, and the repair rounds repeat the whole loop.
Where the tokens go
An agent pays for four kinds of tokens, and the one that grows fastest is the one nobody types: the conversation history.
| Token type | What it is | Why it grows |
|---|---|---|
| Input | Everything sent to the model on a call: instructions, tool results, and all earlier turns | The full history is sent again on every turn |
| Output | What the model writes back: reasoning, tool calls, code | Grows with the size of what is generated |
| Cache write | Input stored so a later call can reuse it | Billed above normal input, so it pays off only if reused |
| Cache read | Stored input reused on a later call | Billed at a fraction of normal input |
Take an agent that reads a 60,000-character chunk of vendor documentation. At the common rule of thumb of four characters per token, that is about 15,000 tokens. If the chunk stays in the history for the next 20 turns, it is billed as input 20 times: roughly 300,000 tokens for one read. This is arithmetic on stated assumptions, not a measurement from our runs.
The lever is therefore not a shorter answer. It is keeping bulky tool results out of the history that gets re-sent.
What a knowledge base and result offloading change
We use three separate mechanisms, each aimed at the input side. They are configured in the builder; none of them was switched on and off in a controlled test.
- Knowledge base. During preparation, the vendor documents and guidance are split into passages and indexed once, using Bedrock Titan embeddings. The agent calls a search tool and receives a few relevant passages instead of loading whole documents. Embedding is a one-off cost per document set.
- Result offloading. Any tool result longer than 4,000 characters is stored outside the prompt and replaced in the history by a short note. The agent can search the stored text later. This stops a long file read from being re-billed on every later turn.
- Prompt caching. The part of the prompt that does not change between turns is stored and reused. The setting is automatic.
The trade-offs are real. Search can miss the passage the agent needs, and each search is an extra tool call, which is an extra turn. We kept a rule that names in the mapping must appear in the vendor documents or samples, so a missed passage shows up as an unknown field rather than an invented one.
What the October cost logs show
Claude model tokens were $58.55 of October’s $82.45 bill, and seven days account for 98% of that. The rest of the bill was tax ($13.74), other Bedrock models ($10.08) and under $0.10 of other services. About $0.45 of the other-model line was Titan embeddings, the model the knowledge base index uses; the file does not show which jobs used it.
Read the input-share row, not the bar heights. Height reflects how much work ran that day, and the logs do not record that. Caching appears only from 22 October, and on 26 October the two regions behaved differently: Frankfurt shows none, London shows cache reads and writes.
Claude spend a month earlier, in September, was $1.15. October is the first month with real volume.
What we can and cannot conclude
The logs support a description of the cost, not a measurement of the knowledge base’s effect.
What the logs support
- Input is the dominant cost: 70% of Claude token spend over the month, and 93–95% on days without caching.
- Spend is concentrated. Three days (13, 22 and 26 October) made up 76% of the month.
- Caching changes the mix. On the two days it was on, plain input fell to 53% and 22% of the day’s cost.
What they do not support
- A saving from the knowledge base. The files carry no flag for knowledge base on or off, no run identifier, and no model per usage line.
- A saving from caching. 22 October, a cached day, was also the most expensive day at $21.14. A heavier workload could explain that; the logs cannot say.
- A comparison between regions. On 26 October the Frankfurt runs cost $13.00 with no caching and the London runs cost $2.37 with caching, but we do not know that the two did the same work.
The month also mixes in things that blur any comparison: repair rounds and failed runs cost the same as good ones, the bill includes experiments on other Bedrock models, and the Sonnet and Haiku split is only available as a monthly total.
What to measure next
- Tag every model call with a run identifier, the phase, and whether the knowledge base was on.
- Record the four token counts from each Bedrock response, not only the daily bill.
- Run the same vendor documents at least three times with the knowledge base on and three times off, then compare tokens per finished connector.
Practical guidance
If your agent reads long documents, start with the input side. These are the steps we would take again, in order.
- Index reference documents once, and give the agent a search tool instead of the full text.
- Store any long tool result outside the prompt and leave a short note in its place.
- Turn on prompt caching for the part of the prompt that does not change, and check that cache reads actually appear in the bill.
- Put a cheaper model on the phases that need less reasoning, and keep the larger model for code.
- Put a person-approved gate before expensive generation, so a wrong mapping is caught before code is written.
- Log the four token counts per run, with a run identifier and the knowledge base setting, so the next review can state a saving as a number.
Do not report a saving until the last item is in place.