AI API costs: the short answer
A business's AI API bill is almost always: input tokens times the input rate, plus output tokens times the output rate, multiplied by the number of calls. To control cost, control those three things: send fewer tokens, have the model write less, and use the right model for each job.
The major providers also offer official discounts for two common situations: prompt prefixes that repeat across calls (prompt caching) and work that does not need an immediate answer (batch APIs). This article contains no price table because prices change often; read current figures on Anthropic's pricing page or your own provider's pricing page.
How AI API costs are calculated
Large language models (LLMs) do not read characters; they read tokens: small pieces of text such as a word, part of a word or a punctuation mark. Providers bill per token, usually quoted per million tokens, with separate rates for input (what you send) and output (what the model writes).
Input includes more than you think
On every API call, the input is not just the user's question. It includes:
- The system prompt: instructions, rules, tone.
- Tool definitions, if the model can call tools. Anthropic's pricing documentation states that when tools are used, the API adds a special system prompt and those tokens count as input.
- Conversation history: in multi-turn chatbots and AI agents the whole history is usually resent on every turn, so input grows as the conversation gets longer.
- Reference material: RAG excerpts, attached files, tool results.
Output usually costs more than input
On the major providers' price lists, the output token rate is higher than the input rate. For models with a reasoning (thinking) mode, reasoning is billed as output too. So a task that asks the model to write a long report can cost more than a task that reads a long document and answers in a few lines.
Hidden costs beyond tokens
- Server-side tools such as web search can be billed per use on top of tokens (per Anthropic's pricing documentation).
- Some options, such as processing data in a specific geography, carry their own price multiplier.
- Errors and retries: a failed request that is re-run still paid for the failed attempt.
Does Vietnamese use more tokens than English?
Yes, with many common tokenizers. Research by Petrov and colleagues (University of Oxford, NeurIPS 2023) showed that the same content translated into different languages can differ greatly in token length, up to 15 times in some cases, which makes token-priced services more expensive for some language communities.
For Vietnamese specifically, a 2026 study from the National University of Singapore (Lee et al.) cites Petrov's results: GPT-4's tokenizer needs about 2.5 times as many tokens for Vietnamese as for English for the same content. That figure belongs to one specific tokenizer; newer models use different tokenizers, so the ratio may differ.
What to do: do not estimate costs from English material and apply them to Vietnamese. Take a sample of your real Vietnamese data, count tokens with the token-counting tool or endpoint of the exact model you will use, and budget from that. When choosing a model for Vietnamese, token efficiency is a criterion to measure alongside quality (see choosing an LLM for Vietnamese).
Five reasons AI API bills grow beyond plan
- Using the strongest model for everything, including simple classification or extraction.
- Bloated context: long system prompts, untrimmed conversation history, whole documents instead of relevant passages.
- Unbounded output: the model writes more than needed, or reasons heavily on simple tasks.
- AI agent loops: an agent calls tools repeatedly, retries without limit, or several agents hand work back and forth.
- Nobody owns the budget: one API key shared by many teams, with no limit and no alert.
10 ways to control LLM costs
The order runs from easy, high-impact changes to changes that need architecture work.
1. Match the model to the task (model routing)
Not every job needs the largest model. Classifying email, extracting fields and short summaries often work well on a small model; reserve large models for multi-step reasoning or high-stakes content. In practice: build a test set from real data, run it on two adjacent model tiers, and pick the cheapest tier that meets your quality bar. A step further is routing: a light classification step decides which requests go to the larger model.
2. Prompt caching for repeated prefixes
If many requests start with the same block (system prompt, tool definitions, a policy document), the provider can store that prefix and bill cache reads at a lower rate than normal input.
With OpenAI, caching applies automatically to prompts of 1,024 tokens or more and only works when the opening of the prompt matches exactly. With Anthropic, you mark cache points with the cache_control parameter; the default lifetime is 5 minutes, refreshed each time the cache is used, with a 1-hour option. Cache writes can cost more than normal input, so caching only pays off when the prefix is actually read again.
Design rule: put fixed content first and variable content (the user's question, RAG results) last. A timestamp or random ID at the top of the prompt will cause a cache miss on every call.
3. Batch APIs for work that can wait
Classifying thousands of support tickets, summarising a document backlog, running nightly evaluations: none of these need results in seconds. Both OpenAI and Anthropic offer batch APIs priced below synchronous calls, in exchange for asynchronous results. OpenAI states a 24-hour processing window; Anthropic says most batches finish in under an hour, with a 24-hour limit. Anthropic's pricing documentation says the batch discount stacks with prompt caching.
Do not use batch for tasks where someone is waiting, or for agent loops where each step depends on the previous one.
4. Trim and compress context
- Shorten the system prompt: remove duplicated instructions and examples you no longer need.
- Limit conversation history: keep the last few turns and summarise older ones.
- Load only the tools the current task needs, not every tool.
- Count tokens on the fully assembled prompt, not just on what the user typed.
5. RAG instead of stuffing whole documents
A large context window does not mean you should fill it. Every token in context is billed on every call that carries it. Beyond cost, the "Lost in the Middle" study (Liu et al., TACL 2024) found that model performance can drop significantly when relevant information sits in the middle of a long context. RAG (retrieve, then generate) puts only the relevant passages into the prompt, which is usually cheaper and makes sources easier to check. See what is RAG.
On the other hand, if the same long document is queried many times in a short period, placing it in the cached prefix can be reasonable. Measure both approaches on your own data.
6. Limit and shape the output
Set an output token limit (for example max_tokens) per task type, ask for concise formats or structured JSON instead of prose, and for reasoning models choose a reasoning level that matches the difficulty. Because output usually costs more than input, this is a very direct lever.
7. Cache answers at the application layer
Provider prompt caching only discounts input. If many people ask the same question (opening hours, return policy), the application can store an approved answer and return it without calling the model. You need a refresh rule for when the source data changes, and it should not be used for answers that depend on personal data.
8. Stop AI agent loops
AI agents that call tools over many steps are where costs most easily get out of hand. Set hard per-task limits on tool calls, token budget and running time; when a limit is reached, stop and hand over to a person. See also five checks for multi-agent orchestration.
9. Per-team budgets and alerts
Separate API keys and budgets by team, project or environment (dev, staging, production). OpenAI lets you set monthly spend limits at organisation and project level; by default a limit only alerts, and you can turn on a hard limit so requests fail once it is reached. OpenAI's documentation also notes enforcement is not instantaneous, so recorded spend can slightly exceed the limit. Anthropic lets you set spend and rate limits per Workspace in the Console, with notifications when spend reaches a threshold.
Good practice: set alerts at several levels before the limit, document who receives them and who may raise the limit, and check which limit will actually cut off traffic so production is not stopped by surprise.
10. Measure cost per useful task
A monthly invoice does not show where the money went. Log for each call: which feature, which model, input tokens, output tokens, cache-read tokens, and whether it succeeded. Then compute cost per completed task, not only monthly totals. A system that is cheaper per call but has to be re-run often can cost more than a pricier one that gets it right first time.
Cost metrics to track
| Metric | How to measure | What it tells you |
|---|---|---|
| Average input tokens per request | The usage field in API responses, by feature | Spots bloated prompts or context |
| Average output tokens per request | The usage field, by feature | Checks output limits and reasoning level |
| Cache read ratio | Cache-read tokens / total input tokens (OpenAI: cached_tokens; Anthropic: cache_read_input_tokens) | Shows whether prompt caching actually works |
| Share of eligible work run via batch | Batch requests / all eligible requests | Finds work still running synchronously for no reason |
| Spend by model tier | Cost grouped by model ID | Checks model routing |
| Cost per completed task | Total cost / tasks that met the bar | Compares options by real value |
| Error and retry rate | Failed or re-run requests / total requests | Finds wasted spend |
| Spend versus budget | Cost reports by project or Workspace | Early warning before a limit is hit |
A five-step way to start
Illustrative example: a customer service team runs a chatbot with a long system prompt and full conversation history. After measuring, they see most tokens are repeated input. They move the fixed policy text to the top of the prompt so it can be cached, keep the last five turns plus a summary, and move end-of-day ticket tagging to batch. This is a hypothetical scenario to show how the steps apply, not a result from a specific customer.
- Tag: every API call carries a feature, team and environment label; split API keys by project.
- Baseline: collect one to two weeks of tokens, cost and cache ratio by feature.
- Pick the two or three most expensive features and apply the easy fixes first: output limits, reordering prompts for caching, moving non-urgent work to batch.
- Try a smaller model against a quality test set; switch only if it meets the agreed bar.
- Set budgets, alerts and an owner for each team; review monthly.
How EYC helps
EYC designs and runs AI systems for businesses, and cost control is part of the architecture from the start: model routing, prompt structure for caching, per-feature cost measurement and per-team budgets. See cloud and data infrastructure and AI automation, or contact EYC to discuss your system.
Frequently asked questions
How are AI API costs calculated?
By tokens: input tokens (prompt, history, documents, tool definitions) times the input rate, plus output tokens times the output rate. Rates differ by model and are listed on each provider's pricing page.
Is the API cheaper than a ChatGPT or Claude subscription?
They serve different purposes. Subscriptions are for people working directly in a chat app; the API is for software that calls the model itself and is billed per token. To compare, estimate the real token volume of your workload and check it against current pricing pages.
Does Vietnamese use more tokens than English?
With many tokenizers, yes. A 2026 National University of Singapore study cites Petrov et al. (2023): GPT-4's tokenizer needs about 2.5 times as many tokens for Vietnamese as for English. The ratio varies by model, so count tokens on your own Vietnamese data.
What is prompt caching and when should I use it?
It is a mechanism where the provider stores a prompt prefix repeated across calls and bills cache reads at a lower rate. Use it when many requests share the same system prompt, tool definitions or documents, and requests arrive often enough within the cache lifetime.
What is a batch API good for?
Work that does not need an immediate answer: bulk classification or summarisation, data labelling, evaluation runs, periodic reports. It is not suited to chatbots with a waiting user or to agent loops where each step depends on the previous result.
How do I avoid sudden spikes in my API bill?
Split API keys and budgets by project, enable alerts at several thresholds, use hard limits where the provider supports them, cap steps and tokens for AI agents, and review cost by feature every week.