Last reviewed: 2026-10-25 · Prices from our dataset, rechecked against each provider’s official page · Price history

Every LLM API bills by the token, not by the word or the request. A token is the smallest unit of text a model reads or writes. It can be a whole word, part of a word, a number fragment or a punctuation mark. This guide shows how real tokenizers cut real text, how far the common rules of thumb are from a real count, and how to estimate the tokens in your own workload before you pay for them.

How text becomes tokens

A tokenizer has a fixed vocabulary of text pieces. Frequent strings get a piece of their own; rarer ones are spelled out from smaller pieces. Here is how two published tokenizers split a few strings. The dots stand for spaces: a leading space is part of the token that follows it.

Three things stand out. Numbers and dates break into many small tokens, so “$1,234.56” costs 6 tokens and a date costs 7. The same English word can be one token in one vocabulary and four in another (“unbelievably”). And languages that the vocabulary covers less well, here Greek, need more pieces per word.

Measured: the same paragraph in seven languages, plus code and JSON

We wrote a 53-word English paragraph about a customer-support inbox, translated it into six languages, added a short Python function and a JSON order record, and counted every text with both tokenizers on 2026-10-25. These are counts, not estimates.

What the table tells you:

• English is the cheapest language to send. The English paragraph came to 59 tokens, about 5.1 characters or 0.90 words per token.
• The same meaning costs more in other languages. The Spanish, French, German and Italian versions took 20–42% more tokens with o200k_base. Greek took 112 tokens, 1.9 times the English count, and 157 on DeepSeek’s tokenizer (2.6 times).
• Chinese is dense in characters, not in tokens. 88 characters produced 57 tokens (o200k_base), close to the English count for the same message.
• Structured data is expensive. The JSON record averaged only 2.3 characters per token because quotes, braces, keys and numbers are all short tokens. The Python function sat at 4.0. If you send JSON tool results or tables to a model, count them; do not estimate them from English ratios.

How good are the rules of thumb?

Providers publish rough conversion rules. Google’s Gemini documentation says a token is about 4 characters and that 100 tokens are about 60–80 English words. DeepSeek’s documentation says 1 English character is about 0.3 tokens and 1 Chinese character about 0.6 tokens, while noting that ratios vary by tokenizer.

Against our measurements, both rules overestimate English. Google’s 4-characters rule predicts 75 tokens for the paragraph; o200k_base counted 59. DeepSeek’s 0.3 rule predicts 90; DeepSeek’s own tokenizer counted 60. For Chinese the DeepSeek rule was close (53 predicted, 49 counted). That makes the rules safe for budgeting English (you will usually land under them) and unsafe for Greek, JSON or code, which ran well above 4 characters per token in the other direction.

Different models, different token counts

A token count belongs to a tokenizer, not to a text. Anthropic’s token-counting documentation says Claude Opus 4.7 and later models use a newer tokenizer that produces roughly 30% more tokens for the same input than earlier Claude models, and tells developers to recount prompts on the model they plan to use. So when you compare two providers’ per-token prices, remember that the token counts underneath are not identical either. The only way to know is to count your real prompts on each candidate model.

How to get an exact count

• OpenAI: the open-source tiktoken library counts offline. It maps GPT-5-family model names to the o200k_base encoding used above.
• Anthropic: the token counting endpoint returns the count for a given model. It is free to use but rate-limited, and Anthropic describes the result as an estimate that can differ slightly from the billed count.
• Google: the Gemini API’s countTokens method returns the count for a request.
• DeepSeek: publishes its tokenizer as a download for offline counting (the one used on this page).
• Everyone: every API response reports the input and output tokens it billed. After a day of real traffic, those usage numbers beat any estimate.

A five-step estimate for your own workload

1. Collect 20–50 real requests, not invented ones. Include the system prompt, earlier conversation turns, tool definitions and any retrieved documents: the model reads all of it, so you pay for all of it.
2. Count them with the tokenizer for the model you plan to use (or its count endpoint).
3. Record the output tokens too, from a test run. On reasoning models, thinking tokens are billed as output: Google’s price table, for example, labels the output price “including thinking tokens”.
4. Use the average for the monthly bill and a high percentile (say the 90th) to size limits and alerts.
5. Put the averages into the cost calculator, and recheck against your real usage numbers after launch.

Worked example: does the language of your documents matter?

Say you summarise 10,000 documents a month, each 1,500 English words long. At the measured 0.90 words per token, each document is about 1,670 input tokens. The Gemini “60–80 words per 100 tokens” rule would have budgeted 1,875–2,500.

On Gemini 3.8 Flash at $0.75 per million input tokens, the input side costs about $12.52 a month in English. If the same documents were in Greek, our measured ratio (1.9× the English tokens) puts it near $23.77, before a single output token. The model, the price and the task are the same; only the language changed. (Output for summaries is priced separately: see why output costs more.)

Prices above come from our Gemini pricing page data and update when we recheck it; Gemini 3.8 Flash is listed at $0.75 in / $3.75 out, checked 2026-10-25.

Read next