Tokens in Hindi
How many tokens does Hindi text use, compared with English? We measured the same copy in both languages with OpenAI's two main tokenizers.
What the numbers say
The same website copy is 200 tokens in English and 298 tokens in Hindi on the newer o200k_base encoding, a ratio of 1.49×. On the older cl100k_base encoding the Hindi copy needs 878 tokens, which is 4.39× the English count of 200. At the same price per token, the same message costs about 1.49× as much in Hindi on the newer encoding. Among the 29 non-English languages we measured, Hindi ranks number 23 from the cheapest.
The newer encoding made a real difference: o200k_base used 66% fewer tokens than cl100k_base for Hindi, compared with 34% across all the non-English languages in the data. That is more than average, so Hindi gained more from the newer vocabulary than most.
Why Hindi? Hindi is written in Devanagari, where a visible syllable can be built from several Unicode characters: a consonant, a vowel sign, and sometimes a joining mark. Each of those characters takes three bytes in UTF-8, so text that looks short on screen contains a lot of bytes for a tokenizer to cover.
Indic scripts build a visible syllable out of several Unicode characters, and every one of those characters takes three bytes in UTF-8. Older tokenizers saw little text in these scripts, so they cut words into many small pieces. Newer vocabularies are larger and include more of these scripts, which is why the improvement from the older to the newer encoding is largest here.
How a Hindi sentence is split into tokens
This is the tagline of this site in Hindi. Each shaded block is one token (or a few tokens that together make one character) on o200k_base.
71 characters become 25 tokens on o200k_base and 69 tokens on cl100k_base. The English tagline, "Copy text on one device and paste it on another — no app, no account.", is 69 characters and 17 tokens on o200k_base and 17 on cl100k_base.
Want to see your own text? Paste it into the token calculator and compare the two tokenizers.
Hindi next to English and the closest languages
| Language | Characters | Tokens o200k_base | Tokens cl100k_base | × English o200k_base | × English cl100k_base |
|---|---|---|---|---|---|
| EnglishEnglish | 902 | 200 | 200 | 1.00× | 1.00× |
| CzechČeština | 874 | 291 | 380 | 1.46× | 1.90× |
| FinnishSuomi | 954 | 295 | 361 | 1.48× | 1.80× |
| Hindiहिन्दी | 910 | 298 | 878 | 1.49× | 4.39× |
| Thaiไทย | 773 | 301 | 710 | 1.50× | 3.55× |
| UkrainianУкраїнська | 967 | 304 | 514 | 1.52× | 2.57× |
See all 30 languages, with a chart and CSV and JSON downloads, on token cost by language.
What it means for your bill
API prices are per token, so the ratio is also a price ratio. If a prompt costs $1.00 in English on o200k_base, the equivalent Hindi prompt costs about $1.49, and about $4.39 on cl100k_base. A document of 1,000 Hindi characters is roughly 327 tokens on o200k_base. These are estimates from one sample: translated text is never exactly the same length as the original, and your text may be formal, technical or full of names, which changes the ratio.
To turn tokens into money, enter your provider's input and output prices per million tokens in the token calculator. For how pricing works, see how LLM API pricing works.
How to use fewer tokens with Hindi text
- Use a model with the newer tokenizer. For Hindi, o200k_base used 66% fewer tokens than cl100k_base in this sample, so a GPT-4o-class model needs fewer tokens than an older one for the same text.
- Count before you send. Paste a typical message into the token calculator to see the real count, then multiply by the number of requests.
- Keep fixed instructions short. A system prompt that is sent with every request is paid for every time. Writing the standing instructions in English is often cheaper per token than in Hindi; test that answers stay as good as you need.
- Cut what is not needed. Remove greetings, repeated context and long examples. The guide to reducing token usage has more.
- Be careful with emoji, rare characters and unusual formatting. They are often split into several byte-level tokens each.
Frequently asked questions
How many tokens is Hindi text?
In our measurement, the same website copy that is 200 tokens in English (902 characters) is 298 tokens in Hindi on o200k_base (910 characters) and 878 tokens on cl100k_base. As a rule of thumb, 1,000 characters of Hindi come to about 327 tokens on o200k_base and about 965 on cl100k_base.
Is Hindi more expensive than English in GPT-4o?
Slightly to noticeably, depending on the text. On o200k_base, the encoding used by GPT-4o and newer OpenAI models, the Hindi copy needs 1.49× the English token count, so at the same price per token the same message costs about 1.49× as much. It was 4.39× on the older cl100k_base.
Which tokenizer is better for Hindi?
o200k_base needed 66% fewer tokens than cl100k_base for Hindi in this sample. Across all 29 non-English languages in the data, the average saving was 34%, so Hindi improved more than average.
Why does Hindi need more tokens than English?
Hindi is written in Devanagari, where a visible syllable can be built from several Unicode characters: a consonant, a vowel sign, and sometimes a joining mark. Each of those characters takes three bytes in UTF-8, so text that looks short on screen contains a lot of bytes for a tokenizer to cover. Indic scripts build a visible syllable out of several Unicode characters, and every one of those characters takes three bytes in UTF-8. Older tokenizers saw little text in these scripts, so they cut words into many small pieces. Newer vocabularies are larger and include more of these scripts, which is why the improvement from the older to the newer encoding is largest here.
Does this apply to Claude, Gemini or Llama?
Not directly. Other providers train their own tokenizers, so counts and ratios differ. These figures describe two OpenAI encodings only. Use your provider's own token counter before relying on a number for billing.
How was this measured?
We counted the tokens of the site's own copy in each language (tagline, two about paragraphs and four how-to steps) with the o200k_base and cl100k_base encodings from gpt-tokenizer 4.0.0, on 2026-10-12, and divided by the English count. It is one sample of marketing-style text, so other kinds of text, such as code or legal prose, will give different ratios.
Other languages and guides
Other languages: Japanese · Korean · Chinese (Simplified) · Chinese (Traditional) · Bengali · Thai · Russian · Vietnamese · Greek · Turkish · Indonesian · German · Spanish · Portuguese · French