Tokens and words are counted by two completely different rules — a token is a chunk of text a model's tokenizer produces, while a word is whatever's separated by whitespace. They correlate closely enough to estimate one from the other, but the ratio between them isn't a fixed constant.
The Basic Conversion
Estimated Words = Token Count × Words-Per-Token Ratio
For ordinary English prose, that ratio measures out to roughly 0.89 words per token — meaning a typical word gets split into a bit more than one token on average, since common word fragments (prefixes, suffixes, whole short words) each tend to map to their own token.
Why the Ratio Isn't Universal
Technical content measured the same way drops to around 0.68 words per token — code, API references, and text dense with numbers or uncommon terminology breaks into more, smaller tokens per word than everyday prose does.
Non-English text often diverges even further. Tokenizers trained primarily on English text tend to represent other languages less efficiently — a passage in a language with different word-formation patterns can require noticeably more tokens per word (or per character) than the equivalent English content, since the tokenizer's vocabulary has fewer dedicated tokens for those patterns.
What This Means in Practice
A single conversion factor is fine for a rough, ballpark estimate — "about 100,000 words" for a 128K-token budget is a reasonable starting assumption for English prose. For anything where the actual number matters — fitting a specific document into a specific context window, for instance — measuring the real text directly removes the guesswork entirely, since the actual tokenizer produces an exact count rather than an estimate built on an average that may not match your content.
Measure your exact token-to-word ratio
Try the Context Window Estimator