🔒 Private by default: your text is tokenised in your browser and never sent to vibecoderjournal.com. The only exception is the optional exact Claude/Gemini count below, which is off by default.
1. Paste your text
2. Choose models to compare
3. Token counts
Token visualiser
See exactly where one model splits your text. Only available for models with exact in-browser tokenizers. Shows the first 300 tokens; hover a token for its ID.
4. API cost calculator
Presets are examples — adjust the numbers to your workload.
5. LLM price comparison
All prices USD per 1M tokens, checked October 10, 2026. “OpenRouter” rows are OpenRouter's hosted prices, not first-party prices. Changelog
6. Chat template preview (tokenizer.apply_chat_template)
Renders the exact prompt string that tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) produces for an open model, using its official chat template loaded in your browser. Special tokens are highlighted.
Same call in Python (verified against the Transformers docs):
Sources & changelog
- 2026-10-10: initial price check — 32 models across 10 providers (OpenAI, Anthropic, Google, DeepSeek, xAI, Mistral via official pricing pages; Qwen, Llama, GLM, Kimi via OpenRouter).
Methodology: exact counts run the provider's own tokenizer in your browser (js-tiktoken for OpenAI; official Hugging Face tokenizers via Transformers.js for DeepSeek, Qwen, Mistral, GLM). Estimates use ≈4 characters per token and are labelled Estimate: measured against exact tokenizers on 2026-10-10, they were usually within about 25% for English prose (median error 21%), with code and JSON varying more. We will re-measure against the official Anthropic/Google count APIs once exact API counting is enabled. Prices are re-checked monthly; entries older than 30 days are flagged.
What is a token?
A token is the chunk of text an AI model actually reads — not a word, not a character, but a piece of a word produced by the model's tokenizer. "Tokenization" turns "unbelievable" into something like "un" + "believ" + "able", and each piece becomes one token.
Why it matters: every LLM API bills per token, and every model has a context window measured in tokens. If you don't know your token count, you can't predict your bill or know whether your prompt fits.
The rough rule you'll see everywhere — about 4 characters per token for English — is real but loose. We measured it against exact tokenizers on 20 prose, code, and JSON samples: estimates were usually within about 25% for English prose (median error 21%), with code and JSON varying more. Good enough for a quick guess; not good enough for a budget. That's why this tool counts exactly wherever the provider's tokenizer can run in your browser, and clearly labels everything else as an estimate.
How to count Claude tokens
There is no official client-side tokenizer for Claude. Anthropic publishes a count-tokens API endpoint (POST api.anthropic.com/v1/messages/count_tokens, returns input_tokens) but no downloadable tokenizer for current Claude models, so your browser cannot count Claude tokens exactly on its own.
This tool gives you two honest options:
- Estimate (default, private): ≈4 characters per token, labelled Estimate. Nothing leaves your browser.
2. Exact via API (opt-in): enable the toggle and your text is sent to Anthropic's count-tokens endpoint for counting only. This needs the site owner to deploy the server proxy and add an ANTHROPIC_API_KEY as an environment secret.
Until the API path is enabled, every Claude count on this page is an estimate — and it says so on the card.
Gemini tokenizer
Google publishes no client-side Gemini tokenizer either. What Google does document is the rule of thumb this tool uses: about 4 characters per token. For exact counts, Google offers a free countTokens endpoint (POST generativelanguage.googleapis.com/v1beta/models/{model}:countTokens).
Same deal as Claude: estimates by default (private, in-browser), exact counts only if you opt into the API toggle once the server side is configured with a GEMINI_API_KEY.
OpenAI and DeepSeek tokenizers
These count exactly, in your browser, right now. OpenAI's tokenizers are open source: this tool runs o200k_base (current GPT models) and cl100k_base via js-tiktoken — the same encodings OpenAI's own tooling uses. DeepSeek's official V4 tokenizer is public on Hugging Face, and the V4-Pro and V4-Flash repos ship byte-identical tokenizer files, so both models count exactly.
Also exact in-browser: Qwen3.8 (official Qwen3.8-27B tokenizer), Mistral Large 3 (official Tekken tokenizer from the public Large 3 repo), and GLM-4.7 (official zai-org tokenizer). Each is verified byte-for-byte against Python reference counts — 76 parity checks pass, including exact token-ID sequences.
Estimate-only: Claude, Gemini, Grok (xAI has never published a tokenizer), Llama (Meta's tokenizers need a login and licence acceptance, so they can't load in-browser), Kimi (no browser-loadable fast tokenizer published), and Mistral Large 4 (weights and tokenizer not public yet).
LLM price comparison
Prices below are USD per 1M tokens, checked October 10, 2026, from official pricing pages (OpenAI, Anthropic, Google, DeepSeek, xAI, Mistral) or OpenRouter's live models endpoint for Qwen, Llama, GLM, and Kimi — OpenRouter rows are labelled as OpenRouter pricing, not first-party pricing. Every row carries its source link and check date in the table.
A few things the table makes obvious:
- The cheapest capable models are not the famous ones. DeepSeek V4.1 Flash and Qwen3.8 Flash sit at a fraction of flagship prices. - Cache reads are the biggest lever. Claude and GPT cache reads cost roughly 10% of fresh input; DeepSeek's off-peak cache reads are a fraction of a cent per 1M. - Long-context tiers change the math. GPT-6 models double input rates past 272K tokens; Gemini 3.1 Pro doubles past 200K; Grok applies long-context rates at or above 200K prompt tokens. The calculator flags this automatically.- Batch APIs cut ~50% where offered (OpenAI, Anthropic, Google).
Prices change monthly — sometimes mid-month (Gemini 3.8 Flash is on promotional pricing through December 31, 2026). Mistral Large 4 is in public preview: the table shows list rates ($1.36/$4.18), but a launch sale ($0.68/$2.09) with no published end date may apply — check Mistral's pricing page before budgeting. We re-check on the 10th of each month and flag any entry older than 30 days.
API cost formula and worked examples
The formula the calculator uses:
``
per-request cost = (fresh input tokens ÷ 1M × input price)
+ (cached input tokens ÷ 1M × cache-read price)
+ (output tokens ÷ 1M × output price)
monthly cost = per-request cost × requests/day × 30
`
Long-context tiers swap in the tier's rates when input tokens cross the provider's threshold; batch mode multiplies by the provider's batch discount (0.5 for OpenAI/Anthropic/Google).
Example 1 — customer-support chatbot (Claude Sonnet 5.5): 1,000 input + 300 output tokens, 100 requests/day. That's (1,000/1M × $2) + (300/1M × $10) = $0.005/request → $15/month.
Example 2 — RAG over long docs (Gemini 3.1 Pro): 8,000 input + 500 output, 200 requests/day, 30% cached. Fresh: 5,600/1M × $2 = $0.0112; cached: 2,400/1M × $0.20 = $0.00048; output: 500/1M × $12 = $0.006 → $0.0177/request → ~$106/month.
Example 3 — coding agent (GPT-6.1 Sol): 100K input + 5K output, 50 requests/day, 50% cached. Stays under the 272K long-context threshold: (50K/1M × $2) + (50K/1M × $0.10) + (5K/1M × $10) = $0.155/request → ~$233/month. Push that input past 272K and the long-context tier doubles the input rate.
tokenizer.apply_chat_template
tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) is the Hugging Face call that turns a chat conversation into the exact prompt string a model was trained on — inserting the right special tokens (<|im_start|>, <|assistant|>, etc.) in the right places. Get it wrong and the model sees a malformed prompt.
The template preview above runs the model's official chat template in your browser: Qwen3.8 and Mistral Large 3 and GLM-4.7 ship theirs in their public repos (Mistral's and GLM's live in a separate
chat_template.jinja, which this tool fetches and passes explicitly — transformers.js doesn't load it on its own). Special tokens are highlighted, both add_generation_prompt values are supported, and the matching Python snippet is verified against the Transformers docs. DeepSeek V4 ships no chat template in its public repo, so it's excluded from the preview rather than guessed.
Reducing token costs
- Cache what repeats. System prompts and few-shot examples are the classic win — cache reads cost ~10% of fresh input on Claude/GPT.
- Use batch APIs for offline work. Evals, classification, and bulk summarisation get ~50% off on OpenAI, Anthropic, and Google.
- Pick the smallest model that does the job. A flash/lite model at 1/10th the price beats a flagship you don't need.
- Watch the long-context cliff. If your inputs hover near 200K–272K tokens, trimming a little can halve the input rate.
5. Count before you send. Paste the prompt here first — the context-window bar shows whether it even fits.
Frequently asked questions
How accurate is this AI token calculator?
Counts labelled Exact run the provider's own tokenizer in your browser and match reference counts byte-for-byte (76/76 parity checks). Counts labelled Estimate use ≈4 characters per token — measured usually within about 25% for English prose.
Does my text leave my browser?
No — unless you enable the optional exact Claude/Gemini API toggle. Counting, file reading, and chat-template rendering are 100% client-side.
Why is Claude an estimate?
Anthropic publishes no client-side tokenizer for Claude, only a count-tokens API. Exact Claude counts need the opt-in API path.
Which is cheaper, GPT or Claude?
It depends on the workload — run your numbers in the calculator. As a rule of thumb from the October 2026 prices: DeepSeek and Qwen flash models are cheapest per token, Gemini flash models sit in the middle, and flagship GPT/Claude models cost the most.
What is the cheapest LLM API?
From the checked prices: DeepSeek V4.1 Flash ($0.30 input / $1.20 output per 1M) and Qwen3.8 Flash ($0.15 / $0.47 via OpenRouter) are among the cheapest, with aggressive cache-read discounts on top.
How do I use tokenizer.apply_chat_template?
Pass a list of
{role, content} messages with tokenize=False and add_generation_prompt=True`. The preview above shows the exact rendered string for Qwen3.8, Mistral Large 3, and GLM-4.7, with a copyable Python snippet.
Do prices include batch discounts?
The table shows list prices; the calculator applies the 50% batch discount where the provider offers it when you tick "Batch API".
How often are prices updated? Checked on the 10th of every month from official pricing pages and OpenRouter's live endpoint. Rows older than 30 days are flagged on the page.