What Is a Token in AI? Context Windows Explained
If you’ve used AI seriously, you’ve bumped into ‘tokens’ and ‘context windows’. They’re not just trivia — they explain why AI has length limits, why long chats ‘forget’, and why APIs charge the way they do.
This guide makes both concepts click, with practical implications for cost and quality.
- A token is a chunk of text (word or word-piece) a model processes.
- ≈1,000 tokens ≈ 750 English words.
- The context window is the max tokens a model handles at once (prompt + reply).
- APIs charge per token — so tokens = money.
- Long chats ‘forget’ when earlier text scrolls out of the context window.
Want to check your understanding as you read? You can take our free AI Quiz any time — it covers this topic across Beginner, Intermediate and Advanced levels.
What is a token?
Models don’t read letters or whole sentences — they read tokens, chunks of text that are often a word or part of a word. ‘Cat’ might be one token; ‘unbelievable’ might be several. As a rough guide, 1,000 tokens is about 750 English words.
Everything a model reads and writes is counted in tokens, including your prompt and its response. That single fact explains a lot of AI’s behavior and pricing.
What is a context window?
The context window is the maximum number of tokens a model can consider at once — your prompt plus everything it generates in reply. Think of it as the model’s short-term memory or ‘desk space’.
A small window means the model can only ‘see’ a limited amount at a time; a large window (millions of tokens in some 2026 models) lets it read entire books, long transcripts or whole codebases in a single pass. Bigger isn’t always necessary, but it unlocks tasks that were impossible before.
Why long chats sometimes ‘forget’
Here’s a common frustration explained: in a very long conversation, once the total text exceeds the context window, the oldest parts fall out of the model’s view. It doesn’t ‘remember’ them anymore — they’ve scrolled off the desk.
That’s why restating key details, or starting a fresh chat for a new topic, often improves results. It’s not the model being forgetful; it’s a hard limit on how much it can hold at once.
🧠 Test what you just learned
Put your knowledge to the test with our free 250-question AI Quiz — 14 categories, instant explanations, a grade and a shareable certificate.
Why tokens affect your cost
Most AI APIs charge per token, for both input (your prompt) and output (the response). Longer prompts and longer answers cost more. A single heavy task generating millions of tokens can cost real money.
This is why cost-savvy teams trim unnecessary context, avoid dumping huge documents when a summary would do, and route simple, high-volume tasks to cheaper models. Understanding tokens is the first step to controlling an AI bill.
Practical tips
A few habits pay off: keep prompts focused (don’t pad them), summarize long inputs when you can, and for repeated tasks, measure roughly how many tokens each run consumes. If you use agents that run autonomously, cap their token usage to avoid surprises.
In short: tokens are the unit of both capability and cost. Manage them, and you manage both quality and budget.
Related reading
Frequently asked questions
How many words is a token?
Roughly three-quarters of a word on average in English, though it varies by text and language.
Why do longer chats lose track of earlier details?
Once the conversation exceeds the context window, older content falls out of the model’s view.
Do bigger context windows cost more?
Processing more tokens generally costs more and can be slower, so use only the context you need.
Is a bigger context window always better?
It’s better for long-document tasks, but for short tasks it makes no difference and may cost more.
Where can I test my knowledge?
The AI Vocabulary category in our free AI Quiz covers tokens, context windows and pricing concepts.
🧠 Test what you just learned
Put your knowledge to the test with our free 250-question AI Quiz — 14 categories, instant explanations, a grade and a shareable certificate.
Keep going: take the free AI Quiz and see if you rank as an AI Master.
