Glossary
What Is a Context Window in AI?
A context window is the maximum amount of text an AI model can read and hold in memory at one time, measured in tokens (small chunks of words). Everything inside it (your question, the chat history, and the model's own reply) shares that one fixed limit, and anything past the limit gets dropped.
Think of the context window as the AI's short-term memory. Before the model writes a single word, it reads everything you've given it plus the conversation so far. The size of that reading space is the context window. Models measure it in tokens, where a token is roughly three quarters of a word. A 128,000-token window can hold somewhere around 90,000 to 100,000 words at once.
Here's a concrete example. Say a customer is chatting with the AI agent on your site. They ask about your return policy, then your hours, then shipping to Canada. Each message and each answer stacks up inside the window. As long as the whole conversation fits, the AI remembers what was said earlier and can answer follow-ups like "and how much does that cost?" without you repeating yourself. If the chat runs very long and spills past the limit, the oldest messages fall out of memory, and the AI can lose track of what you asked at the start.
The window also has to fit the knowledge you feed the agent. When a chatbot answers using your FAQ page, product list, or help docs, those documents take up space in the window too. A bigger window means the AI can keep more of your business details in view while it replies, which usually leads to more accurate answers.
For a website chat or voice agent, the practical lesson is simple. Short, focused conversations stay well inside the limit and feel sharp. Marathon sessions can drift. Most modern systems handle this for you by pulling in only the most relevant pieces of your knowledge base for each question, so the window doesn't fill up with text the customer doesn't need.
Context window is not the same as long-term memory. Once a chat ends, the window clears. If you want an agent to remember a customer across visits, that calls for a separate memory or database feature, not a bigger window.
Frequently asked questions
How big is a typical context window?+
It varies a lot by model. Older models held a few thousand tokens, while many current ones handle 128,000 tokens or more, and some reach a million. A 128,000-token window fits roughly 90,000 to 100,000 words, which is enough for a long conversation plus a chunk of your business documents.
What happens when a chat goes past the context window?+
The oldest text gets pushed out to make room for new messages. The AI can no longer see those early parts, so it may forget details you mentioned at the start. Most chatbot tools manage this automatically by summarizing or trimming older messages so answers stay on track.
Does a bigger context window make my chatbot smarter?+
Not by itself. A larger window lets the AI consider more text at once, which can help with accuracy on long chats or big documents. But answer quality still depends on the model's reasoning and the quality of the knowledge you give it. Feeding it clean, relevant information matters more than raw window size.