Glossary
What Are Tokens in AI?
Tokens are the small pieces of text that an AI model reads and writes, usually a word or part of a word. AI tools count tokens to measure how much text they process, and most pricing and speed limits are based on token counts.
A token is a chunk of text. It can be a whole word, part of a word, a single character, or a piece of punctuation. AI language models don't read sentences the way people do. They break text into tokens first, then process those tokens one at a time.
A rough rule helps here. In English, one token is about four characters, and 100 tokens come out to roughly 75 words. So the sentence "My store is open on weekends" is about 7 tokens. Longer or unusual words can split into two or three tokens each.
Tokens matter because almost every AI service prices and limits its work by token count. You get charged for the tokens going in (the question plus any background info you send) and the tokens coming out (the model's reply). A model also has a maximum number of tokens it can handle at once, called the context window. Send more than that and the oldest part gets cut off.
Here's a concrete example. Say a customer asks your website chatbot, "Do you ship to Canada?" That question is a handful of tokens. The bot also quietly reads your shipping policy to answer well, maybe a few hundred more tokens. Its reply adds another 30 or 40. Every one of those counts toward what you pay and how fast the answer comes back.
For a chat widget or a voice agent on your site, fewer tokens usually means lower cost and quicker replies. Keeping your help content tight and focused, instead of dumping huge pages into the model, is one simple way to keep token use down without hurting answer quality.
Related terms
Frequently asked questions
How many words is a token?+
One token is roughly three-quarters of a word in English, so 100 tokens land around 75 words. Short common words are often one token each, while longer or rare words split into two or more. Numbers, punctuation, and spaces also count.
Why am I charged by tokens instead of by message?+
AI providers bill by tokens because that reflects the actual work the model does, not the number of messages. A short question and a long detailed answer cost very different amounts even though both are one message. Counting tokens keeps pricing fair to the amount of text processed.
Do tokens include both my question and the AI's answer?+
Yes. You pay for input tokens, which include your question plus any background content the system sends along, and output tokens, which are the model's reply. Both add up, so longer prompts and longer answers each raise the total.