Token
Definition
A token is the smallest unit into which a language model breaks down text before processing it. Tokens usually correspond to word fragments, individual words or punctuation marks — as a rule of thumb, 1,000 tokens correspond to roughly 750 English words.
Meaning in a Business Context
Tokens are the unit of computation for language models and at the same time the basis for billing with most providers. Inbound and outbound tokens are counted separately, with output tokens usually being more expensive. In practice this means: anyone building a chatbot with long system prompts and a large knowledge context should factor in token consumption and cost from the outset. The model's context window — the maximum number of tokens that can be processed at the same time — is also a hard limit: a model with a 128,000-token context can read a 200-page PDF in one go, an older model with 4,000 tokens only a few pages.