A language model has no memory between steps: to produce each response it re-reads its entire context — system prompt, your messages, its own thoughts, every file it opened. Press play and watch what a coding agent actually bills you for.
Language models don't read letters or words — they read tokens: frequent chunks like “the ”, “cat”, or “();”. Roughly ¾ of an English word each. Everything a model reads and writes is counted in tokens, and tokens are what you're billed for: typically a few dollars per million in, and about five times more per million out.
A model doesn't “remember” the conversation the way you do. It's autoregressive: to produce anything, the entire context — system prompt, your messages, its own thoughts, every file it opened — is fed through the model again. Nothing is held over; everything is re-sent, every request.
A chatbot re-reads a few thousand tokens. A coding agent makes tool calls that dump whole files into the context — and they never leave. Each new step re-reads everything before it, so total billed input grows roughly with the square of the session. A real six-prompt vibe-coding session can bill ~2 million input tokens for ~47 thousand tokens of actual output.
Providers re-charge already-seen context at ~10% of the price (prompt caching). That helps a lot — toggle it above and watch the cost drop. But caches expire within minutes, the context still fills up, and the re-reading still happens. This is why flat-fee agentic coding plans were quietly subsidised, and why usage caps and per-token pricing arrived.