Tokens, Models & the Cost Curve

I What Are Tokens?

You don't pay per word or per character. You pay per token— a chunk of text that might be a whole word ("cat"), part of a word ("understand" + "ing"), or a single character. Spaces, punctuation, and newlines count too.

Tokens are the unit of cost.If you don't understand them, you can't estimate costs or make informed decisions about prompt design, context length, or model selection. Rule of thumb: 1 token ≈ 0.75 wordsin English (100 words ≈ 130 tokens). Code is more token-dense than prose, and non-English languages use more tokens per word.

Try it yourself:

Notice how code and JSON use more tokens per "word" than prose. A JSON key-value pair like "user_id": "abc123"is 7-8 tokens despite being just a few "words." Structured data eats your token budget for breakfast.

II The Price Tag

Input tokens and output tokens are priced differently. Output tokens cost 3-5x more because generation is computationally harder than comprehension. The meter runs faster on the way out.

Pricing across model tiers in 2026 (per million tokens):

That's a 5x price differencebetween cheapest and most expensive. Choosing the right model tier is one of the highest-leverage decisions you can make. And these decisions add up fast: Anthropic's analysis of millions of API interactions found that roughly 50% of all agentic tool calls go to software engineering tasks alone, with every other domain in the single digits. That concentration means a huge share of real-world LLM spend is driven by one use case — and optimizing model selection there has outsized impact.

III Model Tiers

Every provider offers small, medium, and large tiers. The naming varies — Haiku/Sonnet/Opus, or mini/standard/frontier tiers — but the tradeoff is universal:

• Small models— fast and cheap, but struggle with nuance and complex reasoning. Great for classification, extraction, simple Q&A.
• Medium models — the workhorses. Good enough for most product features. This is where most production traffic should go.
• Large models— most capable, slowest, most expensive. Reserve for tasks where quality genuinely can't be achieved with a smaller model.

Compare the same task across model tiers:

Model selection is task-dependent. A small model classifies a support ticket just as well as a large one — at 1/5th the cost. But ask it to analyze churn, and you'll get the analytical equivalent of "have you tried turning it off and on again." Match the model to the task, not to your ego.

IV Cost at Scale

Here's where AI product plans fall apart. A prototype at $0.02 per request feels free. Multiply by real usage and it stops being free very fast. OpenAI co-founder Greg Brockman has argued that compute scarcity — not model capability — is the real scaling constraint in the agentic era. The models are good enough; the question is whether you can afford to run them.

The equation: cost = requests × tokens per request × price per token. Each variable varies by orders of magnitude:

• Requestsscale with users. 1,000 DAUs × 5 requests = 5,000/day. 100,000 DAUs = 500,000/day.
• Tokens per request depend on prompt design. A 200-token prompt with a 50-token response is fundamentally different from 2,000 in and 500 out.
• Price per token depends on model choice — a 5x range.

Two builders building similar features can end up with costs that differ by 100x or more, based solely on design decisions:

V What's Next

You now understand LLM economics: tokens as the unit of cost, input vs. output pricing, model tiers, and how costs compound at scale. But cost is only half the equation.

In Part 2: latency. Why LLMs feel slow, how streaming transforms the experience, and the optimization levers — caching, prompt engineering, model routing — that let you ship AI features that are both fast and affordable.