The reasoning-model era has a cost problem

TL;DR

• Reasoning models spend tokens thinking before they answer, and those tokens are billed as output.
• The result is a query whose cost varies by an order of magnitude depending on how hard the question is.
• Buyers are responding with routing, budgets and evaluation, not by picking a single model.

For two years the price of a large language model call moved in one direction. Providers cut list prices, competition increased, and the standard advice was to wait a quarter and pay less. Reasoning models have broken that pattern. The same question, asked of a model that thinks first, can cost more than the entire conversation would have cost a year ago.

What changed

A reasoning model does not answer immediately. It generates an intermediate chain of tokens, checks its own work, sometimes backtracks, and only then produces the reply the user sees. Providers bill those intermediate tokens as output, whether or not they are shown. A question that produces a fifty-word answer might generate several thousand words of reasoning on the way.

That is the mechanism behind the cost problem. Output tokens are the expensive kind, and reasoning multiplies them. The per-token price may be similar to a non-reasoning model, but the number of tokens is not.

Why it is worth paying for

The trade is not a bad one. Research on test-time compute has shown that letting a smaller model think longer can match or beat a much larger model on hard problems, at lower total cost. For maths, code, multi-step planning and anything with a verifiable answer, the accuracy gain is real and often decisive.

The question is no longer which model is best. It is how much thinking each request deserves.

The difficulty is that the gain is uneven. Easy questions do not benefit from extended reasoning at all, and they still pay for it if the model is allowed to think. A support chatbot that routes every message to a reasoning model is paying a research-grade price for questions about opening hours.

What buyers are doing about it

Three patterns have emerged among teams running these models in production.

Routing by difficulty. A cheap classifier or a small model decides whether a request needs reasoning. Simple requests go to a fast model; hard ones get the expensive path. This is the single largest lever, and it is why the AI gateway category has grown so quickly.

Thinking budgets. Most providers now expose an effort setting that caps reasoning tokens. Teams tune it per use case, accepting a small accuracy loss for a large cost reduction on the middle of the distribution.

Evaluation before rollout. Because cost is now a function of the question, the only way to forecast a bill is to run a representative sample of real traffic through each candidate configuration and measure both accuracy and spend. Teams that skipped this step have reported monthly invoices several times their estimate.

What to watch

The economics will keep moving. Providers are working on models that decide for themselves how long to think, which would move routing inside the model. Hardware improvements continue to cut the cost of every token. But the structural change is permanent: capability now has a dial, and the dial has a price. Anyone buying model access needs to know where their dial is set.