How exactly do reasoning models work?
If you've spent any time around large language models, you've probably heard the standard explanation: a transformer is just a very sophisticated autocomplete. It looks at a sequence of words and predicts the most likely next word, then the next, then the next. That description is technically accurate, and yet it leaves a nagging question. If these models are merely guessing the next token, how do the newer "thinking" or "reasoning" models seem to plan, backtrack, check their own work, and decide when to reach for a calculator or a search tool? This post digs into what actually changes when a plain language model becomes a reasoning model.
“Thinking” and “reasoning” models are the same thing
Before going further, it is worth clearing up a naming question that trips a lot of people up. If you have been using a “thinking” model and wondering whether that is different from a “reasoning” model, the short answer is no. They are two names for the same idea. The industry never settled on a single label, so different companies and products describe the same underlying technique with different words. You will see it called a reasoning model, a thinking model, and features with names like extended thinking or thinking mode. Underneath the branding, they all refer to a model that generates an explicit chain of intermediate steps before committing to a final answer.
It also helps to know what these models are contrasted with. A plain or non-thinking model reads your question and starts producing the answer immediately, with no visible deliberation in between. A thinking model, by contrast, first produces a stretch of internal reasoning, sometimes shown to you and sometimes hidden, and only then writes its final response. Many modern models can operate in either mode, spending little effort on easy questions and switching into a longer thinking pass for hard ones. So throughout this post, wherever you read “reasoning model,” you can mentally substitute “thinking model,” because they describe the exact same machinery. The rest of the post explains how that machinery actually works.
The next-token trick is more powerful than it sounds
The first thing worth appreciating is that "predict the next token" is not as limiting as it appears. To predict the next word well across billions of documents, a model is forced to build internal representations of grammar, facts, logical relationships, arithmetic patterns, the structure of code, and even rough models of what a writer is trying to accomplish. Predicting the next word in the sentence "The capital of the country whose flag has a maple leaf is ___" requires the model to have effectively resolved a small chain of reasoning first. So the machinery underneath is not really storing sentences; it is storing compressed, generalizable patterns about how the world tends to be described.
The catch is that a single forward pass through the network does a fixed amount of computation before it must commit to a token. Hard problems, like a multi-step math proof or a tricky debugging task, may need far more computation than that one pass allows. This is the key bottleneck that reasoning models are designed to break.
Thinking out loud is thinking with more compute
The central insight behind reasoning models is deceptively simple: let the model use its own generated text as a scratchpad. When a model writes out intermediate steps before giving a final answer, each of those intermediate tokens becomes part of the context that the next tokens are conditioned on. In effect, the model is feeding its own partial reasoning back into itself, turning a fixed-depth computation into an arbitrarily long chain of computation. This is why "chain of thought" works. The extra tokens are not decoration; they are the medium in which the actual reasoning happens.
Put differently, a reasoning model is still predicting the next token. What has changed is what it predicts next. Instead of jumping straight to an answer, it has learned that the highest-value continuation is often a step of deliberation: restating the problem, listing constraints, trying an approach, noticing a mistake, and correcting course. Each of these is still a next-token prediction, but the sequence as a whole performs something that looks a great deal like reasoning.
How the model learns to reason rather than just ramble
A base model left to its own devices will happily produce plausible-sounding text that goes nowhere. The reason modern thinking models actually reason is that they are trained to. After the initial pretraining on next-token prediction, these models go through additional stages, and the most important one for reasoning is reinforcement learning.
The idea is to give the model problems that have checkable answers, such as math questions, coding challenges, or logic puzzles. The model generates a long chain of thought and a final answer, and then a grader checks whether the answer is correct. Chains that led to correct answers are reinforced; chains that led to wrong answers are discouraged. Crucially, the model is not told which specific reasoning steps to take. It discovers, through millions of attempts, which patterns of thinking tend to produce correct results. This is why reasoning models often develop habits like double-checking arithmetic, considering edge cases, or exploring more than one approach before committing. Nobody hand-coded those behaviors; they survived because they raised the success rate. This style of training is often called reinforcement learning from verifiable rewards, and it is the engine behind the recent jump in reasoning ability.
There is a subtle but important consequence here. Because the reward is attached to the final outcome rather than to any individual step, the model is implicitly optimizing over the whole trajectory of thought. It learns to allocate more thinking to harder problems and less to easy ones, and it learns that a wrong turn is recoverable if it notices and corrects it. This is what gives thinking models their characteristic willingness to say, in effect, "wait, that isn't right, let me reconsider."
How training a base model differs from training a reasoning model
It helps to see training as a sequence of stages, because a reasoning model is not trained from scratch by some separate method. It starts life exactly like a plain language model and then goes through extra steps. The first stage, pretraining, is identical for both. The model is shown an enormous corpus of text and trained on the single objective of predicting the next token. This is where essentially all of the model's raw knowledge and linguistic ability come from, and it produces what is called a base model: a powerful next-token predictor that is not yet shaped into a helpful assistant. A base model will happily continue your text, but it has no particular inclination to answer questions or follow instructions.
To turn that base model into a useful non-thinking assistant, you add a post-training stage. Typically this means supervised fine-tuning on curated examples of good question-and-answer behavior, followed by reinforcement learning from human feedback, where human preferences teach the model to be helpful, honest, and well-mannered. The result is the kind of instruct model most people used for years: it answers immediately, in a single pass, without showing any deliberation. Training here is mostly about alignment and style, shaping how the model responds rather than teaching it to think through problems step by step.
Training a reasoning model adds one more stage, and this is the real point of departure. On top of the aligned model, you run a further round of reinforcement learning, but now the reward is tied to whether the model actually gets the right answer on problems that can be checked automatically, such as math, code, and logic. The critical difference from the earlier human-feedback stage is what is being optimized. Human feedback largely rewards how an answer looks and feels; verifiable rewards care only about whether the final result is correct, which frees the model to discover whatever reasoning process leads there. It is under this pressure that the model learns to generate long chains of thought, to check its own work, to backtrack, and to spend more steps on harder problems. Nobody scripts those behaviors; they emerge because they raise the success rate.
The upshot is that a reasoning model isn't trained in some fundamentally different way; it's the same foundation with an extra stage bolted on. Both models share identical pretraining and a similar alignment phase; the reasoning model simply goes one step further, using outcome-based reinforcement learning to cultivate deliberation. This is also why the same underlying model can often be run with thinking turned on or off. The capacity to reason was trained into it as an extra layer, not baked into a wholly separate architecture.
Why reasoning models still feel fast
It is a fair question: if a thinking model works through dozens or hundreds of intermediate steps, why does it not feel dramatically slower than a model that answers in one shot? The answer comes down to what a step actually is and how fast the underlying hardware runs. Each step in the chain of thought is just one more token, and a modern model on modern accelerators generates tokens extremely quickly, often dozens or even hundreds per second. A few hundred tokens of hidden reasoning might add only a second or two of wall-clock time. The steps feel weighty when you read them as sentences, but to the machine they are the same cheap next-token operation it was already doing to write the answer.
Several things compound to keep it snappy. The reasoning tokens are usually generated at the same speed as normal output, so there is no special slowdown for thinking. The heavy engineering that makes token generation fast, including caching the work done on earlier tokens so it never has to be recomputed, applies just as much to the thinking phase as to the final answer. And models learn to scale their effort to the problem, spending only a handful of steps on something easy and reserving long deliberation for the genuinely hard questions, so most everyday queries never trigger a lengthy think in the first place.
There is also a perception effect worth naming. Interfaces often stream the thinking as it happens or collapse it behind a short “thinking…” indicator, so you are rarely left staring at a frozen screen. Because something is visibly moving, the wait feels productive rather than idle. That said, the tradeoff is real: on the hardest problems a reasoning model really is slower, because more thinking means more tokens and more tokens take more time. The point is simply that the slowdown is proportional to the thinking actually done, and for the large majority of questions that amount is small enough that the extra reliability comes essentially for free.
How a model decides to call a tool
Tool use follows the same underlying logic, which is what makes it feel less mysterious once you see it. From the model's perspective, calling a tool is just another kind of text to generate. During training, the model is shown examples and given rewards in settings where the context includes a description of the available tools, usually a list of function names, what each one does, and the arguments it expects. When the model emits a specially formatted block of text that names a tool and its arguments, the surrounding system intercepts that text, actually runs the tool, and pastes the result back into the context. The model then continues generating with that new information in hand.
So the decision of whether to call a tool, which tool to call, and what arguments to pass is nothing more exotic than a next-token prediction shaped by training. The model has learned, from many rewarded examples, that when a question involves precise arithmetic, generating a call to a calculator tool leads to better outcomes than trying to compute it in its head. It has learned that when a question depends on current events beyond its knowledge, emitting a search query and reading the result beats guessing. The tool descriptions in the context act like a menu, and the model's learned judgment picks from that menu based on the shape of the problem in front of it. When the arguments matter, the same chain-of-thought machinery helps: the model can reason about what to search for or what parameters to pass before it actually writes the call.
This is also why a tool-using model can operate in loops. It calls a tool, reads the result, decides whether the result answered the question, and either calls another tool or writes its final response. Each of those decision points is a fork in the token stream where the model weighs its options based on everything it has seen so far, including the outputs of its own earlier actions. An agent working through a multi-step task is really just a reasoning model whose chain of thought is periodically interrupted by real information from the outside world.
So does it really go beyond predicting the next word?
It's both, really, and that tension is where most of the confusion lives. Mechanically, nothing has been added to the transformer that violates the next-token framing. Every action, every reasoning step, every tool call is produced one token at a time by the same fundamental process. In that narrow sense, no, it never stops predicting the next word.
But the behavior that emerges from that process is a lot more than the sum of its parts. By generating a scratchpad, the model converts a fixed-compute predictor into a variable-compute reasoner. By training against checkable outcomes, it acquires strategies for thinking that were never explicitly programmed. By treating tool calls as just more text, it extends itself with abilities like exact computation and live information that no amount of next-token skill could provide on its own. The prediction objective is the same; what it has learned to predict is a full process of problem-solving rather than a plausible-sounding answer.
Maybe the cleanest way to hold both truths at once is this: predicting the next token is the mechanism, but reasoning is what that mechanism was shaped to produce. A pianist is, mechanically, just pressing keys in sequence. That description is true, and it also tells you almost nothing about the music. Reasoning models are the same story. The next-token engine never goes away; we simply taught it to think out loud, to check its work, and to reach for the right tool at the right moment, and those learned habits are what turn autocomplete into something that can genuinely reason through a problem.
Why some models trip over the car wash question
This is a great example to reason about, because the failure is not really about intelligence. When you ask a strong reasoning model whether to walk or drive to a car wash a hundred meters away, it will almost always give the sensible answer: the whole point of a car wash is that you bring the car, so walking there defeats the purpose. The models that stumble tend to do so for a specific reason. The question superficially resembles thousands of training examples of the form "the destination is a short distance away, so walking is the greener and healthier choice." A model that pattern-matches too eagerly latches onto that familiar template and answers "walk, it is only 100m," completely missing the one detail that inverts the whole problem: the car itself is the thing that needs to be at the destination.
And this is where thinking actually pays off. A model that has learned to slow down and actually restate the problem to itself will write something like "the goal is to get the car cleaned, and a car wash requires the car to be physically present." Once that sentence exists in its context, the next-token prediction almost cannot help but arrive at the right conclusion, because the reasoning has surfaced the decisive constraint. Weaker or non-thinking models skip that step and answer from the surface shape of the sentence. So the car wash trap is less a puzzle about distance and more a test of whether the model reasons about the goal or merely reacts to keywords. It is a clean illustration of why deliberation, not raw knowledge, is what these questions probe.
How thinking models crack complex logic problems
Hard logic problems, the kind with several interacting constraints, are difficult for a plain model because a single forward pass simply cannot hold and manipulate all the moving parts at once. The reasoning model gets around this by externalizing the work. It writes down the constraints one by one, tries a tentative assignment, checks it against each rule, and when something breaks it backs up and tries another branch. What it's really doing is searching through the space of possibilities, but the search tree lives in the generated text rather than in some separate module. Each step of that search is still a next-token prediction, yet the accumulated transcript lets the model carry forward conclusions it has already established instead of re-deriving everything from scratch on every step.
The reason this behavior is reliable rather than accidental goes back to training. Because the model was rewarded for reaching correct answers on checkable logic and math problems, the habits that survived are precisely the ones that help on genuinely hard cases: enumerating cases, testing edge conditions, noticing contradictions, and abandoning a doomed line of attack early. A model that has internalized these habits will often catch its own mistake mid-solution and write something to the effect of "that violates the second condition, so this branch fails," then move on. That self-correction is the single biggest reason thinking models outperform their non-thinking siblings on logic. They are not smarter in any fixed sense; they are allowed to spend more steps and have been trained to spend those steps productively.
Complex math: reasoning, tools, or both?
Math is where the question "is this pure model reasoning or is it calling a tool?" becomes sharpest, and the answer is that it can be either, since the two are good at different things. Left to pure reasoning, a thinking model solves math the same way it solves logic: it works the problem out in words and symbols across many steps, setting up equations, applying known identities, and checking intermediate results. This is the model actually reasoning, not looking up an answer. It is remarkably capable at the conceptual parts, choosing a strategy, recognizing that a problem calls for induction or a substitution, and structuring a proof.
Where pure reasoning gets shaky is precise mechanical calculation. Multiplying large numbers, carrying out long arithmetic, or manipulating an unwieldy expression are exactly the tasks where predicting the next token is a poor substitute for actually computing. A model doing this in its head is essentially estimating what the answer should look like, and small slips accumulate. This is why a model can lay out a flawless solution strategy and still botch the final number. The conceptual reasoning and the arithmetic execution are two different skills, and the transformer is far stronger at the former.
That gap is why tools matter for math, and why the best setups combine the two. When a model has access to a code interpreter or a calculator, the ideal division of labor is for the model to do the reasoning and delegate the computation. It figures out what needs to be calculated, writes a small snippet or a calculator call to get an exact result, reads that result back, and continues its reasoning with a number it can trust. The model is still driving; the tool is just a reliable arithmetic unit bolted on. So when you see a thinking model nail a gnarly numerical problem, it is often a hybrid: the strategy and structure come from the model's own reasoning, while the exact numbers come from a tool it chose to call.
The important nuance is that whether a tool gets used is itself a decision the model makes, using the same learned judgment described earlier. On a simple arithmetic step it may just compute inline; on a heavy calculation it recognizes the risk and reaches for the tool. And crucially, without any tools at all, a strong reasoning model can still solve a great deal of advanced math purely through its own step-by-step deliberation, which is often mistaken for memorization but is really the scratchpad mechanism doing its job. Tools do not replace the reasoning; they remove the one weakness, exact computation, that reasoning alone handles poorly.