What reasoning means, and how a verifier makes it measurable

"Reasoning" is used loosely. For engineering purposes we need something we can measure and improve.

#A working definition

A language model reasons on a task when it reaches a correct answer through a sequence of intermediate steps that it generates, where the answer is not simply retrieved from memory. Typical signs: multi-step arithmetic and algebra, logic puzzles, planning, debugging code, combining facts from a long document. The intermediate tokens (the "chain of thought" or thinking tokens) act as working memory: a transformer has a fixed amount of computation per token, so allowing more tokens allows more computation.

Two uncomfortable truths to keep in mind throughout this course:

1. Fluent steps are not necessarily faithful steps. A model's written reasoning may not reflect the computation that produced its answer.
2. Benchmarks can be contaminated or gamed. Honest measurement is half the work (benchmark contamination).

#Checkable tasks and verifiers

Progress needs measurement. The easiest setting is tasks with a verifiable answer:

A verifier is a function verify(problem, answer) -> bool (or a score). Write one:

import re

def extract_final(text):
    """Take the last number after 'Answer:' (a common convention for checkable output)."""
    m = re.findall(r"Answer:\s*(-?\d+(?:\.\d+)?)", text)
    return float(m[-1]) if m else None

def verify(reference, model_output):
    got = extract_final(model_output)
    return got is not None and abs(got - reference) < 1e-9

print(verify(42, "Let x=6, y=7. 6*7=42.\nAnswer: 42"))    # True
print(verify(42, "I think it's about 40"))                  # False (no parseable answer)
print(verify(42, "Answer: 41"))                             # False

Design choices already matter: strict answer-format parsing counts format failures as errors (is that what you want?), and a lenient parser may accept lucky matches. Document the rule you use.

Outcome versus process verification. Outcome verifiers check only the final answer: cheap, objective, but they reward right answers reached by flawed steps. Process verifiers grade each step: richer signal, but they require step-level labels or a learned judge, and are noisier (LLM-as-judge).

#The three levers

1. Inference-time compute   → let the model think longer / sample more / search          (reason-inference track)
2. Training for reasoning   → reinforce chains of thought that reach verified answers    (reason-training track)
3. Tools                    → calculators, code interpreters, retrieval, search          (reason-training track)

All three trade cost for accuracy. A good reasoning engineer reports accuracy and tokens (or dollars) per problem, never accuracy alone. An answer that is 3 points better at 40× the cost is rarely a win.

#What this course builds

• How to evaluate reasoning honestly: pass@k, confidence intervals, contamination, cost curves (next lesson).
• Inference-time methods with runnable simulations: self-consistency voting, best-of-n with verifiers, search and refinement (inference track).
• Training: policy gradients and a GRPO-style method you can run on a toy problem, distillation, tool-augmented reasoning, and a final lab that reports cost versus accuracy (training track).

Prerequisites: you should be comfortable with how a transformer generates text (build an LLM). The experiments here use small simulations and plain Python so that every mechanism is visible; applying them to real models needs an inference engine (inference engineering) and, for training, GPUs.

#Exercises

1. Write a verifier for "sort the list" tasks that accepts any correct permutation format ([1, 2, 3] or 1 2 3).
2. Give one example each of an outcome-verifiable and a process-verifiable task.
3. Why might a lenient answer parser inflate benchmark scores?

Answers: (2) Outcome: "What is 17 × 23?" (check 391). Process: a proof where each step must follow from the previous (checkable with a proof assistant or a step grader). (3) It credits answers that a strict reader would reject (extra text, units, hedged guesses), so reported accuracy overstates what the model reliably delivers.

#Next

Measure reasoning without fooling yourself: verifiers and honest evaluation.