Set up and run the experiments

#What you need

Python 3.9 or newer. Nothing else: no GPU, no libraries. The experiments are pure Python simulations and small trainers (setup help).
A folder for your files and a terminal.

#How the experiments are organised

Each lesson gives a script. They share a few ideas:

Idea	What it does
A task generator with known answers	Makes problems of varying difficulty with an automatic check
A verifier	Marks an answer correct or incorrect
A strategy	Chain of thought, majority voting, best-of-N, or a trained policy
A metric	Accuracy, pass@k, and cost in tokens or samples

#Run things reproducibly

Set a seed (the scripts do) so a run can be repeated exactly.
Change one thing at a time.
Repeat with several seeds (for example 0 to 4) and report the average and the spread, not a single run.
Keep a results file with the settings next to the numbers.

Noise is the main way beginners fool themselves in this area, so we say it twice: a difference smaller than the run-to-run spread is not a result.

#A first check

Save this as check_reasoning.py and run it:

python

import random
random.seed(0)

def make_problem():
    a, b = random.randint(10, 99), random.randint(10, 99)
    return (a, b, a + b)                       # a question and its known answer

def verifier(problem, answer):
    return answer == problem[2]                # automatic check

def noisy_solver(problem, p_correct=0.6):      # a pretend model that is right 60% of the time
    return problem[2] if random.random() < p_correct else problem[2] + random.choice([-10, -1, 1, 10])

n = 2000
ok = sum(verifier(p, noisy_solver(p)) for p in (make_problem() for _ in range(n)))
print("single attempt accuracy:", ok / n)

You should see an accuracy near 0.6. That is the pattern of everything that follows: a task, a verifier and a stochastic solver whose accuracy we then try to raise.

#Reading results

What you see	What to ask
Accuracy rises with more samples	Is the gain worth the extra cost per answer?
A method helps on easy problems and not on hard	Which problems does it fail on, and why?
Training reward rises but accuracy on new problems does not	Is it hacking the reward or overfitting?

#Exercise

Change p_correct to 0.3 and to 0.9 and run each. Predict the printed accuracy first. Then explain why a method that picks the majority answer over many samples would help the 0.6 solver more than the 0.3 one.

Answer: the accuracies come out near 0.3 and 0.9. Majority voting amplifies a solver whose correct answer is the most common one; at 0.6 the correct answer usually wins the vote, while at 0.3 wrong answers can outnumber it. The inference lessons show exactly when this works.

Next: what reasoning means, and how a verifier makes it measurable.