Set up: Python, a project folder and a first script that runs

The first parts of the course need almost nothing: Python and a text editor. This lesson gets you to a working setup and a first script that runs.

#What you need

The plain-Python listings use only the standard library (math, random, collections, time), so there is nothing to install.

#1. Check Python

Open a terminal:

python3 --version

You want 3.9 or newer. If the command is missing or older, install Python from python.org (or with your system's package manager) and open a new terminal. On Windows the command is often python or py instead of python3; use whichever works, and read python3 in the listings as that command.

#2. Make a project folder

mkdir llm-from-scratch
cd llm-from-scratch

Put every listing from the course in this folder, in a file named in the lesson (for example bpe.py, attention.py, microgpt.py). Keeping them together means they can import each other when a lesson says so.

Optionally, create a virtual environment (you can skip this until the PyTorch lesson):

python3 -m venv .venv
source .venv/bin/activate        # on Windows: .venv\Scripts\activate

#3. Run a first script

Save this as check.py:

import sys, time, math, random

print("Python", sys.version.split()[0])
assert sys.version_info >= (3, 9), "Please use Python 3.9 or newer"

random.seed(0)
t = time.time()
s = 0.0
for i in range(1_000_000):
    s += math.sqrt(i) * random.random()
dt = time.time() - t

print(f"1,000,000 loop steps took {dt:.2f} s")
print("Looks fine." if dt < 5 else "Slow machine: the labs will take a while, but they will still work.")

Run it:

python3 check.py

You should see your Python version, a time (typically well under a second on a modern laptop, though it varies a lot between machines) and Looks fine.

The point of the timing: pure Python is slow on purpose, because nothing is hidden behind fast native libraries. The lab that trains the tiny GPT runs on one CPU core and took about 105 seconds on the author's laptop. If your machine is much slower, give it time.

#How to run a listing

1. Copy the code into a file with the name given in the lesson.
2. Run python3 filename.py from the project folder.
3. Compare your output with the lesson's. Where a number depends on the random seed or on the Python version, the lesson says so.
4. Change one thing, predict the result, run again.

#Reading error messages

Python errors read from the bottom up: the last line says what went wrong, and the lines above show where.

Two habits save hours: print values while debugging (shapes, lengths, the first few numbers), and change one thing at a time.

#Optional: PyTorch later

You will only need it for the last lesson of the course. When you get there:

pip install torch

The CPU version is enough. The lesson explains the mapping from your code to the library, so it makes sense even if you do not run it.

#Working with the code

• Keep your files. Later lessons reuse earlier ones.
• Use version control if you like (git init), and commit after each lesson that works.
• Seeds: the listings call random.seed(...). Keep them while learning, so your numbers match the text; remove them later to see the natural variation.
• Do not paste into a notebook cell unless you know how to rerun it cleanly; a notebook with stale state confuses beginners. A plain .py file is the reliable way.

#Exercise

1. Run check.py. Write down your Python version and the loop time.
2. Change the loop to 2,000,000 steps. Predict the new time before you run it (about twice as long). Was your prediction right?
3. Remove the line random.seed(0) and run it three times. Which printed numbers change and which stay the same?

Answers: the time roughly doubles because the work doubles. Without the seed the time and the sum s are different on each run, while the Python version stays the same; the seed fixes the sequence of random numbers, which is why the course always sets one.

Next: the maths and Python you need.