Set up, and a short C primer for the one benchmark

The engine is Python, so you need Python 3.9 or newer (set-up notes if you have not done this). One benchmark is in C, so you also need a C compiler, and a little C to read it.

#1. Tools

Tool	Check	If missing
Python 3.9+	`python3 --version`	Install from python.org
A C compiler	`cc --version` (or `gcc`, `clang`)	macOS: `xcode-select --install`. Linux: install `gcc`. Windows: use WSL, or MinGW
A text editor and a terminal

#2. A first C program

Save as hello_mem.c:

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <time.h>

int main(void) {
    int n = 1 << 20;                         /* 1,048,576 floats */
    float *w = malloc(n * sizeof(float));    /* allocate memory by hand */
    for (int i = 0; i < n; i++) w[i] = (float)(i % 7) * 0.5f;

    clock_t t0 = clock();
    double sum = 0.0;
    for (int rep = 0; rep < 50; rep++)
        for (int i = 0; i < n; i++) sum += w[i];   /* stream through memory */
    double secs = (double)(clock() - t0) / CLOCKS_PER_SEC;

    double gb = 50.0 * n * sizeof(float) / 1e9;
    printf("sum=%.1f  read %.2f GB in %.3f s\n", sum, gb, secs);
    free(w);                                  /* give it back */
    return 0;
}

Compile and run:

bash

cc -O2 -o hello_mem hello_mem.c
./hello_mem

On the machine used to write this it printed sum=78643050.0 read 0.21 GB in 0.043 s. Yours will differ in the time. The sum should match.

#3. The C you need

Idea	Example	Python equivalent
Types are declared	`int n = 5; float x = 0.5f; double s = 0.0;`	`n = 5`
Arrays are blocks of memory	`float w = malloc(n sizeof(float));`	`w = [0.0] * n`
Indexing	`w[i]`	`w[i]`
Loops	`for (int i = 0; i < n; i++) { ... }`	`for i in range(n):`
Memory is manual	`malloc(...)` then `free(w)`	automatic
Pointers	`float *w` means "address of floats"	a reference to a list
Functions need types	`float dot(float a, float b, int n)`	`def dot(a, b, n)`
Integer division truncates	`7 / 2` is 3; `7 / 2.0` is 3.5	`7 // 2`
Timing	`clock()` and `CLOCKS_PER_SEC`	`time.time()`
Printing	`printf("%.2f %d\n", x, n)`	`print(f"{x:.2f} {n}")`

Two warnings that matter. Reading past the end of an array compiles and may "work" until it crashes or gives wrong results, so always loop to n. Forgetting to free leaks memory, which is harmless in a short benchmark and a bug in a server.

#4. Why the benchmark exists

Decode reads every weight once per token, so its speed is set by how fast memory can deliver bytes (roofline). In Python the interpreter overhead hides that effect. In C a simple loop shows it: the program's speed is the speed of memory. The Engine Fast lessons use this to show why INT8 weights speed up a matrix-vector product (fewer bytes to read), and what threads and SIMD add.

#5. If it does not compile

Message	Cause	Fix
`cc: command not found`	No compiler	Install one (see the table)
`error: expected ';'`	A missing semicolon or brace	Look at the line above the one reported
`undefined reference to ...`	A missing library or a typo in a function name	Check spelling and includes
Very slow run	Compiled without `-O2`	Add `-O2`

#Exercise

Change rep < 50 to rep < 100 and predict the new time and the new "GB read". Run it.
Change float to double everywhere the array is declared and used (and sizeof(float) to sizeof(double)). Predict what happens to the GB read, and to the time.

Answers: doubling the repetitions doubles the GB read and roughly doubles the time. Doubles are twice as big as floats, so the program reads twice the bytes for the same number of elements, and the time rises accordingly because the loop is limited by memory traffic.

Next: start the course proper with the forward pass end to end.