Foundations
Tokens, context windows, prefill vs decode and sampling: the mental model behind every local LLM.
0 of 4 complete
- ▶ Up next10 min+50 XPWhat is local LLM inference?What actually happens when you run a model on your own machine, the four pieces you need, what local does well and…
- 11 min+50 XPTokens and context windowsHow text becomes tokens, why non-English text and code cost more, how the context window works as a budget shared by…
- 12 min+50 XPHow generation works: prefill, decode and the KV cacheThe two phases of inference, why prefill is limited by compute and decode by memory bandwidth, how to estimate both…
- 12 min+50 XPSampling parameters that matterHow a model's scores become a chosen token: temperature, top-k, top-p, min-p, repetition penalties and seeds with…