Inference visualizer
Animated simulations of prefill and decode, speculative decoding and batching.
Phase 1: Prefill (all prompt tokens in parallel, compute-bound)t = 0.00s
Tokens
KV cache (Llama 3.1 8B, f16): 0.0 MiB
Time to first token
0.04s
Decode time
1.00s
Total
1.04s
Tip: raise the prompt length and watch TTFT grow; lower the decode speed to feel what a slow memory system does.
Learn more