Model takes a very long time to load
Symptom
Startup takes minutes; first token is delayed after loading.
Likely causes
- Slow disk or network drive
- mmap on a slow external disk
- Not enough RAM, so pages are swapped
How to fix it
- Store models on a fast local SSD.
- Ensure enough free RAM; avoid --no-mmap on low-memory machines (or use it to force fully loading from a slow disk once).
- Keep the server running instead of restarting per request.
Still stuck? Check the lessons or measure first with the benchmark lab.