Start Here: The Plan
What evaluation is for, the statistics you need and your first harness.
0 of 3 complete
- ▶ Up next10 min+50 XPWhat evaluation is for, and the planWhy evaluation is the foundation of every LLM change, the loop it supports, the route from golden datasets to…
- 20 min+50 XPThe statistics you need: averages, spread and confidence intervalsWhy small test sets mislead, how to put an interval around a pass rate with a few lines of Python (bootstrap and a…
- 15 min+50 XPYour first evaluation harness: cases, a scorer and a reportA complete harness in about twenty lines of Python: a case list, a stand-in model, an exact-match scorer, repeated…