Structured output: JSON schemas and grammars
How constrained decoding guarantees valid JSON from small models, how to use JSON Schema and GBNF in llama.cpp, Ollama and vLLM, and the design rules that keep the output correct as well as parseable.
Asking a small model politely for JSON works perhaps nine times in ten. The tenth time you get a trailing comma, a markdown fence, or a friendly sentence before the brace, and your parser throws. At 10,000 requests a day that is 1,000 failures.
Constrained decoding removes the failure class entirely. At every step, tokens that would break the format get probability zero. The model can only produce output that fits.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.