Prompt workshop
What is Constrained Decoding?
Constrained decoding lets the sampler emit only tokens that match a grammar, enum, or regex.
Guidance is one templating layer over that idea. “Please return JSON” is not a constraint.
Why it matters
Constrained decoding forces the model to emit only tokens that match a schema — JSON objects, enums, regex patterns. This is fundamentally different from asking the model to produce structured output in prose.
With constrained decoding, malformed output is impossible. Without it, you are parsing freeform text and handling failures.
Key takeaways
- 1Constrained decoding enforces structure at the token level.
- 2Malformed output becomes impossible, not just unlikely.
- 3"Please return JSON" is not constrained decoding.
Common mistakes
- ✕Using prompt instructions for structure instead of token-level constraints.
- ✕Over-constraining output, preventing the model from expressing uncertainty.
Related terms
Concept neighborhood
Terms linked from Constrained Decoding in the glossary graph.
- Constrained Decoding
- Guidance
- Prompt Engineering
- Structured Output