Techniques & Methods
Decoding Rules
Decoding rules determine how a model selects tokens from its probability distribution during generation. Greedy decoding takes the highest-probability token; beam search explores multiple hypotheses; temperature scaling adjusts distribution sharpness; top-p (nucleus) sampling limits to the most probable tokens summing to probability p.
Choosing the right decoding strategy depends on the task: deterministic tasks (code, structured data) favor greedy or low-temperature decoding; creative tasks favor higher temperature sampling. Constrained decoding enforces output formats like valid JSON.
Authority Links
Related Terms
Techniques & Methods
Sequence Generation
Process where models produce sequences—such as words or tokens—based on learned patterns.
Techniques & Methods
Beam Search
Search algorithm that maintains multiple candidate sequences to find high-quality generated outputs.
Techniques & Methods
Greedy Algorithms
Algorithms that make the locally optimal choice at each step to find a global solution.
Techniques & Methods
Generation
Producing new text, code, or content based on learned patterns and a given input prompt.