Chip Huyen • 1/16/2024

Generation configurations: temperature, top-k, top-p, and test time compute

This technical article delves into the probabilistic nature of AI language models and explains core generation configurations. It details how sampling strategies, including temperature, top-k, and top-p, influence output creativity, consistency, and factuality. The piece also covers test time compute and structured outputs, providing a guide for developers to better control model behavior.

0 comments

#Machine Learning #sampling #Temperature