CyberFreezeDev
Loading
A setting that makes a model's output more focused when low, more varied when high.
Temperature sharpens or flattens the odds of each next token. Low gives focused, consistent output; higher gives variety and more risk of wandering.
A large language model writes one token at a time. For each step it produces a score for every possible next token, and those scores are turned into probabilities. Temperature changes how sharp that probability distribution is before a token is picked.
Raw scores are divided by the temperature before becoming probabilities. Top-p and top-k also shape choices, and providers accept different ranges.
Technically, the model's raw scores (logits) are divided by the temperature before the softmax function turns them into probabilities. Dividing by a number below 1 stretches the gaps between scores, so the top choice dominates. Dividing by a number above 1 shrinks the gaps, flattening the distribution.
Temperature often works alongside top-p (nucleus sampling, which keeps only the smallest set of tokens whose probabilities add up to p) and top-k (keep only the k most likely tokens). Providers set different ranges: OpenAI's API accepts 0 to 2, while Anthropic's API accepts 0 to 1. Some reasoning models fix the value or ignore it, so check the documentation for the model you use.
| Task | Typical setting | Why |
|---|---|---|
| Extracting fields from invoices or forms | Low, near 0 | You want the same answer every time, not variety. |
| Classifying and routing messages | Low | Consistency makes results testable. |
| Customer support replies | Low to moderate | Natural wording, but no improvisation on facts. |
| Ad copy, headlines, brainstorming | Moderate to higher | Variety is the point; a person picks the best. |
Starting points only. Test settings against your own examples.
One model, two settings: invoice extraction runs at temperature 0, while subject line drafting runs higher and asks for five options.
A team uses one model for two jobs. The workflow that reads GST invoices and fills a JSON object runs at temperature 0 with a fixed schema. The workflow that drafts five subject lines for a marketing email runs at a higher temperature and asks for five options in one call. Same model, two settings, two very different behaviours.
Higher temperature adds variety, not accuracy. Fix the prompt, data or retrieval instead.
If answers are wrong, fix the prompt, the data or the retrieval. Higher temperature adds variety, not accuracy, and increases the risk of hallucination.
Even at 0, hosted models can vary slightly. Do not expect identical output.
Even at 0, hosted models can give slightly different outputs because of how requests are batched and computed. Do not rely on byte-identical output.
Change temperature or top-p one at a time, and test each change.
Change one at a time, and test each change.
Use a low value for extraction, classification and anything with a right answer. Use a moderate value for natural replies and a higher one for creative options. Then test on real examples.
No. It makes the model pick its most likely answer, but the most likely answer can still be wrong. Grounding with retrieval-augmented generation and validation does more.
Temperature reshapes all probabilities. Top-p cuts off the unlikely tail and samples only from the most probable tokens. Both control randomness from different angles.
Last updated