Module 10: Settings and prompting · Lesson 10.1 · about 20 min
Temperature, top-p and the other dials
What temperature, top-p, top-k, min-p, repeat penalty and seed do to a local model's answers, with settings for facts, writing and code.
Module 10 · about 20 minutes · read first: 2.1. You’ll be able to tune a model’s answers from careful and repeatable to varied and creative.
1. Why this matters
The same model can feel dull, wild or just right, depending on a few settings. Cloud apps hide them. Local engines let you set them yourself. Getting them right fixes many “this model is bad” complaints.
2. The dials
At each step the model gives every possible next token a score (Lesson 2.1). The scores become chances, and one token is picked. The dials change how that pick happens:
- Temperature: how bold the pick is. 0 always takes the top token; 0.7 is balanced; above 1 gets adventurous.
- top-k: keep only the k most likely tokens (for example 40) and ignore the rest.
- top-p: keep the most likely tokens until their chances add up to p (for example 0.9), and ignore the rest.
- min-p: drop any token whose chance is less than a share of the top token’s chance (for example 0.05, so under 5% of the leader’s chance). Works well at higher temperatures.
- Repeat penalty: make tokens that were already used a little less likely (about 1.05 to 1.1).
- Seed: a fixed starting number for the dice, so the “random” choices repeat exactly.
- Max tokens (
num_predictin Ollama): a cap on the answer’s length.
Think of a waiter suggesting a dish. Temperature is how adventurous the waiter is. top-k, top-p and min-p take the dishes nobody orders off the menu before the waiter suggests anything.
The scores. After "The weather today is", the model scores four words: sunny 2, cloudy 1.5, rainy 1, purple −1. Higher means more likely.
Temperature 1. Softmax turns the scores into chances: sunny 49%, cloudy 30%, rainy 18%, purple 2.5%. They add up to 100%.
Temperature 0.5. Dividing the scores by 0.5 doubles the gaps, so the leader pulls ahead: sunny 66%, cloudy 24%, rainy 9%, purple 0.2%. Answers get steadier.
Temperature 1.5. Dividing by 1.5 shrinks the gaps: sunny 42%, cloudy 30%, rainy 22%, purple 6%. Answers get more varied, and odd words creep in.
top-p 0.9 (at temperature 1). Sunny + cloudy is 79%, not yet 90%; adding rainy makes 98%, so those three stay and purple is cut. Their chances are scaled back up to 100%: about 51%, 31% and 19%. Then one is picked.
3. Starting points
| Job | Temperature | Other |
|---|---|---|
| Extraction, JSON, classification | 0 to 0.2 | fixed seed |
| Facts and summaries | 0.2 to 0.5 | top-p 0.9 |
| Writing and ideas | 0.7 to 1.0 | min-p 0.05 |
| Code | 0.2 or the maker’s value | |
| Reasoning models | the maker’s value | many suggest about 0.6 to 1.0 |
Always check the model card first. Makers publish recommended settings, and they matter.
“Temperature 0 makes the model tell the truth.” It only makes the model repeat its single most likely answer. If that answer is wrong, you get the same wrong answer every time. Low temperature buys consistency, not accuracy. For facts, give the model the source text (Lesson 10.5 and Module 12).
4. Where to set them
- LM Studio: the right-hand panel of the chat.
- Ollama: in the chat type
/set parameter temperature 0.2; in code,options={"temperature": 0.2}(Lesson 11.2); permanently in a Modelfile withPARAMETER temperature 0.2. - llama.cpp:
--temp 0.2 --top-p 0.9on the command line.
Maths and principle
Principle: temperature divides the scores before they are turned into chances; the other dials cut the list before the pick.
Equation: the chance of token i is
where is token i’s score and is the temperature.
Analogy: a quiz contestant. Low temperature: always says their first instinct. High: sometimes takes a wild guess. top-p: only guesses from answers they think are plausible.
Try it: two tokens with scores 2 and 1 at T = 1, then T = 0.5? (T = 1: e² : e¹ ≈ 7.4 : 2.7, about 73% and 27%. T = 0.5: e⁴ : e² ≈ 54.6 : 7.4, about 88% and 12%.)
- si (score)
- the model's raw rating of token i as the next token; bigger means more likely
- T (temperature)
- a number you choose that stretches or squeezes the gaps between scores; a small T makes the leader win more often
- es/T
- turns each score into a positive weight, so even low scores keep a small chance
- ÷ the sum
- shares out 100% among all the tokens in proportion to their weights
- Physical meaning
- Temperature is a "boldness" knob. Near 0 the model always says the same safest thing, like a cautious clerk. Near 1.5 it takes risks, like a brainstorming friend. top-p and min-p remove the silly options before the dice are rolled.
5. Worked example
Made-up numbers. Bilal extracts invoice totals to JSON. At temperature 0.8, one file in ten has a made-up field. At temperature 0 with a JSON schema (Lesson 11.3), every file comes out the same and the checks pass.
6. Practice
- Ask your model for five names for a bakery at temperature 0, then 1.0. Compare.
- Find the recommended settings on the model card of your main model.
- Set them permanently in a Modelfile or an LM Studio preset.
Answers
- At 0 you get the same list every time; at 1.0 it varies.
- Often in a section called “Best practices” or “Usage”.
- Ollama:
ollama create mymodel -f Modelfile; LM Studio: save a preset.
New words: temperature, top-k, top-p, min-p, repeat penalty, seed.
- At each step the model scores every possible next token; the dials decide how one is picked.
- Temperature divides the scores: 0 always picks the top token, about 0.7 is balanced, above 1 is adventurous.
- top-k, top-p and min-p cut unlikely tokens from the list before the pick.
- Use 0 to 0.2 (and a fixed seed) for extraction and JSON, 0.7 to 1.0 for writing, and the model card’s values when they exist.
- Low temperature gives the same answer every time, not a truer one.
Check yourself
Sources
- Ollama Modelfile parameters (temperature, top_k, top_p, min_p, repeat_penalty, seed, num_predict), official documentation, https://docs.ollama.com/modelfile
- llama.cpp server sampling options, project documentation (official page), https://github.com/ggml-org/llama.cpp/tree/master/tools/server
- Holtzman, A. et al. (2019), The Curious Case of Neural Text Degeneration (nucleus or top-p sampling), https://arxiv.org/abs/1904.09751
- Nguyen, M. et al. (2024), min-p sampling for creative and coherent outputs at high temperature, https://arxiv.org/abs/2407.01082
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Prices and model names change fast: check the official source before you buy.