Temperature & Parameters
These three settings control how an LLM generates its response. Understanding them is fundamental to prompt engineering.
Temperature
Temperature controls the randomness of the model's output. It affects how the model samples from its probability distribution at each token generation step.
- 0.0 — Deterministic. Always picks the most likely next token. Identical input = identical output every time.
- 0.7 — Balanced. Some variation while remaining coherent. Good general default.
- 1.0–1.2 — Creative. Explores less likely paths. More varied, surprising responses.
- 2.0 — Very high randomness. Outputs become incoherent at extreme values.
Temperature is not about "creativity" in the human sense — it's about sampling distribution. High temperature makes the model consider less probable word choices. Low temperature makes it stick to the most statistically likely continuations.
Max tokens
Max tokens sets an upper limit on the response length. One token is roughly 0.75 words in English.
| Tokens | Approximate length |
|---|---|
| 256 | ~190 words — short paragraph |
| 512 | ~380 words — half a page |
| 1024 | ~760 words — full page |
| 4096 | ~3000 words — long document |
Setting max tokens too low causes the model to truncate mid-sentence. Setting it too high wastes tokens (and API cost) for tasks that need short answers. Match max tokens to your expected output length.
Model selection
PromptLab uses Groq-hosted models. Groq's inference is exceptionally fast due to their LPU hardware — you'll notice very low latency compared to standard GPU inference.
Larger models (70B parameters) are more capable but slightly slower and more expensive per token. Smaller models (7B–8B) are faster and cheaper but may struggle with complex reasoning tasks.
Quick reference
| Task type | Temperature | Max tokens |
|---|---|---|
| Factual Q&A | 0.0 | 256–512 |
| Data extraction / classification | 0.0–0.2 | 256–512 |
| Summarization | 0.3–0.5 | 512–1024 |
| General assistant | 0.7 | 1024 |
| Creative writing | 1.0–1.2 | 1024–4096 |
| Brainstorming / ideation | 1.0–1.5 | 512–1024 |