Temperature
AITemperature is a parameter that controls how random an LLM’s completion is. Lower temperature (e.g., 0) makes output more deterministic and repeatable; higher temperature (e.g., 1) increases diversity and creativity but also variability and risk of hallucination.
Typical Use
- Extraction, structured output — Low temperature for consistent, schema-following output.
- Chat, creative tasks — Slightly higher temperature for more natural variation (still often kept moderate for RAG to avoid straying from sources).
DocLD sets temperature appropriately for extraction and RAG chat. Top-p is another parameter that can be used alone or with temperature to control sampling.
Related Concepts
Temperature affects completion and inference. Top-p is a related sampling parameter. Both influence hallucination and consistency.