Emergent Behaviour
Capabilities that appear suddenly in large models without being explicitly trained for.
Full Definition
Emergent behaviour refers to capabilities that are absent in smaller models and appear abruptly as model scale (parameters, data, or compute) crosses certain thresholds, seemingly without being directly optimised for. Examples include multi-step arithmetic, chain-of-thought reasoning, in-context learning, and theory-of-mind reasoning. The phenomenon challenges the intuition that model capabilities should scale smoothly and predictably. Some researchers argue that apparent emergent abilities are artefacts of discontinuous evaluation metrics rather than truly discontinuous capabilities. Understanding emergence is critical for forecasting when dangerous capabilities might appear and for setting compute-based safety thresholds in responsible scaling policies.
Examples
GPT-3 demonstrating in-context few-shot learning — a capability not explicitly trained for that emerged only at the 175B parameter scale.
Arithmetic reasoning appearing abruptly in models above ~50B parameters when evaluated with chain-of-thought prompting.
Apply this in your prompts
PromptITIN automatically uses techniques like Emergent Behaviour to build better prompts for you.
Related Terms
Scaling Laws
Empirical relationships describing how model performance improves predictably wi…
View →Large Language Model
A neural network with billions of parameters trained on text to understand and g…
View →Reasoning
The capacity of an AI model to derive conclusions from premises through logical …
View →