Emergence - a phenomenon associated with behaviors appearing in llms that were not present in the smaller-scale models.
These behaviors are likely due to the vast scale of data used for training the LLMs.
The emergence phenomenon leads to the belief that the capabilities of today's LLMs are not yet fully known.
Examples
- ability to translate from one language to another, even though the model wasn't precisely trained for this
- it had contact with these languages though