In-Context Learning (ICL)

In-context learning is the remarkable property of large language models: they can perform new tasks from examples given purely in the context window, without updating any weights. Show the model a few (input, output) pairs and it generalizes to new inputs in the same style.

This is different from fine-tuning (which changes weights) and different from few-shot learning in spirit — ICL refers to the underlying mechanism, while few-shot refers to the prompting strategy that exploits it.

Why it works is still partially a research question, but practically it means: the bigger the model, the better its ICL. Smaller models (7B-13B) need more examples to "get it." Frontier models (GPT-4o, Claude 3.5+) often nail the task from one or two examples. The limit is context window — you can only fit so many examples before you run out of tokens.