Prompting: change the context

Suppose we want a model to label a review as positive or negative. We could change its parameters through training, or keep the parameters fixed and put the task in its input.

A prompt supplies that input. For example:

Classify the sentiment as positive or negative. Return only the label. Review: “The battery lasted all day.”

No examples are supplied, so this is zero-shot prompting. Add labeled reviews before the question and it becomes few-shot prompting. Those examples can communicate the task, labels, and desired output format.

What changes when examples help?

During ordinary inference, the model’s weights stay fixed. The extra tokens change its intermediate representations and next-token scores. This is in-context learning.

It does not require instruction tuning in every case: GPT-3’s few-shot experiments used prompts and demonstrations without task-specific gradient updates. Instruction tuning is a separate way to improve how a model responds to requests.

Does this ability appear only above a certain model size? The discussion of scale and emergence separates the experimental findings from the idea of a universal threshold.

The replies are scripted to illustrate the distinction. This is not a live model evaluation or a guarantee that adding examples will improve an answer.

System instructions are also context

An application may provide system instructions describing its role, format, and constraints. Chat models are trained to interpret message roles; the exact format and priorities depend on the system. These instructions are not necessarily shown in the chat interface.

The application may also include conversation history or retrieved documents. More context increases the information available, but can increase cost and distract from relevant details. A document inside the context should not automatically be treated as an instruction from the user.

Context and training have different effects

Change	What it modifies	What persists
Prompting	Tokens supplied for this prediction	Context that the application retains or supplies again
Fine-tuning	Model parameters	The saved updated parameters
Retrieval	Documents selected and supplied as context	The external document collection, independently of model weights

Fine-tuning can target many behaviors; it does not necessarily bake in just one task. Retrieval can supply changing or private information without storing it in model weights. Neither guarantees correct answers.

To compare prompts, keep a set of examples separate from those used to revise the prompt. Otherwise you may improve the prompt for the examples you kept inspecting while learning little about its performance on new ones. This is the same validation distinction used for other models.

Sources · 5

1. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., et al. (2020). Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems 33 (NeurIPS 2020). arXiv:2005.14165.
2. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., et al. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (NeurIPS 2022). arXiv:2203.02155.
3. Anthropic. System Prompts. Claude Platform Documentation (Release Notes). Accessed 2026-06-25.
4. Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685.
5. Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022). Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? In Proceedings of EMNLP 2022 (pp. 11048–11064). arXiv:2202.12837.

Full bibliography →