Language models are few Shot learners

Many NLP tasks can be framed as text generation tasks. If you train a language model to autoregressively generate text, you get a model able to extrapolate on a handful of examples of a custom task. This avoids the burden of fine-tuning on new tasks.