Techniques & Methods
Multitask Learning
Multitask learning (MTL) jointly trains a model across multiple tasks, sharing representations that capture common structure. For example, a model trained on translation, summarization, and question answering simultaneously may develop better language understanding than training on each task separately.
MTL is a key reason large LLMs generalize so well: pre-training on diverse tasks creates shared representations that transfer broadly. Fine-tuning on a single task can cause catastrophic forgetting of other capabilities, making MTL approaches during fine-tuning valuable.
Authority Links
Related Terms
Techniques & Methods
Transfer Learning
Leveraging knowledge learned from one task or domain to improve performance on a related one.
Techniques & Methods
Fine-Tuning
Continuing the training of a pre-trained foundation model on a smaller, curated dataset to specialize its behavior, style, or domain expertise without losing its general capabilities.
Techniques & Methods
Pre-training
Initial phase where a model learns general representations from large datasets before task-specific fine-tuning.
Techniques & Methods
Supervised Fine-Tuning
Refining a pre-trained model's performance on a specific task using labeled example data.