Core Concepts
Overfitting
An overfit model performs exceptionally well on training data but fails on new, unseen examples because it has memorized noise rather than learned general patterns. It is one of the most common failure modes in machine learning.
Prevention techniques include regularization (L1/L2), dropout, early stopping, cross-validation, and using more diverse training data. In LLMs, overfitting on fine-tuning data can cause the model to lose its general capabilities.
Authority Links
Related Terms
Core Concepts
Variance
Amount by which model predictions vary from average, reflecting sensitivity to training data.
Techniques & Methods
Regularization
Techniques that prevent overfitting by penalizing model complexity during training.
Miscellaneous
Training Data
The labeled or unlabeled dataset used to fit a model's parameters during the learning process.
Techniques & Methods
Validation
Evaluating model performance on data held separate from the training set.