Techniques & Methods
Regularization
Regularization methods add constraints to the training objective to discourage models from memorizing training data. L1 (Lasso) and L2 (Ridge) regularization add penalty terms for large weights. Dropout randomly deactivates neurons during training, preventing co-adaptation.
Regularization is essential for building models that generalize to new data. In deep learning, batch normalization, dropout, and weight decay are standard techniques applied throughout training.
Authority Links
Related Terms
Core Concepts
Overfitting
Model learns detail and noise in training data too thoroughly, reducing generalization.
Core Concepts
Variance
Amount by which model predictions vary from average, reflecting sensitivity to training data.
Techniques & Methods
Training
Teaching a model to make accurate predictions by exposing it to large datasets.
Techniques & Methods
Backpropagation
Training algorithm that adjusts neural network weights by propagating prediction errors backward through the network.