Techniques & Methods
Training
Training is the process of adjusting a model's parameters by minimizing a loss function over a dataset through iterative optimization (typically stochastic gradient descent). The model learns to map inputs to correct outputs by repeatedly seeing examples and adjusting its weights via backpropagation.
Training frontier LLMs requires thousands of GPUs running for weeks or months and costs tens to hundreds of millions of dollars. Data quality, learning rate schedules, and regularization strategies are critical to training success.
Authority Links
Related Terms
Techniques & Methods
Backpropagation
Training algorithm that adjusts neural network weights by propagating prediction errors backward through the network.
Miscellaneous
Training Data
The labeled or unlabeled dataset used to fit a model's parameters during the learning process.
Techniques & Methods
Fine-Tuning
Continuing the training of a pre-trained foundation model on a smaller, curated dataset to specialize its behavior, style, or domain expertise without losing its general capabilities.
Techniques & Methods
Pre-training
Initial phase where a model learns general representations from large datasets before task-specific fine-tuning.