Model Components
Foundational Model
Foundational models (or foundation models) are trained at massive scale on diverse data, giving them general capabilities that can be adapted to specific tasks through fine-tuning or prompting. Examples include GPT-4, Claude, Gemini, Llama, and DALL-E.
The term, coined by Stanford's CRFM in 2021, emphasizes that these models serve as a "foundation" for building applications—reducing AI development costs by amortizing expensive pre-training across many downstream use cases.
Authority Links
Related Terms
Model Components
Large Language Model (LLM)
A transformer-based neural network with billions to trillions of parameters, trained on broad text corpora to predict the next token and able to generate, summarize, classify, and reason over natural language.
Techniques & Methods
Pre-training
Initial phase where a model learns general representations from large datasets before task-specific fine-tuning.
Techniques & Methods
Fine-Tuning
Continuing the training of a pre-trained foundation model on a smaller, curated dataset to specialize its behavior, style, or domain expertise without losing its general capabilities.
Techniques & Methods
Transfer Learning
Leveraging knowledge learned from one task or domain to improve performance on a related one.