Model Components
Model Architecture
Model architecture defines how a neural network is organized: the types and sizes of layers, how they connect, what activation functions are used, and how information flows through the network. Architecture choices profoundly affect what tasks a model can learn, its computational requirements, and how it scales.
Key architectural choices include depth (number of layers), width (size of layers), attention heads, feed-forward dimension, and normalization strategy. Architecture search and ablation studies help identify optimal designs for given tasks and compute budgets.
Authority Links
Related Terms
Model Components
Transformer
A neural-network architecture, introduced by Vaswani et al. in 2017, that uses self-attention and parallel computation across all sequence positions — the foundation under virtually every frontier language and multimodal model in production today.
Model Components
Neural Network
Computational system of interconnected nodes inspired by the human brain that learns to recognize patterns.
Model Components
Parameter
A learnable variable within a model whose value is adjusted during training to minimize prediction error.
Core Concepts
Deep Learning
Subset of ML using neural networks with many layers to analyze complex data representations.