Model Components
Recurrent Neural Network (RNN)
RNNs process sequences step-by-step, maintaining a hidden state vector that summarizes information from all previous steps. LSTM (Long Short-Term Memory) and GRU (Gated Recurrent Unit) variants were designed to mitigate the vanishing gradient problem that made early RNNs ineffective for long sequences.
RNNs dominated NLP from 2014–2017 before being largely superseded by transformers. They remain relevant in some latency-sensitive, streaming applications where full sequence parallel processing is impractical.
Authority Links
Related Terms
Model Components
Neural Network
Computational system of interconnected nodes inspired by the human brain that learns to recognize patterns.
Model Components
Transformer
A neural-network architecture, introduced by Vaswani et al. in 2017, that uses self-attention and parallel computation across all sequence positions — the foundation under virtually every frontier language and multimodal model in production today.
Model Components
Sequence-to-Sequence (Seq2Seq) Models
Models that transform input sequences into output sequences, used in translation and summarization.
Techniques & Methods
Backpropagation
Training algorithm that adjusts neural network weights by propagating prediction errors backward through the network.