Browse Teach Curate

LSTM Forget Gate

A sigmoid layer over the current input and previous hidden state that outputs a 0-to-1 factor per cell-state dimension, elementwise-multiplied into the cell state to decay or erase old memory. Its bias is typically initialized to 1 so the network starts by remembering everything.

jeremy Video

Long Short-Term Memory LSTM Architecture and Training in Recurrent Neural Networks

Long Short-Term Memory (LSTM) networks address the vanishing and exploding gradient problems inherent in vanilla Recurrent Neural Networks by architecting a distinct cell state path for long-term mem…