Lstms mitigate rnn issues

LSTMs mitigate some of the issues of RNNs, namely the vanishing and exploding gradients, by introducing some specialized neural ensembles. Through three gates (i.e. input, forget, output), the representations stored in the RNN are carefully modulated across time. The input gate controls the influence of the input on memory, the output gate controls how much of the memory is written to output, and the forget gate controls how much of the memory is preserved to the next step.

Resources

Backlinks