In this blog post, I review the 2016 NIPS paper “learning to learn by gradient descent by gradient descent” (I abbreviate as LLGG) by Andrychowicz et al.

Recurrent neural networks (RNNs) are neural networks to model sequential data. RNNs are often used in speech recognition and natural language processing. In this blog post, I discuss one of the most popular RNNs, a long short-term memory (LSTM) network. Then I briefly address a training procedure for a LSTM.