Recurrent Neural Networks

Recurrent neural networks (RNNs) are neural networks with hidden states.

Hidden Layer

Layers that are hidden from view on the path from input to output.

Hidden State

A hidden layer output which captured and retained the sequence’s historical information up to their current time step, just like the short-term state or memory of the neural network’s current time step.

Recurrent

Since the hidden state uses the same definition of the previous time step in the current time step, the computation of hidden states is recurrent.

Recurrent Layer

Layers that perform the computation of hidden states in RNNs are called recurrent layers.

Recurrent Neural Network

Neural networks with hidden states based on recurrent computation are named recurrent neural networks.