What is a Simple Recurrent Neural network ? - Unrolled Explanation

A simple recurrent neural network is the function of tanH and can be expressed by a recursive formula as:

Simple RNN (Unrolled)

S0 = Previous state or initial state
X1 = input at time step 1

We inputted these value in recursive tanH function and gives “ S1 ” and then multiply with weight  which would be the input for the next layer

Here, S0 & X1 are the input for time step 1 function and give S1 as an output which is input for the next time step2.  And hence this iteration goes on.

As far as Y1 is concerned it is the product of Wn and S1, and Y1 & Y2 So far will be treated as an input for another time step in a similar way. Please mind that Ws Wx & Wy remains the same though out the whole network.

In Recurrent Neural Network, we need not go in much deeper, hence the majority of people use 3-4 layers, deep network models.

Also, the Nth time step has a memory of N-1 time steps and N-1 has some memory of N-2 time step and so on, hence memory can flow through this bunch of time steps.

Illustration using one input:

Illustration using two inputs, the output is calculated using a sigmoid function: