Basic RNN Cell

The simplest recurrent network cell, the basic RNN cell, is depicted in Fig. 21. It has a single internal layer, symbolized by the activation a of the layer. In addition to the input x_t at point t in time, the internal state of the cell at the previous point in time, h_t−1, is appended to x_t. In this way, previous information can be persisted through time. As such, this information can then influence the reaction on later events. Delayed reactions on signals can be modeled particularly well this way, for instance.

Fig. 21: Schematic layout of one basic RNN cell. Input x_t at time step t is combined with the cells’ state at the previous time step, h_t−1, to produce the current output y_t and state h_t.