Lesson 22 Recurrent Neural Network
Recurrent Neural Network is a type of network which has a layer with recurrent input from its own output.
This network uses an output from the last calculation step as an current input. Therefore, it is valid when you have consecutive input values, and the sequence has its own meaning. There are some achievements on time series analysis, sentence generation, automatic translation, and so on.
The diagram above shows one example of automatic translation. The system inputs four Japanese words, and gets four English words. As you see here, the sequence of words has a big meaning, in terms of natural language. Recurrent network uses all previous outputs as its own inputs. That's why it is suitable for this type of application.
You can easily understand the idea of recurrent network by expanding the whole steps along the time series.
This type of network uses an output from the last step as a current input. This behavior is different from a normal network. Now, we suppose four consecutive words as the inputs. As you see in the diagram, the network takes the sequence of words into consideration for its outputs.