I am currently writing my thesis and, as a part of my text, I created some very nice vector diagrams I would like to share. To give credit, I was inspired by Christopher Olah’s Understanding LSTM Networks post, but I wanted to make vector images of those, as PNG diagrams don’t look very professional in PDFs. If you are not sure about the topic of this post, please visit the mentioned Christopher’s page to get the basic knowledge about RNNs and LSTMs.
First of all, I created a visualization of RNN unrolling (svg file).
Then, LSTM with peephole connections (svg file). Peepholes can be easily removed.
And finally GRU (svg file).
I spent some time creating those diagrams, so feel free to save time and reuse them. If you will, send me a message, I will be glad somebody found this useful.