If you read the papers[1] on RProp it seems like a great algorithm that should tremendously speed up the time to convergence of a large neural network. I thought it was a no-brainer to apply this to modern CNNs that were becoming so popular, like GoogLeNet, VGG, and ResNet. When I went looking it seemed like almost everyone was using SGD. There was almost no mention of RProp.
The promise of RProp is fast convergence with no hyperparameter tuning. The penalty is that it is designed to operate on the whole training set as one unit, not on mini-batches. Does it live up to its promise, and can we do anything about the penalty?
Showing posts with label rprop. Show all posts
Showing posts with label rprop. Show all posts
Monday, May 30, 2016
Subscribe to:
Posts (Atom)