Today we posted a new Tech Note in which Greg Diamos, a research scientist at Baidu’s Silicon Valley AI Lab, discusses a new technique for speeding up the training of deep recurrent neural networks.
At SVAIL, our mission is to create AI technology that lets us have a significant impact on hundreds of millions of people. We believe that a good way to do this is to improve the accuracy of speech recognition by scaling up deep learning algorithms on larger datasets than what has been done in the past.
These algorithms are very compute intensive, so much so that the memory capacity and computational throughput of our systems limits the amount of data and the size of the neural network that we can train. So a big challenge is figuring out how to run deep learning algorithms more efficiently.
Doing so would allow us to train bigger models on bigger datasets, which so far has translated into better speech recognition accuracy. Here we want to discuss a new technique for speeding up the training of deep recurrent neural networks.
For more details on how Greg tackles this challenge, please read his post on the SVAIL GitHub blog.
SVAIL Tech Notes are written by engineers for engineers on topics related to AI technologies, techniques, tips and trends.
Around the World in 60 Days, by Ryan Prenger and Tony Han
Deploying Deep Neural Networks Efficiently, by Chris Fougner
Optimizing RNN Performance, by Erich Elsen
SVAIL GitHub Blog