SVAIL Tech Notes: A Look at Persistent Recurrent Neural Nets

Today we posted a new Tech Note in which Greg Diamos, a research scientist at Baidu’s Silicon Valley AI Lab, discusses a new technique for speeding up the training of deep recurrent neural networks.

Greg explains:

At SVAIL, our mission is to create AI technology that lets us have a significant impact on hundreds of millions of people. We believe that a good way to do this is to improve the accuracy of speech recognition by scaling up deep learning algorithms on larger datasets than what has been done in the past. 

These algorithms are very compute intensive, so much so that the memory capacity and computational throughput of our systems limits the amount of data and the size of the neural network that we can train. So a big challenge is figuring out how to run deep learning algorithms more efficiently. 

Doing so would allow us to train bigger models on bigger datasets, which so far has translated into better speech recognition accuracy. Here we want to discuss a new technique for speeding up the training of deep recurrent neural networks.

For more details on how Greg tackles this challenge, please read his post on the SVAIL GitHub blog.

SVAIL Tech Notes are written by engineers for engineers on topics related to AI technologies, techniques, tips and trends.

Previous issues:

Around the World in 60 Days, by Ryan Prenger and Tony Han

Deploying Deep Neural Networks Efficiently, by Chris Fougner

Optimizing RNN Performance, by Erich Elsen

Relevant Links:

SVAIL GitHub Blog

2017-05-22T04:01:04+00:00 March 25th, 2016|
Skip to toolbar