(November 16) Deep Learning: Data-driven function approximation for information processing

In this talk, I will show how deep neural networks, also known as deep learning algorithms, could be utilized as powerful data-driven function approximators for various information processing problems. After giving a brief overview on deep learning, I will give two concrete examples from my own research on how we can effectively train deep neural networks for given information processing tasks. First, for the speech recognition example, I will present RNNDrop, a new regularization algorithm for recurrent neural networks (RNN). I will show that when the deep RNN-based acoustic models are trained with RNNDrop, they can significantly outperform the previous state-of-the-arts on the two speech recognition benchmark datasets, TIMIT and WSJ.  Second, for the discrete denoising example, I will present Neural DUDE, a novel extension of the Discrete Universal DEnoiser (DUDE) algorithm, which was originally proposed in 2005 based on the information theoretic principles. I will show that Neural DUDE, a deep neural network-based denoiser trained with  “pseudo-labels” (and without any ground-truth labels), more than halves the error rates of DUDE on several different applications, such as binary image denoising and DNA sequence denoising. I will conclude the talk with some other on-going and future projects.