In this talk, I will show how deep neural networks, also known as deep learning algorithms, could be utilized as powerful data-driven function approximators for various information processing problems. After giving a brief overview on deep learning, I will give two concrete examples from my own research on how we can effectively train deep neural networks for given information processing tasks. First, for the speech recognition example, I will present RNNDrop, a new regularization algorithm for recurrent neural networks (RNN). I will show that when the deep RNN-based acoustic models are trained with RNNDrop, they can significantly outperform the previous state-of-the-arts on the two speech recognition benchmark datasets, TIMIT and WSJ. Second, for the discrete denoising example, I will present Neural DUDE, a novel extension of the Discrete Universal DEnoiser (DUDE) algorithm, which was originally proposed in 2005 based on the information theoretic principles. I will show that Neural DUDE, a deep neural network-based denoiser trained with “pseudo-labels” (and without any ground-truth labels), more than halves the error rates of DUDE on several different applications, such as binary image denoising and DNA sequence denoising. I will conclude the talk with some other on-going and future projects.
Taesup Moon received the B.S. degree in electrical engineering from Seoul National University in 2002 and the M.S. and Ph.D. degrees in electrical engineering from Stanford University in 2004 and 2008, respectively. From 2008 to 2012, he was a Research Scientist with Yahoo! Labs, and he held a Postdoctoral Researcher appointment with the Department of Statistics, University of California at Berkeley, from 2012 to 2013. From 2013 to 2015, he was a Research Staff Member with Samsung Advanced Institute of Technology (SAIT). Currently, he is an assistant professor at the Department of Information and Communication Engineering, Daegu-Gyeongbuk Institute of Science and Technology (DGIST). His research interests include diverse areas such as statistical machine learning / deep learning, data science, signal processing, optimization, and information theory.