Menu
Recently, deep learning approaches with various network architectures have achieved significant performance improvement over existing iterative reconstruction methods in various imaging problems. However, it is still unclear why these deep learning architectures work for specific inverse problems. Moreover, in contrast to the usual evolution of signal processing theory around the classical theories, the link between deep learning and the classical signal processing approaches, such as wavelets, nonlocal processing, and compressed sensing, are not yet well understood. To address these issues, here we show that the long-sought missing link is the convolution framelets for representing a signal by convolving local and nonlocal bases. The convolution framelets were originally developed to generalize the theory of low-rank Hankel matrix approaches for inverse problems, and this paper further extends this idea so as to obtain a deep neural network using multilayer convolution framelets under rectified linear unit (ReLU) nonlinearity. This discovery reveals the limitations of many existing deep learning architectures for inverse problems, and leads us to propose a novel theory for a deep convolutional framelet neural network. Using numerical experiments with various inverse problems, we demonstrate that our deep convolutional framelets network shows consistent improvement over existing deep architectures. This discovery suggests that the success of deep learning stems not from a magical black box, but rather from the power of a novel signal representation using a nonlocal basis combined with a data-driven local basis, which is indeed a natural extension of classical signal processing theory.
Jong Chul Ye is currently KAIST Endowed Chair Professor and Professor of the Dept. of Bio/Brain Engineering and Adjunct Professor at Dept. of Mathematical Sciences of Korea Advanced Institute of Science and Technology (KAIST), Korea. He received the B.Sc. and M.Sc. degrees from Seoul National University, Korea, and the Ph.D. from Purdue University, West Lafayette. Before joining KAIST, he worked at Philips Research and GE Global Research in New York. He has served as an associate editor of IEEE Trans. on Image Processing, and an editorial board member for Magnetic Resonance in Medicine. He is currently an associate editor for IEEE Trans. on Medical Imaging, IEEE Trans. on Computational Imaging, and Journal of Electronic Imaging, and an international advisory board member for Physics in Medicine and Biology. He is also a Senior Editor of IEEE Signal Processing Magazine. He is an elected member of IEEE SPS Technical Committee on Bio-imaging and Signal Processing (BISP), IEEE EMBS Technical Committee on Biomedical Imaging and Image Processing (BIIP), and IEEE SPS Special Interest Group (SiG) on Computational Imaging, and a Technical Liaison Committee of IEEE Trans. on Computational Imaging. He is/was on the organizing committee for IEEE Symp. on Biomedical Imaging (ISBI) 2006, 1st ISMRM Workshop on Machine Learning 2018, and International BASP Frontiers Workshop 2019. He is/was a tutorial/keynote/plenary speaker in various conferences including ISBI, ISMRM, SPIE Medical Imaging, CT Meeting, MICCAI Workshop, IFMIA, etc. His group was the first place winner of the 2009 Recon Challenge at the ISMRM workshop with k-t FOCUSS algorithm, the second winners at 2016 Low Dose CT Grand Challenge organized by the American Association of Physicists in Medicine (AAPM) with the world’s first deep learning algorithm for low-dose CT, and the third place winner for 2017 CVPR NTIRE challenge on example-based single image super-resolution. He was an advisor of student’s best paper awards (1st, and runner-up) at 2013 and 2016 IEEE Symp. on Biomedical Imaging (ISBI). His current research interests include machine learning, compressed sensing and statistical signal processing for various image reconstruction problems in various medical and bioimaging modalities such as MRI, CT, optics, ultrasound imaging, PET, fNIRS, etc..
Copyright ⓒ 2015 KAIST Electrical Engineering. All rights reserved. Made by PRESSCAT
Copyright ⓒ 2015 KAIST Electrical Engineering. All rights reserved. Made by PRESSCAT
Copyright ⓒ 2015 KAIST Electrical
Engineering. All rights reserved.
Made by PRESSCAT
