Deep Neural Network Structured Sparse Coding for Online Processing

作     者：Zhao, Haoli Ding, Shuxue Li, Xiang Huang, Huakun 

作者机构：Univ Aizu Sch Comp Sci & Engn Aizu Wakamatsu Fukushima 9650005 Japan 

出 版 物：《IEEE ACCESS》 (IEEE Access)

年 卷 期：2018年第6卷

页      面：74778-74791页

核心收录：

基　　金：Ministry of Education, Culture, Sports, Science and Technology, Japan [16K00335] Grants-in-Aid for Scientific Research [16K00335] Funding Source: KAKEN 

主　　题：Sparse coding deep neural network weighted iterative shrinkage thresholding algorithm unsupervised learning real-time video denoising 

摘      要：Sparse coding, which aims at finding appropriate sparse representations of data with an overcomplete dictionary set, has become a mature class of methods with good efficiency in various areas, but it faces limitations in immediate processing such as real-time video denoising. Unsupervised deep neural network structured sparse coding (DNN-SC) algorithms can enhance the efficiency of iterative sparse coding algorithms to achieve the goal. In this paper, we first propose a sparse coding algorithm by adding the idea weighted in the iterative shrinkage thresholding algorithm (ISTA), named WISTA, which can enjoy the benefit of the l(p) norm (0 p 1) sparsity constraint. Then, we propose two novel DNN-SC algorithms by combining deep learning with WISTA and the iterative half thresholding algorithm (IHTA), which is the 10.5 norm sparse coding algorithm. Furthermore, we present that by changing the loss function, the DNN can be learned supervisedly and unsupervisedly. Unsupervised learning is the key to ensure the DNN to be learned online during processing, which enables the use of the DNN-SC algorithms in applications lacking labels for signals. Synthetic data experiments show that WISTA can outperform ISTA and IHTA. Moreover, the DNN-structured WISTA can successfully achieve converged results of WISTA. In real-world data experiments, the procedure of utilizing DNN-SC algorithms in image denoising is first presented. All DNN-SC algorithms can accelerate at least 45 times while maintaining PSNR results compared with their corresponding sparse coding algorithms. Finally, the strategy of utilizing DNN-SC algorithms in real-time video denoising is presented. The video-denoising experiments show that the DNN-structured ISTA and WISTA can conduct real-time video denoising for 25 frames/s 360 x 480 pixels gray-scaled videos.