基于稀疏表示和结构自相似性的单幅图像盲解卷积算法
 首页 关于本刊 编 委 会 期刊动态 作者中心 审者中心 读者中心 下载中心 联系我们 English
 自动化学报  2017, Vol. 43 Issue (11): 1908-1919 PDF

1. 清华大学电子工程系 北京 100084;
2. 北京工业大学计算机学院 北京 100124

Single Image Blind Deconvolution Using Sparse Representation and Structural Self-similarity
CHANG Zhen-Chun1, YU Jing2, XIAO Chuang-Bai2, SUN Wei-Dong1
1. Department of Electronic Engineering, Tsinghua University, Beijing 100084;
2. College of Computer Science and Technology, Beijing University of Technology, Beijing 100124
Manuscript received : April 22, 2016, accepted: September 30, 2016.
Foundation Item: Supported by National Natural Science Foundation of China (61501008), The Capital Health Research and Development of Special (2014-2-4025)
Corresponding author. YU Jing Lecturer at the College of Computer Science and Technology, Beijing University of Technology. She received her Ph. D. degree from Tsinghua University in 2011. Her research interest covers image processing and pattern recognition. Corresponding author of this paper
Recommended by Associate Editor YANG Jian
Abstract: Blind image deconvolution aims to recover the latent sharp image from a blurry image when the blur kernel is unknown. Since blind deconvolution is an underdetermined problem, existing methods take advantage of various prior knowledge directly or indirectly. In this article, we propose a single image blind deconvolution method based on sparse representation and structural self-similarity. In our method, we add the image sparsity prior and structural self-similarity prior to the blind deconvolution objective function as regularization constraints, and we utilize the structural self-similarity between different image scales by taking the down-sampled version of observed blurry image as the sparse representation dictionary training set so that the sparsity of the latent sharp image under this dictionary can be ensured. Finally, we estimate the blur kernel and sharp image alternately. Experimental results on both simulated and real blurry images demonstrate that the blur kernels estimated by our method are accurate and robust, and that the restored images have high visual quality with sharp edges.
Key words: Sparse representation     structural self-similarity     blind deconvolution     blur kernel     deblurring

 $$$\pmb{y}=\pmb{h}*\pmb{x}+\pmb{n} \label{eq:BlurModel}$$$ (1)

 $$$(\hat{\pmb{x}}, \hat{\pmb{h}})=\arg\max\limits_{\pmb{x}, \pmb{h}}P(\pmb{y}|\pmb{h}, \pmb{x})P(\pmb{x})P(\pmb{h}) \label{eq:mapxk}$$$ (2)

 $$$\begin{array}{l} P(\pmb{y}|\pmb{h}, \pmb{x}) = \prod\limits_{*}\prod\limits_{i}\mathcal{N}\left((\partial_{*}\pmb{n})_{i}|0, \sigma_{*}\right) \end{array} \label{eq:gaussian}$$$ (3)

 \begin{align} \lg &(P(\pmb{y}|\pmb{h}, \pmb{x})P(\pmb{x})P(\pmb{h})) = \nonumber\\ &\sum\limits_{*}\sum\limits_{i}\lg\left( \dfrac{1}{\sqrt{2\pi}\sigma_{*}} \exp(-\dfrac{(\partial_*\pmb{y}-\pmb{h}*\partial_*\pmb{x})_i}{2\sigma_*^2})\right)+\nonumber\\ & \lg(P(\pmb{x})) +\lg(P(\pmb{h})) =\nonumber \\ &-\sum\limits_{*}\frac{1}{2\sigma_*^2}||\partial_*\pmb{y}-\pmb{h}* \partial_*\pmb{x}||_2^2 + \lg(P(\pmb{x}))+\nonumber\\ & \lg(P(\pmb{h})) - \sum\limits_*\sum\limits_i\lg\sqrt{2\pi}\sigma_* \label{eq:logpost} \end{align} (4)

 \begin{align} (\hat{\pmb{x}}, \hat{\pmb{h}}) = &\arg\min\limits_{\pmb{x}, \pmb{h}}\Big\{\sum\limits_{*}\omega_{*}||\partial_{*}\pmb{y}- \pmb{h}*\partial_{*}\pmb{x}||_2^2+\nonumber \\&\lambda_x\rho(\pmb{x}) + \lambda_h\rho(\pmb{h})\Big\} \end{align} (5)

1 图像的结构自相似性与稀疏表示 1.1 图像的结构自相似性

 图 1 图像不同尺度间的结构自相似性 Figure 1 Structural self-similarity cross scales of image

1.2 图像的稀疏表示

 $$$Q_j\pmb{X} = \Psi\pmb{\alpha}_j, ||\pmb{\alpha}_j||_0 \ll n$$$ (8)

 $$$\min\limits_{\Psi, \pmb{\alpha}_1, \cdots, \pmb{\alpha}_m}\sum\limits_{i=1}^{m}||\pmb{s}_i-\Psi\pmb{\alpha}_i||_2^2\quad {{\rm s. t.}}\ \forall i \ ||\pmb{\alpha}_i||_0 \leq T$$$ (9)

 $$$\min\limits_{\pmb{\alpha}_j} ||Q_j\pmb{X}-\Psi\pmb{\alpha}_j||_2^2\quad {{\rm s. t.}}\ ||\pmb{\alpha}_j||_0 \leq T$$$ (10)

 $$$\min\limits_{\pmb{\alpha}_j} ||\pmb{\alpha}_j||_0 \quad {{\rm s. t.}}\ ||Q_j\pmb{X}-\Psi\pmb{\alpha}_j||_2^2 \leq \epsilon$$$ (11)

 $$$\pmb{q}(\xi, \eta) = \pmb{f}(\xi, \eta) * \pmb{h}(\xi, \eta) \label{eq:q}$$$ (12)

3) 固定$\hat{\pmb{h}}_k$, 给定$\hat{\pmb{x}}_k$, 更新$\hat{\pmb{x}}_{k+1}$.

 $$$\begin{array}{l} \hat{\pmb{x}}_{k+1} = \arg\min\limits_{\pmb{x}} \Big\{||\nabla\pmb{y}-\nabla\pmb{x}\otimes \hat{\pmb{h}}_k||_2^2+ \\ \qquad\quad\ \lambda_c\sum\limits_{j} ||Q_j\pmb{X}-\Psi\pmb{\alpha}_j||_2^2+ \\ \qquad\quad\ \lambda_s\sum\limits_j||Q_j\pmb{X}-\sum\limits_{i\in S_j}w_i^j R_i\pmb{X}^{\alpha}||_2^2+ \\ \qquad\quad\ \lambda_g||\nabla\pmb{x}||_2^2\Big\}\quad{\rm s. t.}\ \forall j\ ||\pmb{\alpha}_j||_0\leq T \end{array} \label{eq:ObjFuncX}$$$ (18)

 $$$\begin{array}{l} \hat{\pmb{X}}_{k+1} = \arg\min\limits_{\pmb{X}} \Big\{||G_x\pmb{Y}-H_kG_x\pmb{X}||_2^2+ \\ \qquad\quad\ \ ||G_y\pmb{Y}-H_kG_y\pmb{X}||_2^2+ \\ \qquad\quad\ \ \lambda_c\sum\limits_{j}||Q_j\pmb{X}-\Psi\pmb{\alpha}_j||_2^2+ \\ \qquad\quad\ \ \lambda_s\sum\limits_j||Q_j\pmb{X}-\sum\limits_{i\in S_j}w_i^j R_i\pmb{X}^{\alpha}||_2^2+ \\ \qquad\quad\ \ \lambda_g(||G_x\pmb{X}||_2^2+||G_y\pmb{X}||_2^2)\Big\} \\ \qquad\quad\ \ {\rm s. t.}\ \forall j\ ||\pmb{\alpha}_j||_0\leq T \end{array} \label{eq:ObjFuncXVector}$$$ (19)

 图 4 各算法在Levin等[4]数据集上的ER累积分布 Figure 4 Cumulative distribution of error ratios on Levin et al.[4] dataset

 图 5 各算法在Sun等[19]数据集上的ER累积分布 Figure 5 Cumulative distribution of error ratios on Sun et al.[19] dataset

 图 6 Sun等[19]、Michaeli等[9]以及本文算法实验结果比较 Figure 6 Visual comparisons with methods of Sun et al.[19], Michaeli et al.[9] and ours

 图 7 各算法对有噪模糊图像的复原结果比较 Figure 7 Visual comparisons with some methods on noisy blurry image
 图 8 不同强度高斯噪声下各算法盲解卷积结果的均方误差 Figure 8 Mean squared error of some methods under different noise levels

3.2 真实模糊图像实验

 图 9 各算法在真实模糊图像(模糊核未知)上的实验结果比较 Figure 9 Visual comparisons with some state-of-the-art methods on real-world photos with unknown kernel
 图 10 各算法在真实模糊图像(模糊核未知)上的实验结果比较 Figure 10 Visual comparisons with some state-of-the-art methods on real-world photos with unknown kernel
4 结论

 1 Joshi N, Szeliski R, Kriegman D J. PSF estimation using sharp edge prediction. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK, USA:IEEE, 2008. 1-8 http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=4587834 2 Cho S, Lee S. Fast motion deblurring. ACM Transactions on Graphics, 2009, 28(5): Article No.145. 3 Xu L, Jia J Y. Two-phase kernel estimation for robust motion deblurring. In: Proceedings of the 11th European Conference on Computer Vision. Heraklion, Crete, Greece:Springer, 2010. 157-170 http://dl.acm.org/citation.cfm?id=1886077 4 Levin A, Weiss Y, Durand F, Freeman W T. Understanding and evaluating blind deconvolution algorithms. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL, USA:IEEE, 2009. 1964-1971 http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=5206815 5 Fergus R, Singh B, Hertzmann A, Rowels S T, Freeman W T. Removing camera shake from a single photograph. ACM Transactions on Graphics, 2006, 25(3): 787-794. DOI:10.1145/1141911 6 Levin A, Weiss Y, Durand F, Freeman W T. Efficient marginal likelihood optimization in blind deconvolution. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA:IEEE, 2011. 2657-2664 http://dl.acm.org/citation.cfm?id=2192016 7 Shan Q, Jia J Y, Agarwala A. High-quality motion deblurring from a single image. ACM Transactions on Graphics, 2008, 27(3): Article No. 73. 8 Perrone D, Favaro P. Total variation blind deconvolution:the devil is in the details. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA:IEEE, 2014. 2909-2916 http://ieeexplore.ieee.org/document/6909768/ 9 Michaeli T, Irani M. Blind deblurring using internal patch recurrence. In: Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland:Springer International Publishing, 2014. 783-798 http://www.researchgate.net/publication/291197096_Blind_Deblurring_Using_Internal_Patch_Recurrence 10 Lai W S, Ding J J, Lin Y Y, Chuang Y Y. Blur kernel estimation using normalized color-line priors. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA:IEEE, 2015. 64-72 http://doi.ieeecomputersociety.org/10.1109/CVPR.2015.7298601 11 Zhang H C, Yang J C, Zhang Y N, Huang T S. Sparse representation based blind image deblurring. In: Proceedings of the 2011 IEEE International Conference on Multimedia and Expo. Barcelona, Spain:IEEE, 2011. 1-6 http://ieeexplore.ieee.org/document/6012035/ 12 Li H S, Zhang Y N, Zhang H C, Zhu Y, Sun J Q. Blind image deblurring based on sparse prior of dictionary pair. In: Proceedings of the 21st International Conference on Pattern Recognition. Tsukuba, Japan:IEEE, 2012. 3054-3057 http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=6460809 13 Pan Zong-Xu, Yu Jing, Xiao Chuang-Bai, Sun Wei-Dong. Single-image super-resolution algorithm based on multi-scale nonlocal regularization. Acta Automatica Sinica, 2014, 40(10): 2233-2244.( 潘宗序, 禹晶, 肖创柏, 孙卫东. 基于多尺度非局部约束的单幅图像超分辨率算法. 自动化学报, 2014, 40(10): 2233-2244.) 14 Pan Zong-Xu, Yu Jing, Hu Shao-Xing, Sun Wei-Dong. Single image super resolution based on multi-scale structural self-similarity. Acta Automatica Sinica, 2014, 40(4): 594-603.( 潘宗序, 禹晶, 胡少兴, 孙卫东. 基于多尺度结构自相似性的单幅图像超分辨率算法. 自动化学报, 2014, 40(4): 594-603.) 15 Pan Zong-Xu, Yu Jing, Xiao Chuang-Bai, Sun Wei-Dong. Dictionary learning and structural self-similarity-based codebook mapping for single image super resolution. Journal of Computer-Aided Design & Computer Graphics, 2015, 27(6): 1032-1038.( 潘宗序, 禹晶, 肖创柏, 孙卫东. 基于字典学习与结构自相似性的码本映射超分辨率算法. 计算机辅助设计与图形学学报, 2015, 27(6): 1032-1038.) 16 Aharon M, Elad M, Bruckstein A. rmK-SVD:an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 2006, 54(11): 4311-4322. DOI:10.1109/TSP.2006.881199 17 Tropp J A, Gilbert A C. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 2007, 53(12): 4655-4666. DOI:10.1109/TIT.2007.909108 18 Zoran D, Weiss Y. From learning models of natural image patches to whole image restoration. In: Proceedings of the 2011 IEEE International Conference on Computer Vision. Barcelona, Spain:IEEE, 2011. 479-486 http://dl.acm.org/citation.cfm?id=2356307 19 Sun L B, Cho S, Wang J, Hays J. Edge-based blur kernel estimation using patch priors. In: Proceedings of the 2013 IEEE International Conference on Computational Photography. Cambridge, MA, USA:IEEE, 2013. 1-8 http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6528301 20 Glasner D, Bagon S, Irani M. Super-resolution from a single image. In: Proceedings of the 12th IEEE International Conference on Computer Vision. Kyoto, Japan:IEEE, 2009. 349-356 http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=5459271 21 Olonetsky I, Avidan S. TreeCANN-k-d tree coherence approximate nearest neighbor algorithm. In: Proceedings of the 12th European Conference on Computer Vision. Florence, Italy:Springer, 2012. 602-615 http://link.springer.com/chapter/10.1007%2F978-3-642-33765-9_43 22 Liu S G, Wang H B, Wang J, Cho S, Pan C H. Automatic blur-kernel-size estimation for motion deblurring. The Visual Computer, 2015, 31(5): 733-746. DOI:10.1007/s00371-014-0998-2 23 Perrone D, Diethelm R, Favaro P. Blind deconvolution via lower-bounded logarithmic image priors. In: Proceedings of the 10th International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition. Hong Kong, China:Springer International Publishing, 2015. 112-125 http://link.springer.com/chapter/10.1007/978-3-319-14612-6_9 24 Cho T S, Paris S, Horn B K P, Freeman W T. Blur kernel estimation using the radon transform. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA:IEEE, 2011. 241-248 http://doi.ieeecomputersociety.org/10.1109/CVPR.2011.5995479 25 Krishnan D, Tay T, Fergus R. Blind deconvolution using a normalized sparsity measure. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA:IEEE, 2011. 233-240 https://www.computer.org/csdl/proceedings/cvpr/2011/0394/00/05995521-abs.html