«上一篇
 文章快速检索 高级检索

 智能系统学报  2019, Vol. 14 Issue (5): 922-928  DOI: 10.11992/tis.201805045 0

### 引用本文

WANG Jian, WU Xisheng. Image fusion based on the improved sparse representation and PCNN[J]. CAAI Transactions on Intelligent Systems, 2019, 14(5): 922-928. DOI: 10.11992/tis.201805045.

### 文章历史

Image fusion based on the improved sparse representation and PCNN
WANG Jian , WU Xisheng
School of Internet of Things Engineering, Jiangnan University, Wuxi 214122, China
Abstract: To improve the clarity of image fusion, in this paper, we propose an image-fusion algorithm based on improved sparse representation and a pulse-coupled neural network (PCNN). First, using a non-subsampled shearlet transform (NSST), source images are decomposed into low-frequency and high-frequency sub-band coefficients, which contain different information. Then, we use the K-singular value decomposition algorithm to fuse the improved sparse representation with low-frequency sub-band coefficients and construct a joint dictionary from the adaptive learning multiple sub-dictionaries in the source images. The high-frequency sub-band coefficients are fused with the improved PCNN. To stimulate the PCNN model, we use the modified spatial frequency as neuron feedback input. The high-frequency coefficients are selected according to the fusion rule for the maximum amplitude of fire output. Finally, we reconstruct the fused image with the NSST inverse transform of the fused low-frequency and high-frequency sub-band coefficients. The experimental results show that the proposed algorithm preserves the edge information of the source images very well; additionally, the fused image achieves good results on the evaluation criteria, thus verifying the effectiveness of the proposed algorithm.
Key words: image processing    image fusion    NSST    sparse representation    adaptive learning dictionary    joint dictionary    PCNN    improved spatial frequency

1 基本原理 1.1 图像的NSST变换

 ${M_{{{{AB}}}}}\left( {\psi } \right){\rm{ = \{ }}{{\psi }_{{j},{l},{k}}}\left( {x} \right){\rm{ = }}{\left| {{\rm{det}}\left( {{{A}}} \right)} \right|^{\frac{{\rm{j}}}{{\rm{2}}}}}{\psi }\left( {{{{B}}^{{l}}}{{{A}}^{j}}{x}{\rm{ - }}{k}} \right):{j},{l} \in {{\bf{R}}}{\rm{;}}{k} \in {{{\bf{R}}}^{\rm{2}}}{\rm{\} }}$ (1)

 ${{{A}}_0} = \left[ {\begin{array}{*{20}{l}} 4&0\\ 0&2 \end{array}} \right],\quad {{{B}}_0} = \left[ {\begin{array}{*{20}{l}} 1&1\\ 0&1 \end{array}} \right]$

NSST由非下采样金字塔分解(non-subsample pyramid，NSP)和剪切滤波器组成，图像经过k级NSP分解之后，会分解成由k个高频子带和1个低频子带组成的子图像。分解后的k+1个子带与源图像大小相同，但是尺度不同。3级NSST分解过程如图1所示。

 Download: 图 1 3级NSST分解过程示意 Fig. 1 Diagram of three-level NSST decomposition process
1.2 稀疏表示

 ${{X}} = \sum\limits_{i = 1}^k {{a}} (i){{\phi}} (j)$ (2)

1.3 PCNN模型

PCNN由多个神经元构成，每个神经元由3个部分组成，分别是连接域、调制域和脉冲产生器。PCNN神经元对应着源图像中的每个像素，每个神经元的点火输出有点火和不点火2种状态。PCNN简化模型如图2所示。

 Download: 图 2 PCNN单个神经元简化模型 Fig. 2 Simple model of single neuron in PCNN

 \left\{ \begin{aligned} & {{F}_{{ij}}}{\rm{(}}{n}{\rm{) = }}{{I}_{{ij}}}\\ & {{L}_{{ij}}}{\rm{(}}{n}{\rm{) = exp}}\left( {{\rm{ - }}{{\alpha }_{L}}} \right){{L}_{{ij}}}{{(n - 1) + }}{{V}_{L}}\mathop \sum \limits_{{kl}} {{W}_{{ijkl}}}{{Y}_{{ij}}}{\rm{(}}{n}{\rm{ - 1)}}\\ & {{U}_{{ij}}}{\rm{(}}{n}{\rm{) = }}{{F}_{{ij}}}{\rm{(}}{n}{\rm{)}}\left( {{\rm{1 + }}{\beta }{{L}_{{ij}}}{\rm{(}}{n}{\rm{)}}} \right)\\ & {{\theta }_{{ij}}}{\rm{(}}{n}{\rm{) = exp}}\left( {{\rm{ - }}{{\alpha }_{\theta }}} \right){{\theta }_{{ij}}}{{(n - 1)}}{{V}_{\theta }}{{Y}_{{ij}}}{\rm{(}}{n}{\rm{ - 1)}}\\ & {{Y}_{{ij}}}{\rm{(}}{n}{\rm{) = }}\left\{ \begin{array}{l} \begin{array}{*{20}{c}} \!\!\!\!\! {\rm{1}},&{{{U}_{{ij}}}{\rm{(}}{n}{\rm{)> }}{{\theta }_{{ij}}}{\rm{(}}{n}{\rm{)}}} \end{array}\\ \begin{array}{*{20}{c}} \!\!\!\!\! {\rm{0}},&\text{其他} \end{array} \end{array} \right. \end{aligned} \right. (3)

2 改进的融合算法

 Download: 图 3 基于改进的稀疏表示和PCNN融合流程图 Fig. 3 Flowchart of fusion based on the improved sparse presentation and PCNN
2.1 低频子带的融合规则

 $\begin{array}{l} {{\widetilde {{V}}}_{\rm{A}}} = {{{V}}_{\rm{A}}} - {{\overline {{V}}}_{\rm{A}}}\\ {{\widetilde {{V}}}_{\rm{B}}} = {{{V}}_{\rm{B}}} - {{\overline {{V}}}_{\rm{B}}} \end{array}$

 ${{D}}= \left[ {{{{D}}_{A}}\;\;{{{D}}_B}} \right]$

 ${{M}}_{\rm{A}}^i = {\left( {{{\left\| {{{a}}_{\rm{A}}^i} \right\|}_1}} \right)^{{\omega _t}}} \times {\left( {{{\left\| {{{a}}_{\rm{A}}^i} \right\|}_0}} \right)^{{\omega _O}}}$ (4)
 ${{M}}_{\rm{B}}^i = {\left( {{{\left\| {{{a}}_{\rm{B}}^i} \right\|}_1}} \right)^{{\omega _t}}} \times {\left( {{{\left\| {{{a}}_{\rm{B}}^i} \right\|}_0}} \right)^{{\omega _O}}}$ (5)

 ${\alpha }_{F}^{i}{\rm{ = }}\left\{ \begin{array}{l} \begin{array}{*{20}{c}} \!\!\!\! {{\alpha }_{{\rm{A}}}^{i}},&{{{{M}}}_{{\rm{A}}}^{i} > {{{M}}}_{{\rm{B}}}^{i}} \end{array}\\ \begin{array}{*{20}{c}} \!\!\!\! {{{\alpha}_{{\rm{B}}}^{i}}},&\text{其他} \end{array} \end{array} \right.$

 ${{V}}_F^i = Da_F^i$

 ${{\overline V}_F}{\rm{ = }}\left\{ \begin{array}{l} {{\overline V}_A},\;\;\;\;{M_A} > {M_B}\\ {{\overline V}_B},\;\;\;\;\text{其他} \end{array} \right.$

 ${V_F} = V_F^i + {{\overline V}_F}$

2.2 高频子带的融合规则

 ${M_{{\rm{SF}}}} = \sqrt {{R_F}^2 + {C_F}^2 + {D_F}^2}$ (6)

 ${{R}_{F}} = \sqrt {\frac{{\rm{1}}}{{\left( {2M{\rm{ + 1}}} \right)\left( {2N{\rm{ + 1}}} \right)}}\mathop \sum \limits_{{m} = - M}^M \mathop \sum \limits_{{n} = - N}^N {{{\rm{[}}{X}\left( {{i} + {m},{j} + {n}} \right) - {X}\left( {{i} + {m},{j} + {n}{\rm{ - 1}}} \right){\rm{]}}}^{\rm{2}}}}$ (7)
 ${C_F} = \sqrt {\frac{{\rm{1}}}{{\left( {2M + 1} \right)\left( {2N + 1} \right)}}\mathop \sum \limits_{m = - M}^M \sum\limits_{n = - N}^N {{{[X\left( {i + m,j + n} \right) - X\left( {i + m,j + n} \right)]}^{\rm{2}}}} }$ (8)
 $\begin{split} {D_F}{\rm{ = }} & \sqrt {\frac{{\rm{1}}}{{\left( {2M} \right)\left( {2N} \right)}}\mathop \sum \limits_{m = - M + 1}^M \mathop \sum \limits_{n = - N + 1}^N {{{\rm{[}}X\left( {i + m,j + n} \right) - X\left( {i + m - 1,j + n - 1} \right){\rm{]}}}^{\rm{2}}}} + \\ & \sqrt {\frac{{\rm{1}}}{{\left( {2M} \right)\left( {2N} \right)}}\mathop \sum \limits_{m = - M}^{M - 1} \mathop \sum \limits_{n = - N }^{N-1} {{{\rm{[}}X\left( {i + m - 1,j + n} \right) - X\left( {i + m,j + n - 1} \right){\rm{]}}}^{\rm{2}}}} \end{split}$ (9)

 ${{X}_{{ij}}}{\rm{(}}{n}{\rm{) = }}\frac{{\rm{1}}}{{{\rm{1 + exp[}}{{\theta }_{{ij}}}\left( {n} \right){\rm{ - }}{{U}_{{ij}}}{\rm{(}}{n}{\rm{)]}}}}$ (10)
 $M_{i j}(n)=M_{i j}(n-1)+X_{i j}(n)$ (11)

 ${{I}_{{F}{\rm{,}}{ij}}}{\rm{ = }}\left\{ \begin{array}{l} \begin{array}{*{20}{c}} \!\!\!\! {{{I}_{{\rm{1,}}{ij}}}},&{{{M}_{{\rm{1,}}{ij}}}{\rm{ > }}{{M}_{{\rm{2,}}{ij}}}} \end{array}\\ \begin{array}{*{20}{c}} \!\!\!\! {{{I}_{{\rm{2,}}{ij}}}},&\text{其他} \end{array} \end{array} \right.$ (12)

3 实验结果与分析

 Download: 图 5 各种算法Pepsi图像融合结果 Fig. 5 Pepsi image fusion of each algorithm
 Download: 图 6 各种算法Clock图像融合结果 Fig. 6 Clock image fusion of each algorithm