舰船科学技术  2019, Vol. 41 Issue (6): 115-119 PDF

Error concealment algorithm based on correlation motion extrapolation for video communication
HUANG Zhi-hua, CAI Quan-wang
No.722 Research Institute of CSIC, Wuhan 430079, China
Abstract: To overcome the dramatic degradation of video quality caused by transmission errors in video communication, a new temporal error concealment algorithm based on motion extrapolation is proposed. For a damaged image block, the preceding image block and the preceding adjacent image blocks are projected into the three-dimensional coordinate system, and the first-order planar set of the preceding adjacent image block is constructed. Then the distances between the preceding image block and the first-order planar set are calculated, and the image blocks represented by the shortest distance first-order plane are selected as the strongest preceding adjacent image blocks. Finally, the strongest correlation preceding adjacent image blocks are extrapolated to the damaged image frame and the motion vector of the damaged image block is estimated by weighting the overlap area between the extrapolated image blocks and the damaged image block, and then the damaged image block is restored. The simulation results show that for different video sequences, the PSNR value of the proposed algorithm is improved by 0.5～2.7 dB compared with the traditional temporal error concealment algorithm, and the decoded video quality also achieves better subjective visual effect.
Key words: video communication     error concealment     motion vector     motion extrapolation     boundary matching
0 引　言

1 相关最强前续相邻图像块确定

 图 1 图像块位置关系示意图 Fig. 1 Position relation of image blocks

$B_0'$ 中心点为原点， $B_1'$ $B_8'$ 的中心点与原点的横向距离为 $X$ 轴、 $B_1'$ $B_8'$ 的中心点与原点的纵向距离为 $Y$ 轴、 $B_1'$ $B_8'$ 运动矢量的水平分量为 $Z$ 轴建立三维坐标系。前续相邻图像块横向距离和纵向距离如表1所示。

$B_1'$ $B_8'$ 映射到此坐标系中，得到坐标点 $B_1'(4,0,x_1')$ $B_2'(4,4,x_2')$ $B_3'(0,4,x_3')$ $B_4'( - 4,4,x_4')$ $B_5'( - 4,0,x_5')$ $B_6'( - 4, - 4,x_6')$ $B_7'(0, - 4,x_7')$ $B_8'(4, - 4,x_8')$ ，坐标点可统一标记为 $B_i'(d{s_{x,i}},d{s_{y,i}},x_i')$ ，其中 $d{s_{x,i}}$ $d{s_{y,i}}$ $x_i'$ 分别为 $B_i'$ 的横向距离、纵向距离和运动矢量水平分量。

$B_1'(4,0,x_1')$ $B_8'(4, - 4,x_8')$ 的任意三点相连，得到前续相邻图像块的1阶平面集，记为：

 $S = \{ B_i'B_j'B_k'|i > j > k,(i,j,k) \in \{ 1,2,3,4,5,6,7,8\} \} \text{。}$ (1)

 $ax + by + cz + d = 0\text{。}$ (2)

$B_i'$ $B_j'$ $B_k'$ 的坐标代入式（2）得到以下方程组：

 $\left\{ \begin{gathered} a \times d{s_{x,i}} + b \times d{s_{y,i}} + c \times x_i' + d = 0 \text{，}\\ a \times d{s_{x,j}} + b \times d{s_{y,j}} + c \times x_j' + d = 0 \text{，}\\ a \times d{s_{x,k}} + b \times d{s_{y,k}} + c \times x_k' + d = 0 \text{。}\\ \end{gathered} \right.$ (3)

 \left\{ \begin{aligned} & a = (d{s_{y,j}} - d{s_{y,i}}) \times (x_k' - x_i') - (d{s_{y,k}} - d{s_{y,i}}) \times (x_j' - x_i') \text{，}\\ & b = (d{s_{x,k}} - d{s_{x,i}}) \times (x_j' - x_i') - (d{s_{x,j}} - d{s_{x,i}}) \times (x_k' - x_i') \!\!\! \text{，}\\ & c = (d{s_{x,j}} - d{s_{x,i}}) \times (d{s_{y,k}} - d{s_{y,i}}) - (d{s_{x,k}} - d{s_{x,i}}) \times \\ & \quad(d{s_{y,j}} - d{s_{y,i}}) \text{，}\\ &d = - (a \times d{s_{x,i}} + b \times d{s_{y,i}} + c \times x_i')\text{。} \end{aligned} \right. (4)

 图 2 前续图像块与1阶平面距离 Fig. 2 Distance between preceding image blocks and first-order plane

$B_0'(0,0,x_0')$ $S$ 中一阶平面（以下以一阶平面 $B_i'B_j'B_k'$ 为例描述）的距离计算公式为：

 ${\rm {dist}}(B_0',B_i'B_j'B_k') = \frac{{\left| {c \times x_0' + d} \right|}}{{\sqrt {{a^2} + {b^2} + {c^2}} }}\text{。}$ (5)

 $(B_\alpha ',B_\beta ',B_\delta ') = \mathop {\min \arg }\limits_{i,j,k \in \{ 1\sim 8\} ,i > j > k} \{ dist(B_0',B_i'B_j'B_k')\}\text{。}$ (6)

2 相关运动矢量外推的错误掩盖 2.1 运动矢量外推

 图 3 运动矢量外推示意图 Fig. 3 Diagram of motion vector extrapolation

 $\!\left\{ \begin{gathered} m{v_x} = \frac{{S(B_\alpha ',{B_0}) \times x_\alpha ' + S(B_\beta ',{B_0}) \times x_\beta ' + S(B_\gamma ',{B_0}) \times x_\gamma '}}{{S(B_\alpha ',{B_0}) + S(B_\beta ',{B_0}) + S(B_\gamma ',{B_0})}}\text{，}\!\!\!\!\!\!\!\!\!\!\! \\ m{v_y} = \frac{{S(B_\alpha ',{B_0}) \times y_\alpha ' + S(B_\beta ',{B_0}) \times y_\beta ' + S(B_\gamma ',{B_0}) \times y_\gamma '}}{{S(B_\alpha ',{B_0}) + S(B_\beta ',{B_0}) + S(B_\gamma ',{B_0})}}\text{。}\!\!\!\!\!\!\!\!\!\!\! \\ \end{gathered} \right.$ (7)

2.2 运动矢量优化

 图 4 最优运动矢量查找过程示意图 Fig. 4 Searching process for optimal motion vector

 $\begin{split} dif{f_{(\Delta x,\Delta y)}} = & \sum\limits_{m = 0}^3 \left| pi{x_n}({x_{start}} + m,{y_{start}} - 1)-\right. \\ & \left. pi{x_{n - 1}}({x_{start}} - \Delta x + m,{y_{start}} - \Delta y) \right| {\rm{ + }} \\ & \sum\limits_{m = 0}^3 \left| pi{x_n}({x_{start}} - 1,{y_{start}} + m)-\right. \\ & \left. pi{x_{n - 1}}({x_{start}} - \Delta x,{y_{start}} - \Delta y + m) \right| \text{。}\\ \end{split}$ (8)

 $({x_0},{y_0}) = (\Delta x,\Delta y) = \mathop {\arg \min }\limits_{\begin{array}{*{20}{c}} {\Delta x \in [m{v_x} - \delta ,m{v_x} + \delta ]}\\ {\Delta y \in [m{v_y} - \delta ,m{v_y} + \delta ]} \end{array}\text{。}} dif{f_{(\Delta x,\Delta y)}}\text{，}$ (9)
3 仿真结果

 图 5 各算法解码视频主观视觉效果 Fig. 5 Decoded visual effects of various algorithms

4 结　语

 [1] ANDREAS F.M. 无线通信[M]. 北京: 电子工业出版社, 2015. [2] 高文, 赵德斌, 马思伟. 数字视频编码技术基础[M]. 北京: 科学出版社, 2018. [3] 彭强, 张蕾. 视频传输差错掩盖技术研究现状与发展趋势[J]. 西南交通大学学报, 2009, 44(4): 473-483. DOI:10.3969/j.issn.0258-2724.2009.04.001 [4] 陈健. 基于H.264的视频差错掩盖算法研究[D]. 成都: 西南交通大学, 2016: 6–7. [5] ZHENG J. H, CHAU L. P. A Motion vector recovery algorithm for digital video using lagrange interpolation[J]. IEEE Transactions on Broadcasting, 2008, 49(4): 383-389. [6] 姚伟新, 杨丹, 葛愿. H.264/AVC自适应时空域错误隐藏算法[J]. 计算机工程, 2016, 42(2): 261-265. DOI:10.3969/j.issn.1000-3428.2016.02.046 [7] YANG S. H, CHANG C, W, CHAN C. C. An Object based error concealment technique for H.264 Coded Video[J]. Multimedia Tools and Applications, 2015, 74(23): 10785-10800. DOI:10.1007/s11042-014-2206-9 [8] RADMEHR A, GHASEMI A. Error concealment via particle filter by gaussian mixture modeling of motion vectors for H.264/AVC[J]. Signal, Image and Video Processing, 2016, 10(2): 311-318. DOI:10.1007/s11760-014-0743-3 [9] [10] H.264/AVC Reference Software JM18.0. Available: http://iphome.hhi.de/suehring/tml/download.