首页 关于本刊 编 委 会 期刊动态 作者中心 审者中心 读者中心 下载中心 联系我们 English
 自动化学报  2018, Vol. 44 Issue (5): 819-828 PDF

1. 大连理工大学计算机科学与技术学院 大连 116023

Multi-view Learning and Reconstruction Algorithms via Generative Adversarial Networks
SUN Liang1, HAN Yu-Xuan1, KANG Wen-Jing1, GE Hong-Wei1
1. College of Computer Science and Technology, Dalian University of Technology, Dalian 116023
Manuscript received : September 8, 2017, accepted: December 23, 2017.
Foundation Item: Supported by National Natural Science Foundation of China (61402076, 61572104, 61103146), Project of Key Laboratory of Symbolic Computation and Knowledge Engineering of Jilin University (93K172017K03), and Fundamental Research Funds for Central Universities (DUT17JC04)
Corresponding author. GE Hong-Wei  Associate professor at the College of Computer Science and Technology, Dalian University of Technology. He received his Ph. D. degree in computer application technology from Jilin University in 2006. His research interest covers computational intelligence, machine learning, system modeling and optimization. Corresponding author of this paper
Recommended by Associate Editor WANG Kun-Feng
Abstract: Generally, objects often require to represent in different views. However, real-world applications in complex scenarios can hardly have complete views of a given object. In this paper, we propose generative adversarial network (GAN) based multi-view learning and reconstruction algorithms. A novel representation learning algorithm is proposed, which guarantees different views of the same object are mapped to the same representation. Meanwhile, the algorithm guarantees the representation carries enough reconstructed information. To construct multi-views of a given object, a generative adversarial network based reconstruction algorithm is proposed, which includes the representation information in the generation and discrimination models to guarantee the constructed views perfectly map the source view. The merits of the proposed algorithms lie in the fact that they avoid direct mapping among different views, and can solve the problem of missing views in training data and the problem of mapping between constructed views and the source views. Simulated experiments on handwritten digit dataset (MNIST), street view house numbers dataset (SVHN) and CelebFaces attributes dataset (CelebA) indicate that the proposed algorithms yield satisfactory reconstruction performances.
Key words: Multi-view reconstruction     conditional generative adversarial networks (CGAN)     multi-view representation learning     generative models

1 问题描述

2 基于DNN的多视图表征学习

 \begin{align} H(x)=\sum-P(x)\log{P(x)}= {\rm E}[-\log{P(x)}] \end{align} (2)

 $I(x;c)=H(x)-H(x|c)=H(c)-H(c|x)$ (3)
 图 2 原始视图数据$x$, 表征向量$\pmb c$, 重构视图数据$\hat{x}$间的互信息示意图 Figure 2 Schematic diagram of mutual information among original view data $x$, representative vector $\pmb c$, reconstructed data $\hat{x}$

 \begin{align} SSIM(I_x, I_y)=\frac{(2\mu_x\mu_y+c_1)(2\sigma_{xy}+c_2)}{(\mu_x^2+\mu_y^2+c_1)(\sigma_x^2+\sigma_y^2+c_2)} \end{align} (7)

PSNR是一种评价图像的客观标准.图像经过处理之后, 输出的图像都会在某种程度与原始图像不同.将真实图像与生成图像对比, 得到生成的图像的PSNR值来测试模型的重构效果.

 $MSE = \dfrac{\sum\limits_{n=1}^{FrameSize}(I_n - P_n)^2}{FrameSize}$ (8)
 $PSNR = 10\times\log \frac{255^2}{MSE}$ (9)

4.3 实验设置与结果 4.3.1 MNIST数据集实验结果

 图 4 MNIST视图3数据经过PCA后的可视化二维图 Figure 4 The 2D-visualization of view 3 on MNIST after PCA

 图 5 以视图2为源数据在MNIST上的重构结果 Figure 5 Reconstruction results that take view 2 as source data on MNIST
 图 6 以视图3为源数据在MNIST上的重构结果 Figure 6 Reconstruction results that take view 3 as source data on MNIST

4.3.2 SVHN数据集实验结果

 图 7 以视图2为源数据在SVHN上的重构结果 Figure 7 Reconstruction results that take view 2 as source data on SVHN
 图 8 以视图3为源数据在SVHN上的重构结果 Figure 8 Reconstruction results that take view 3 as source data on SVHN

4.3.3 CelebA数据集实验结果

 图 9 以视图2为源数据在CelebA上的重构结果 Figure 9 Reconstruction results that take view 2 and view 3 as source data respectively on CelebA

5 结论