文章快速检索 高级检索

1. 江南大学数字媒体学院, 江苏无锡 214122;
2. 江阴职业技术学院江苏省信息融合软件工程技术研发中心, 江苏江阴 214405;
3. 江阴职业技术学院计算机科学系, 江苏江阴 214405

Transfer learning classification algorithms based on minimax probability machine
WANG Xiaochu1,2 , BAO Fang2,3, WANG Shitong1, XU Xiaolong1
1. School of Digital Media, Jiangnan University, Wuxi 214122, China;
2. Information Fusion Software Engineering Research and Development Center of Jiangsu Province, Jiangyin Pdyteehnie College, Jiangyin 214405, China;
3. Department of Computer Science, Jiangyin Pdyteehnie College, Jiangyin 214405, China
Abstract: Traditional transfer learning classification algorithms solve related (but not identical) data classification issues by using a large number of labeled samples in the source domain and small amounts of labeled samples in the target domain. However, this technique does not apply to the transfer learning of data from different categories of learned source domain data. To solve this problem, we constructed a transfer learning constraint term using the source domain data and the limited labeled data in the target domain to generate a regularized constraint for the minimax probability machine. We propose a transfer learning classification algorithm based on the minimax probability machine known as TL-MPM. Experimental results on 20 Newsgroups data sets demonstrate that the proposed algorithm has higher classification accuracy for small amounts of target domain data. Therefore, we confirm the effectiveness of the proposed algorithm.
Key words: transfer learning     minimax probability machine     classification     source domain     target domain     regularization

1 最小最大概率机 1.1 线性部分

1.2 非线性部分

γ=[α1α2αNxβ1β2βNy]， [Kx]i=K(xj, zi)为核函数，[${\tilde k}$x]i=${1 \over {{N_x}}}\sum {_{j = 1}^{{N_x}}}$ K(xj,zi)，[${\tilde k}$y]i=${1 \over {{N_y}}}\sum {_{j = 1}^{{N_y}}}$ K(yj,z i)则约束条件可表示为

2 基于最小最大概率机的迁移学习分类算法 2.1 TL-MPM算法的应用背景

2.2 TL-MPM算法的理论依据

TL-MPM算法的理论依据是：若两个领域相关，源域的数据和目标域的数据在超平面所在空间的均值应相近。通过在MPM线性目标式(3)中增加 λL，非线性目标式(5)中增加λLk实现两个域之间的迁移学习。加入迁移学习项后的线性目标函数可以写为

 图 1 TL-MPM算法 Fig. 1 Flowchart of TL-MPM

 图 2 源域样本分布 Fig. 2 Source domain samples

 图 3 二维数据模拟实验 Fig. 3 Two-dimensional simulation experiment
2.3 具体推导过程 2.3.1 线性部分

a取最优解a*时，对应k取得最小值k*，式(17)中不等式变为等式，可得

2.3.2非线性部分

3 TL-MPM算法解法流程

1)令a=a0+FvF的列正交于 ${\bar x}$t－ ${\bar y}$t

2)式(16)可写为

3)令a0=( ${\bar x}$t－ ${\bar y}$ t)/‖${\bar x}$t－ ${\bar y}$t22β0=1，δ0=1,ε0=1并带入(18)得到最小二乘问题，求解得v0;

4)k=1;

5)ak=ak－1+Fvk－1βk=$\sqrt {a_{k - 1}^T\sum {_x} {a_{k - 1}}}$，δk= λ$\sqrt {a_{k - 1}^T\sum {_y} {a_{k - 1}}}$，εk=$\sqrt {a_{k - 1}^TD{a_{k - 1}}}$并带入(18)式求解得vk;

6)k=k+1;

7)重复步骤5、6直到收敛或满足停止条件;

8)最后求得：a*=akk*=1/(βk+δk+εk), b*=a*T${\bar x}$－k*(β<sub>k+ηεk)。

4 实验结果与分析

 大类 小类 样本个数 comp comp.graphics 997 comp.windows.x 998 comp.os.mswindows.misc 992 comp.sys.ibm.pc.hardware 997 comp.sys.mac.hardware 996 rec rec.motorcycles 997 rec.autos 998 rec.sport.baseball 998 rec.sport.hockey 998 talk talk.politics.mideast 1 000 talk.politics.misc 998 talk.politics.guns 1 000 talk.religion.misc 999 sci sci.crypt 998 sci.med 998 sci.space 999 sci.electronics 999

 符号 定义 Ds 源域数据集 Dt 目标域数据集 T-LMPM 仅利用目标域少量数据训练得到的目标域MPM线性分类器 T-KMPM 仅利用目标域少量数据训练得到的目标域MPM非线性分类器 LTL-MPM 本文提出的TL-MPM算法得到的线性分类器 KTL-MPM 本文提出的TL-MPM算法得到的非线性分类器
4.1 20News Groups数据集预处理

 Datasets Ds Dt comp vs rec comp.graphics rec.motorcycles comp.windows.x rec.autos comp vs sci comp.os.mswindows.misc sci.crypt comp.sys.ibm.pc.hardware sci.med comp vs talk comp.sys.mac.hardware talk.politics.mideast comp.os.mswindows.misc talk.politics.guns rec vs talk rec.autos talk.politics.misc rec.sport.baseball talk.religion.misc rec vs sci rec.autos sci.space rec.sport.hockey sci.electronics sci vs talk sci.med talk.religion.misc sci.space talk.politics.mideast

4.2 少量目标域训练样本上的实验结果与分析

 数据集 T-LMPM T-KMPM LTL-MPM KTL-MPM comp vs rec 71.71 72.96 72.56 78.42 comp vs sci 66.46 74.16 72.22 76.49 comp vs talk 90.96 99.15 99.48 99.75 rec vs talk 60.51 71.29 66.26 76.14 rec vs sci 62.49 71.00 64.17 76.48 sci vs talk 68.21 77.24 67.98 76.44

 数据集 T-LMPM T-KMPM LTL-MPM KTL-MPM comp vs rec 61.46 80.92 64.19 83.94 comp vs sci 58.72 76.33 63.29 79.56 comp vs talk 94.13 99.41 97.74 99.83 rec vs talk 57.36 76.30 60.80 78.60 rec vs sci 57.12 78.81 57.49 81.50 sci vs talk 57.42 86.34 63.98 86.75

 数据集 T-LMPM T-KMPM LTL-MPM KTL-MPM comp vs rec 76.46 85.51 80.05 86.06 comp vs sci 70.87 84.75 75.00 83.70 comp vs talk 96.16 99.32 98.66 99.69 rec vs talk 65.30 81.64 66.74 81.56 rec vs sci 64.93 84.89 70.17 85.90 sci vs talk 70.49 90.18 72.17 89.09

4.3 目标域训练样本变化对分类结果的影响

 图 4 comp vs rec分类精度变化 Fig. 4 Classification accuracy trends of comp vs rec

 图 5 comp vs sci分类精度变化 Fig. 5 Classification accuracy trends of comp vs sci

 图 6 comp vs talk分类精度变化 Fig. 6 Classification accuracy trends of comp vs talk

 图 7 rec vs talk分类精度变化 Fig. 7 Classification accuracy trends of rec vs talk

 图 8 rec vs sci分类精度变化 Fig. 8 Classification accuracy trends of rec vs sci

 图 9 sci vs talk分类精度变化 Fig. 9 Classification accuracy trends of sci vs talk

5 结束语

 [1] PAN S J, YANG Qiang. A survey on transfer learning[J]. IEEE transactions on knowledge and data engineering, 2010, 22(10):1345-1359. [2] DAI Wenyuan, YANG Qiang, XUE Guirong, et al. Boosting for transfer learning[C]//Proceedings of the 24th International Conference on Machine learning. New York, NY, USA, 2007:193-200. [3] PAN S J, KWOK J T, YANG Qiang. Transfer learning via dimensionality reduction[C]//Proceedings of the 23rd National Conference on Artificial Intelligence. AAAI, 2008, 2:677-682. [4] 张景祥, 王士同, 邓赵红, 等. 具有协同约束的共生迁移学习算法研究[J]. 电子学报, 2014, 42(3):556-560. ZHENG Jingxiang, WANG Shitong, DENG Zhaohong, et al. Symbiosis transfer learning method with collaborative constraints[J]. Acta electronica sinica, 2014, 42(3):556-560. [5] 于重重, 吴子珺, 谭励, 等. 非平衡集成迁移学习模型及其在桥梁结构健康监测中的应用[J]. 智能系统学报, 2013, 8(1):46-51. YU Chongchong, WU Zijun, TAN Li, et al. Unbalanced integrated transfer learning model and its application to bridge structural health monitoring[J]. CAAI transactions on intelligent systems, 2013, 8(1):46-51. [6] 毕安琪, 王士同. 基于SVC和SVR约束组合的迁移学习分类算法[J]. 控制与决策, 2014, 29(6):1021-1026. BI Anqi, WANG Shitong. Transfer classification learning based on combination of both SVC and SVR's constraints[J]. Control and decision, 2014, 29(6):1021-1026. [7] 蒋亦樟, 邓赵红, 王士同. ML型迁移学习模糊系统[J]. 自动化学报, 2012, 38(9):1393-1409.JIANG Yizhang, DENG Zhaohong, WANG Shitong. Mamdani-larsen type transfer learning fuzzy system[J]. Acta automatica sinica, 2012, 38(9):1393-1409. [8] LONG Mingsheng, WANG Jianmin, DING Guiguang, et al. Adaptation regularization:a general framework for transfer learning[J]. IEEE transactions on knowledge and data engineering, 2014, 26(5):1076-1089. [9] LONG Mingsheng, WANG Jianmin, DING Guiguang, et al. Transfer learning with graph co-regularization[J]. IEEE transactions on knowledge and data engineering, 2014, 26(7):1805-1818. [10] PERLICH C, DALESSANDRO B, RAEDER T, et al. Machine learning for targeted display advertising:transfer learning in action[J]. Machine learning, 2014, 95(1):103-127. [11] LANCKRIET G R G, EL GHAOUI L, BHATTACHARYYA C, et al. A robust minimax approach to classification[J]. The journal of machine learning research, 2003(3):555-582. [12] ISII K. On the sharpness of Chebyshev-type inequalities[J]. Annals of the institute of statistical mathematics, 1963(14):185-197. [13] BERTSIMAS D, SETHURAMAN J. Moment problems and semidefinite optimization[M]//WOLKOWICZ H, SAIGAL R, VANDENBERGHE L. Handbook of Semidefinite Programming. 3rd ed. US:Springer, 2000:469-509. [14] CHERNOFF H. The selection of effective attributes for deciding between hypotheses using linear Discriminant functions[C]//The Proceedings of the International Conference on Frontiers of Pattern Recognition. New York, USA, 1972:55-60. [15] 陈宝林. 最优化理论与算法[M]. 2版. 北京:清华大学出版社, 2005. CHEN Baolin. Optimization theory and algorithm[M]. 2nd ed. Beijing:Tsinghua University Press, 2005. [16] PLAN Y, VERSHYNIN R. Robust 1-bit compressed sensing and sparse logistic regression:a convex programming approach[J]. IEEE transactions on information theory, 2012, 59(1):482-494. [17] BENSOUSSAN A, TEMAM R. Equations aux derivees partielles stochastiques non iineaires[J]. Israel journal of mathematics,1972,11(1):95-129. [18] BOYD S, VANDENBERGHE L, FAYBUSOVICH L. Convex optimization[J]. IEEE transactions on automatic control, 2006, 51(11):1859.
DOI: 10.11992/tis.201505024

0

#### 文章信息

WANG Xiaochu, BAO Fang, WANG Shitong, XU Xiaolong

Transfer learning classification algorithms based on minimax probability machine

CAAI Transactions on Intelligent Systems, 2016, 11(01): 84-92.
DOI: 10.11992/tis.201505024