文章快速检索 高级检索

1. 辽宁工程技术大学 理学院, 辽宁 阜新 123000;
2. 辽宁工程技术大学 矿业学院, 辽宁 阜新 123000

Reducing training times in neural network classifiers by using dynamic data reduction
LIU Wei1, LIU Shang1, BAI Runcai2, ZHOU Xuan1, ZHOU Dingning1
1. College of Science, Liaoning Technical University, Fuxin 123000, China;
2. Mining Institute, Liaoning Technical University, Fuxin 123000, China
Abstract: In this paper, we present a neural network classifier training method based on dynamic data reduction (DDR) to address long training times and the poor generalization ability of neural network classifiers. In our approach, we assigned each sample a weight value, which was then dynamically updated based on the classification error rate at each iteration of the training sample. Subsequently, the training sample was reduced based on the weight of the sample so as to increase the proportion of boundary samples in error-prone classification environments and to reduce the role of redundant kernel samples. Our numerical experiments show that our neural network training method not only substantially shortens the training time of the given networks, but also significantly enhances the classification and generalization abilities of the network.
Key words: neural network     data reduction     classification boundary     sample weight     boundary sample     kernel sample

1 DDR训练方法 1.1 BP神经网络

BP (back propagation) 神经网络是一种单向传播的多层前馈网络，采用误差反向传播权值学习算法 (BP算法)，是目前应用较多的一种模型。BP神经网络的基本单元是神经元，按照神经元的功能不同将其分成若干层，通常最左侧的为输入层，最右侧的为输出层，而中间的为隐层，只有相邻层神经元之间存在权值连接，每层内部神经元无连接，其结构如图 1所示。

 图 1 BP神经网络结构 Fig. 1 BP neural network structure

BP神经网络的信息传递过程主要分为两个阶段：信息前馈传递阶段和误差反馈阶段。信息前馈阶段，每层的输入信息，首先通过连接权值进行融合计算，再通过相应类型的激活函数进行激活变换得到输出信号，然后将输出信号作为输入传入下一层进行相似的信息变换，最终传递到输出层得到网络最终输出。误差反馈阶段，由于神经网络是一种监督学习算法，将信号的前馈输出和真实标签之间的误差，通过连接权值从输出层反向传播至输入层，并依据梯度值来更新连接权值，从而达到学习的目的。

1.2 DDR训练方法设计思想

1.3 DDR训练方法算法描述

1) 初始化网络结构，随机初始化网络权值；

2) 训练样本规则化预处理；

3) 对当前训练样本Xreduction进行随机乱序操作，重新排列样本的顺序；

4) 按照训练样本排列序号，依次提取批量s个样本，样本分成n个批次，n=round (m/s)。

5) 计算网络各批量的均值误差

6) 子批量内均值修正网络的权值：

7) 计算所有样本的均值误差：

8) 依据分类错误率更新样本权重值：

9) 样本权重约束

10) 样本约简选择

11) 根据迭代次数进行判断是否达到收敛要求，若达到要求则网络完成训练，否则循环3)~11)。

 图 2 动态数据约简神经网络训练方法流程图 Fig. 2 Flow chart of neural network training method for dynamic data reduction
2 实验分析 2.1 实验参数设置

ChangeIndex =T*scaleIndex

FOR k=1:K

IF k>ChangeIndex&&curTimes < ChangeTimes

ChangeIndex =k+scaleIndex*(K-k)

curLr =curLr*scaleLr

curTimes =curTimes+1

2.2 人工数据可视化分析

 图 5 神经网络分类器边界图 Fig. 5 Neural network classifier boundary map

 图 3 训练样本权重分布图 Fig. 3 Training sample weight distribution graph

 图 4 选择的训练样本分布图 Fig. 4 Selected training sample distribution map

2.3 标准数据集实验分析

 名称 样本个数 训练样本 属性个数 类别数 Forest 523 198 27 4 Glass 214 100 9 6 IP 180 90 34 2 Iris 150 75 4 3 IS 2 310 210 19 7 LIR 20 000 10 000 61 10 Seeds 210 105 7 3 SL 6 435 4 000 36 6 Wine 178 90 13 3 Mnist 60 000 10 000 784 10

 数据集名称 Method loss train-Avg test-Avg time Forest STD 0.007 5 0.07 15.66 5.34 DDR 0.011 6 0.00 15.79 2.11 Glass STD 0.036 6 4.47 35.37 7.70 DDR 0.050 4 1.63 33.48 3.29 IP STD 0.004 4 0.30 30.04 1.24 DDR 0.010 8 0.00 29.96 0.41 Iris STD 0.025 8 3.07 4.22 0.67 DDR 0.058 5 1.87 3.69 0.36 IS STD 0.027 4 3.51 10.14 2.52 DDR 0.032 6 1.14 8.72 1.53 LIR STD 0.114 1 12.92 14.41 122.33 DDR 0.144 6 8.05 11.01 77.80 SL STD 0.058 2 7.06 9.62 78.14 DDR 0.065 2 5.80 9.78 34.15 Seeds STD 0.034 0 3.40 6.44 1.42 DDR 0.013 2 0.13 4.98 3.73 Wine STD 0.001 0 0.04 2.65 0.71 DDR 0.001 7 0.00 2.50 0.49 Mnist STD 0.004 5 0.10 1.51 2 104.83 DDR 0.004 7 0.03 1.61 1 129.18

3 结论与展望

 [1] 毛勇. 基于支持向量机的特征选择方法的研究与应用[D]. 杭州: 浙江大学, 2006. MAO Yong. A study on feature selection algorithms based on support vector machine and its application[D]. Hangzhou: Zhejiang University, 2006. [2] 覃政仁, 吴渝, 王国胤. 一种基于Rough Set的海量数据分割算法[J]. 模式识别与人工智能, 2006, 19(2): 249-256. QIN Zhengren, WU Yu, WANG Guoyin. A partition algorithm for huge data sets based on rough set[J]. Pattern recognition and artificial intelligence, 2006, 19(2): 249-256. [3] ABDI H, WILLIAMS L J. Principal component analysis[J]. Wiley interdisciplinary reviews: computational statistics, 2010, 2(4): 433-459. DOI:10.1002/wics.v2:4. [4] RIFAI S, VINCENT P, MULLER X, et al. Contractive auto-encoders: explicit invariance during feature extraction[C]//Proceedings of the 28th International Conference on Machine Learning. Bellevue, WA, USA: ICML, 2011. [5] 周玉, 朱安福, 周林, 等. 一种神经网络分类器样本数据选择方法[J]. 华中科技大学学报:自然科学版, 2012, 40(6): 39-43. ZHOU Yu, ZHU Anfu, ZHOU Lin, et al. Sample data selection method for neural network classifier[J]. Journal of Huazhong university of science and technology: natural science edition, 2012, 40(6): 39-43. [6] 郝红卫, 蒋蓉蓉. 基于最近邻规则的神经网络训练样本选择方法[J]. 自动化学报, 2007, 33(12): 1247-1251. HAO Hongwei, JIANG Rongrong. Training sample selection method for neural networks based on nearest neighbor rule[J]. Acta automatica sinica, 2007, 33(12): 1247-1251. [7] HARA K, NAKAYAMA K. A training method with small computation for classification[C]//Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. Como, Italy: IEEE, 2000: 543-548. [8] 邓乃扬, 田英杰. 数据挖掘中的新方法——支持向量机[M]. 北京: 科学出版社, 2004. [9] 刘刚, 张洪刚, 郭军. 不同训练样本对识别系统的影响[J]. 计算机学报, 2005, 28(11): 1923-1928. . LIU Gang, ZHANG Honggang, GUO Jun. The influence of different training samples to recognition system[J]. Chinese journal of computers, 2005, 28(11): 1923-1928. DOI:10.3321/j.issn:0254-4164.2005.11.020. [10] SCHAPIRE R E, SINGER Y. Improved boosting algorithms using confidence-rated predictions[J]. Machine learning, 1999, 37(3): 297-336. DOI:10.1023/A:1007614523901. [11] 韦岗, 贺前华. 神经网络模型学习及应用[M]. 北京: 电子工业出版社, 1994.
DOI: 10.11992/tis.201605031

0

#### 文章信息

LIU Wei, LIU Shang, BAI Runcai, ZHOU Xuan, ZHOU Dingning

Reducing training times in neural network classifiers by using dynamic data reduction

CAAI Transactions on Intelligent Systems, 2017, 12(2): 258-265
http://dx.doi.org/10.11992/tis.201605031