﻿ 基于<i>L</i><sub>1/2</sub>范数正则化的塑性回声状态网络故障诊断模型<sup>*</sup>
 文章快速检索 高级检索

1. 海军航空大学 岸防兵学院, 烟台 264001;
2. 中央军委联合参谋部 第55研究所, 北京 100094

A fault diagnosis model of plasticity echo state network based on L1/2-norm regularization
LU Cheng1, XU Tingxue1, WANG Hong2
1. Coastal Defense College, Naval Aeronautical and Astronautical University, Yantai 264001, China;
2. The 55 th Institute, Joint Staff Department of the Central Military Commission, Beijing 100094, China
Received: 2017-04-10; Accepted: 2017-08-11; Published online: 2017-10-19 11:12
Foundation item: National Natural Science Foundation of China (51605487); Shandong Provincial Natural Science Foundation, China (ZR2016FQ03)
Corresponding author. XU Tingxue, E-mail: xtx-yt@163.com
Abstract: In order to improve the dynamic adaptability of reservoir, overcome the ill-posed problems of output weights in echo state network (ESN), and balance the fitting and generalization ability, a fault diagnosis model of plasticity echo state network based on L1/2-norm regularization is presented. BCM rule was introduced into the reservoir construction to train the connection weight matrix. Meanwhile, the L1/2-norm penalty term was added to the objective function in order to improve the sparsification efficiency. An iterative numerical oscillation problem was solved by using a smoothing L1/2 regularizer, and finally the model was solved by using the half threshold iteration method. The model is applied to the fault diagnosis of airborne radio station, and the simulation results prove the validity and superiority of the model.
Key words: reservoir     echo state network (ESN)     BCM rule     L1/2-norm regularization     half threshold iteration method     fault diagnosis

1 基于ESN的分类基本原理

 (1)
 图 1 ESN模型结构 Fig. 1 Architecture of ESN model

 (2)
 (3)

 (4)

 (5)
 (6)

2 储备池连接权矩阵优化

BCM规则[20]遵循Hebbian学习原理，其通过一个相当于稳定器功能的滑动阈值来控制突触的变化，阈值的改变能够控制神经元活动的增强或减弱，这一自我调节的可塑特性能够改善储备池的学习稳定性。BCM规则不仅对前后突触的时间移动平均值起作用，还能够调节突触后的活动，当积极权值变化水平较高时，减少相应的改变量。在BCM规则下，阈值大小与突触变化速率呈反比关系，当突触变化速率增加时，阈值处于较小值，反之阈值也相应增加，这也就意味着修正阈值决定了突触效能改变的方向，原理如图 2所示。

BCM规则有多种变式，本文采用如下一种形式：

 (7)
 (8)
 图 2 突触权值修正规则 Fig. 2 Synaptic weight modification rule

3 基于L1/2范数正则化的回声状态网络 3.1 光滑L1/2范数正则化模型

 (9)

 (10)

L1/2正则子[25]为解决特征提取及变量选择问题提供了一个新的思路，实验[26]已经证明L1/2范数正则化模型较流行的L1范数模型有更稀疏的解，且鲁棒性更优。本文利用L1/2正则子的优良性质，将其与ESN结合，在目标函数中增加L1/2正则项：

 (11)
 (12)

 (13)
 (14)

3.2 L1/2-ESN模型

R0M×N={Z=(z1, z2, …, zM)T, zi0}，若WR0M×N为式(11)所示正则化模型的解，根据W的最优化一阶条件，可得

 (15)

 (16)

 (17)

 (18)
 (19)
 (20)
 (21)

 (22)

 (23)
 (24)

 (25)

 (26)

 (27)

 (28)

3.3 L1/2-PESN模型

 图 3 L1/2-PESN模型结构 Fig. 3 Architecture of L1/2-PESN model

4 机载电台故障诊断实例分析

 图 4 机载通信电台组成 Fig. 4 Composition of airborne communication station

 序号 测试参数 故障模块 c1 c2 c3 c4 c5 c6 1 2 0 1 1 1 1 d1 2 1 2 1 2 1 2 d2         174 1 1 1 1 1 2 d2 175 1 2 1 0 1 1 d1

 方法 储备池生成时间/s 训练时间/s 诊断正确率/% BPNN 43.76 79.6 传统ESN 0.49 14.13 88.4 BCM-ESN 6.04 14.37 90.5 L1/2-ESN 0.36 16.78 91.2 L1/2-PESN 6.58 16.64 93.1

5 结论

1) 本文提出的L1/2-PESN模型在储备池生成过程中通过引入BCM规则对储备池连接权值进行自组织优化，使得权值与训练样本输入数据相适应，并可根据输入样本的改变而做出相应调整，改善了储备池的动态适应性能和样本训练过程中的数据拟合能力。

2) 借助于L1/2正则子优秀的稀疏性表现，对储备池进行了有效的输出特征选择，在控制网络规模的同时提升了模型的泛化能力。基于机载电台的故障诊断实验结果表明，本文方法较BPNN和传统ESN模型，具有良好的稳定性及更高的诊断准确率。

 [1] CHINE W, MELLIT A, LUGHI V, et al. A novel fault diagnosis technique for photovoltaic systems based on artificial neural networks[J]. Renewable Energy, 2016, 90 : 501–512. DOI:10.1016/j.renene.2016.01.036 [2] UNAL M, ONAT M, DEMETGUL M, et al. Fault diagnosis of rolling bearings using a genetic algorithm optimized neural network[J]. Measurement, 2014, 58 : 187–196. DOI:10.1016/j.measurement.2014.08.041 [3] SHATNAWI Y, AL-KHASSAWENEH M. Fault diagnosis in internal combustion engines using extension neural network[J]. IEEE Transactions on Industrial Electronics, 2014, 61 (3): 1434–1443. DOI:10.1109/TIE.2013.2261033 [4] JAEGER H. The "echo state" approach to analysing and training recurrent neural networks-with an erratum note[R]. Bonn: German National Research Center for Information Technology GMD Technical Report, 2001. [5] LUN S X, YAO X S, QI H Y, et al. A novel model of leaky integrator echo state network for time-series prediction[J]. Neurocomputing, 2015, 159 : 58–66. [6] VARSHNEY S, VERMA T. Half hourly electricity load prediction using echo state network[J]. International Journal of Science and Research, 2014, 3 (6): 885–888. [7] MORANDO S, JEMEI S, HISSEL D, et al. ANOVA method applied to proton exchange membrane fuel cell ageing forecasting using an echo state network[J]. Mathematics and Computers in Simulation, 2017, 131 : 283–294. DOI:10.1016/j.matcom.2015.06.009 [8] 许美玲, 韩敏. 多元混沌时间序列的因子回声状态网络预测模型[J]. 自动化学报, 2015, 41 (5): 1042–1046. XU M L, HAN M. The model of factor echo state network prediction for multivariate chaotic time series[J]. Acta Automatica Sinica, 2015, 41 (5): 1042–1046. (in Chinese) [9] 郭嘉, 雷苗, 彭喜元. 基于相应簇回声状态网络静态分类方法[J]. 电子学报, 2011, 39 (3A): 14–18. GUO J, LEI M, PENG X Y. Static classification method based on corresponding cluster echo state network[J]. Acta Sinica, 2011, 39 (3A): 14–18. (in Chinese) [10] SCARDAPANE S, UNCINI A. Semi-supervised echo state networks for audio classification[J]. Cognitive Computation, 2017, 9 (1): 125–135. DOI:10.1007/s12559-016-9439-z [11] SONG Q S, FENG Z R. Effects of connectivity structure of complex echo state network on its prediction performance for nonlinear time series[J]. Neurocomputing, 2010, 73 (10-12): 2177–2185. DOI:10.1016/j.neucom.2010.01.015 [12] MARTIN C E, REGGIA J A. Fusing swarm intelligence and self-assembly for optimizing echo state networks[J]. Computational Intelligence and Neuroscience, 2015, 2015 (5-6): 642429. [13] DUTOIT X, SCHRAUWEN B, VAN CAMPENHOUT J, et al. Pruning and regularization in reservoir computing[J]. Neurocomputing, 2009, 72 (7): 1534–1546. [14] KUMP P, BAI E W, CHAN K, et al. Variable selection via RIVAL(removing irrelevant variables amidst Lasso iterations) and its application to nuclear material detection[J]. Automatica, 2012, 48 (9): 2107–2115. DOI:10.1016/j.automatica.2012.06.051 [15] SHI Z, HAN M. Support vector echo-state machine for chaotic time-series prediction[J]. IEEE Transactions on Neural Networks, 2007, 18 (2): 359–372. DOI:10.1109/TNN.2006.885113 [16] 刘建伟, 李双成, 罗雄麟. p范数正则化支持向量机分类算法[J]. 自动化学报, 2012, 38 (1): 76–87. LIU J W, LI S C, LUO X L. Classification algorithm of support vector machine via p-norm regularization[J]. Acta Automatica Sinica, 2012, 38 (1): 76–87. (in Chinese) [17] 韩敏, 李德才. 基于替代函数及贝叶斯框架的1范数ELM算法[J]. 自动化学报, 2011, 37 (11): 1344–1350. HAN M, LI D C. An norm 1 regularization term ELM algorithm based on surrogate function and Bayesian framework[J]. Acta Automatica Sinica, 2011, 37 (11): 1344–1350. (in Chinese) [18] ZOU H, HASTIE T. Regularization and variable selection via the elastic net[J]. Journal of the Royal Statistical Society, 2005, 67 (2): 301–320. [19] LUKOŠEVIČIUS M, JAEGER H. Reservoir computing approaches to recurrent neural network training[J]. Computer Science Review, 2009, 3 (3): 127–149. DOI:10.1016/j.cosrev.2009.03.005 [20] CASTELLANI G C, INTRATOR N, SHOUVAL H, et al. Solutions of the BCM learning rule in a network of lateral interacting nonlinear neurons[J]. Network:Computation in Neural Systems, 1999, 10 (2): 111–121. DOI:10.1088/0954-898X_10_2_001 [21] LEFORT M, BONIFACE Y, GIRAU B. Self-organization of neural maps using a modulated BCM rule within a multimodal architecture[C]//Brain Inspired Cognitive Systems 2010. Berlin: Springer, 2010: 26-38. [22] TIBSHIRANI R. Regression shrinkage and selection via the lasso[J]. Journal of the Royal Statistical Society, 1996, 58 (1): 267–288. [23] 彭义刚, 索津莉, 戴琼海, 等. 从压缩传感到低秩矩阵恢复:理论与应用[J]. 自动化学报, 2013, 39 (7): 981–994. PENG Y G, SUO J L, DAI Q H, et al. From compressed sensing to low-rank matrix recovery:Theory and applications[J]. Acta Automatica Sinica, 2013, 39 (7): 981–994. (in Chinese) [24] ZOU H. The adaptive lasso and its oracle properties[J]. Journal of the American Statistical Association, 2006, 101 (476): 1418–1429. DOI:10.1198/016214506000000735 [25] XU Z, ZHANG H, WANG Y, et al. L 1/2 regularization[J]. Science China Information Sciences, 2010, 53 (6): 1159–1169. DOI:10.1007/s11432-010-0090-0 [26] DAUBECHIES I, DEVORE R, FORNASIER M, et al. Iteratively reweighted least squares minimization for sparse recovery[J]. Communications on Pure and Applied Mathematics, 2010, 63 (1): 1–38. [27] XU Z, CHANG X, XU F, et al. L1/2 regularization:A thresholding representation theory and a fast solver[J]. IEEE Transactions on Neural Networks and Learning Systems, 2012, 23 (7): 1013–1027. DOI:10.1109/TNNLS.2012.2197412 [28] ZENG J, LIN S, WANG Y, et al. L1/2 regularization:Convergence of iterative half thresholding algorithm[J]. IEEE Transactions on Signal Processing, 2014, 62 (9): 2317–2329. DOI:10.1109/TSP.2014.2309076

#### 文章信息

LU Cheng, XU Tingxue, WANG Hong

A fault diagnosis model of plasticity echo state network based on L1/2-norm regularization

Journal of Beijing University of Aeronautics and Astronsutics, 2018, 44(3): 535-541
http://dx.doi.org/10.13700/j.bh.1001-5965.2017.0214