﻿ <i>ℓ</i> <sub>1</sub>−<i>ℓ</i> <sub>1</sub>双范数的最优下边界回归模型辨识
«上一篇
 文章快速检索 高级检索

 智能系统学报  2020, Vol. 15 Issue (5): 934-942  DOI: 10.11992/tis.201902006 0

### 引用本文

LIU Xiaoyong, YE Zhenhuan. Optimal lower boundary regression model based on double norms 1 1 optimization [J]. CAAI Transactions on Intelligent Systems, 2020, 15(5): 934-942. DOI: 10.11992/tis.201902006.

### 文章历史

1 1双范数的最优下边界回归模型辨识

Optimal lower boundary regression model based on double norms 1 1 optimization
LIU Xiaoyong , YE Zhenhuan
College of Engineering, Zunyi Normal University, Zunyi 563006, China
Abstract: In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable and one or more independent variables. Considering the uncertainties in the structure and parameters of the model derived from sensor measurement data, a new model called optimal lower boundary model is proposed to remove the uncertainties in parameters and characteristics. The proposed method is a combination of structural risk minimization theory (SRM) and some ideas from approximation error minimization. An optimal lower boundary regression model (LBRM) is presented using ${\ell _1} - {\ell _1}$ double norms optimization. First, constraint conditions subjected to LBRM are defined. Then, ${\ell _2}$ -norm optimization based on structural risk is converted into simple ${\ell _1}$ -norm optimization so that approximation error between the measurements based on ${\ell _1}$ -norm is computed and minimized. Next, LBRM is integrated into ${\ell _1}$ -norm optimization (based on structural risk). Thus, simpler linear programming can be applied to the constructed double-norms optimization problem to solve parameters of LBRM. Finally, the proposed method is demonstrated by experiments regarding uncertain measurements and parameters of nonlinear system. It has the following prominent features: modeling accuracy of LBRM can be guaranteed by introducing the ${\ell _1}$ -norm minimization on approximation error; model’s structural complexity is under control by ${\ell _1}$ -norm optimization based on structural risk, thus the performance of the model can be improved further.
Key words: ${\ell _1}$-norm-based structural risk minimization    ${\ell _1}$-norm on approximation error    lower boundary regression model    generalization performance    modeling accuracy    optimality    linear programming

1 支持向量回归的1范数问题转化 1.1 支持向量回归问题

 $f({{x}},{{\theta }}) = \sum\limits_{k = 1}^m {{\theta _k}{g_s}({{x}})} + b$ (1)

 $\min :\;\;\;R(f) = \sum\limits_{k = 1}^N {{L_\varepsilon }\left( {{y_k} - f({{{x}}_k})} \right)} + \gamma \left\| {{w}} \right\|\;_2^2$ (2)

$R(f)$ 为结构风险， $\gamma$ 表示规则化常量， $\left\| {{w}} \right\|\;_2^2$ 的引入在于控制模型的复杂度， ${L_\varepsilon }( \cdot )$ 描述 $\varepsilon -$ 不敏感损失函数，定义为

 ${L_\varepsilon }({y_k} - f({y_k} - f({{{x}}_k})) = \left\{ \begin{array}{l} 0\;,\;\;\;\;\;\;\;|{y_k} - f({{{x}}_k})| \leqslant \varepsilon \\ |{y_k} - f({{{x}}_k})| - \varepsilon ,\;\;{\text{其他}} \end{array} \right.$

 $\begin{array}{c} \min :\;\;\;W({{{\alpha }}^ + },{{{\alpha }}^ - }) = \varepsilon \displaystyle\sum\limits_{k = 1}^N {{L_\varepsilon }\left( {\alpha _k^ - + \alpha _k^ + } \right)} - \displaystyle\sum\limits_{k = 1}^N {{y_k}\left( {\alpha _k^ + - \alpha _k^ - } \right)} + \\ \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; \dfrac{1}{2}\displaystyle\sum\limits_{k,i = 1}^N {\left( {\left( {\alpha _k^ + - \alpha _k^ - } \right)\left( {\alpha _k^ + - \alpha _k^ - } \right)\displaystyle\sum\limits_{s = 1}^m {{g_s}({{{x}}_k}){g_s}({{{x}}_i})} } \right)} \; \\ {\rm{s}}.{\rm{t}}\;\;\;\;\displaystyle\sum\limits_{k = 1}^N {\alpha _k^ + } = \;\displaystyle\sum\limits_{k = 1}^N {\alpha _k^ - ,\;\;\;\;\;0 \leqslant \alpha _k^ + \;,\alpha _k^ - \leqslant \gamma },\;\;\; \\ {\rm{for}}\;\;{k = 1,2, \cdots ,N} \end{array}$ (3)

${{\bf{\alpha }}^ + }$ ${{\bf{\alpha }}^ - }$ $\alpha _k^ +$ $\alpha _k^ -$ 表示拉格朗日乘子。式（3） ${g_s}({{x}})$ 的内积可用如下核函数代替：

 $K({{{x}}_k},{{{x}}_i}) = \sum\limits_{s = 1}^m {{g_s}({{{x}}_k}){g_s}({{{x}}_i})}$

 $\begin{array}{c} \min :\;\;\;W({{\bf{\alpha }}^ + },{{\bf{\alpha }}^ - }) = \varepsilon \displaystyle\sum\limits_{k = 1}^N {{L_\varepsilon }\left( {\alpha _k^ - + \alpha _k^ + } \right)} - \displaystyle\sum\limits_{k = 1}^N {{y_k}\left( {\alpha _k^ + - \alpha _k^ - } \right)} + \\ \dfrac{1}{2}\displaystyle\sum\limits_{k,i = 1}^N {\left( {\left( {\alpha _k^ + - \alpha _k^ - } \right)\left( {\alpha _k^ + - \alpha _k^ - } \right)K({{{x}}_k},{{{x}}_i})} \right)} \end{array}$

 $f({{x}},{{{\alpha }}^ + },{{{\alpha }}^ - }) = \sum\limits_{k = 1}^m {({{{\alpha }}^ + } - {{{\alpha }}^ - })K({{x}},{{{x}}_i})} + b$ (7)

 $b = {y_k} - \sum\limits_{k = 1}^N {\left( {\alpha _k^ + - \alpha _k^ - } \right)K({{{x}}_k},{{{x}}_i}) + \varepsilon \cdot {\rm{sign}}\left( {\alpha _k^ - - \alpha _k^ + } \right)}$

 $f({{x}},{{\bf{\alpha }}^ + },{{\bf{\alpha }}^ - }) = \sum\limits_{k = 1}^m {\left( {\alpha _k^ + - \alpha _k^ - } \right)\exp \left\{ {\frac{{ - {{\left\| {{{x}} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right\}} + b$ (5)

1.2 SVR的 ${\ell _1}$ 范数优化问题转化

 $\begin{array}{l} \min :\;\;\;R(f) = C\displaystyle\sum\limits_{k = 1}^N {{L_\varepsilon }\left( {{\xi _k} + \xi _k^*} \right)} + \dfrac{1}{2}\left\| {{w}} \right\|\;_2^2 \\ \;\;{\rm{s}}.{\rm{t}}.\;\;\;\;\;\;\;\;\;\;\;\;\;\left\{ \begin{array}{l} {y_k} - \left\langle {{{w}},\;\varphi ({{{x}}_k})} \right\rangle - b \leqslant \varepsilon + {\xi _k}, \\ \left\langle {{{w}},\;\varphi ({{{x}}_k})} \right\rangle + b - {y_k} \leqslant \varepsilon + \xi _k^* \\ {\xi _k},\xi _k^* \geqslant 0 \\ \end{array} \right. \\ \end{array}$

 $f({{x}},{\bf{\beta }}) = \sum\limits_{k = 1}^N {{\beta _k}\exp \left( {\frac{{ - {{\left\| {{{x}} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right)} + b$ (6)

$\;{{\beta }} = {[{\beta _1},{\beta _2}, \cdots ,{\beta _N}]^{\rm{T}}}$ 。考虑到式（2）的优化问题， $\left\| {{w}} \right\|\;_2^2$ 范数的引入是为了控制模型的复杂度，根据范数的等价性可知，在结构风险中引入其他范数也可以同样对模型复杂性进行控制。接下来，将QP-SVR的优化问题(2)变成

 $\min :\;\;\;R(f) = \sum\limits_{k = 1}^N {{L_\varepsilon }\left( {{y_k} - f({{{x}}_k})} \right)} + \gamma {\left\| {\bf{\beta }} \right\|_{\;1}}$

 $\begin{array}{c} \min :\;\;\;R(f) = C\displaystyle\sum\limits_{k = 1}^N {\left( {{\xi _k} + \xi _k^*} \right)} + {\left\| {\bf{\beta }} \right\|_{\;1}} \\ \;\;{\rm{s}}.{\rm{t}}.\;\;\;\;\;\;\;\;\;\;\;\;\;\left\{ \begin{array}{l} {y_k} - \displaystyle\sum\limits_{k = 1}^m {{\beta _k}\exp \left\{ {\dfrac{{ - {{\left\| {{{x}} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right\} - } b \leqslant \varepsilon + \xi _k^* \\ \displaystyle\sum\limits_{k = 1}^m {{\beta _k}\exp \left\{ {\dfrac{{ - {{\left\| {{{x}} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right\}} + b - {y_k} \leqslant \varepsilon + \xi _k^* \\ {\xi _k},\xi _k^* \geqslant 0 \end{array} \right. \end{array} \!\!\!$ (7)

 $\begin{array}{c} \min :\;\;\;R(f) = C\displaystyle\sum\limits_{k = 1}^N {{\xi _k}} + {\left\| {\bf{\beta }} \right\|_{\;1}} \\ \;\;{\rm{s}}.{\rm{t}}.\;\;\;\;\;\;\;\;\;\;\;\;\;\left\{ \begin{array}{l} {y_k} - \displaystyle\sum\limits_{k = 1}^m {{\beta _k}\exp \left\{ {\dfrac{{ - {{\left\| {{{x}} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right\} - } b \leqslant \varepsilon + {\xi _k} \\ \displaystyle\sum\limits_{k = 1}^m {{\beta _k}\exp \left\{ {\dfrac{{ - {{\left\| {{{x}} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right\}} + b - {y_k} \leqslant \varepsilon + {\xi _k} \\ {\xi _k} \geqslant 0 \end{array} \right. \end{array}$ (8)

 $\begin{array}{l} {\beta _k} = \alpha _k^ + - \alpha _k^ - \\ |{\beta _k}| = \alpha _k^ + + \alpha _k^ - \\ \end{array}$ (9)

 $\begin{array}{c} \min :\;\;\;R(f) = C\displaystyle\sum\limits_{k = 1}^N {{\xi _k}} + \displaystyle\sum\limits_{i = 1}^N {(\alpha _k^ + + \alpha _k^ - )} \\ \;\;{\rm{s}}.{\rm{t}}\;\;\;\left\{ \begin{array}{l} {y_k} - \displaystyle\sum\limits_{k = 1}^m {(\alpha _k^ + - \alpha _k^ - )\exp \left\{ {\dfrac{{ - {{\left\| {{{x}} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right\} - } b \leqslant \varepsilon + {\xi _k} \\ \displaystyle\sum\limits_{k = 1}^m {(\alpha _k^ + - \alpha _k^ - )\exp \left\{ {\dfrac{{ - {{\left\| {{{x}} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right\}} + b - {y_k} \leqslant \varepsilon + {\xi _k} \\ {\xi _k} \geqslant 0 \\ \end{array} \right. \\ \end{array}$ (10)

 $\begin{array}{c} \min \;\;\;\;\;\;{{{c}}^{\rm{T}}}\left( \begin{array}{l} {{{\alpha }}^ + } \\ {{{\alpha }}^ - } \\ {{\xi }} \\ \end{array} \right) \\ {\rm{s}}.{\rm{t}}.\;\;\;\;\;\left\{ \begin{array}{l} \left( \begin{array}{l} \;{{K}}\;\;\; - {{K}}\;\;\; - {{I}} \\ - {{K}}\;\;\;\;{{K}}\;\;\;\; - {{I}} \\ \end{array} \right) \cdot \left( \begin{array}{l} {{{\alpha }}^ + } \\ {{{\alpha }}^ - } \\ {{\xi }} \\ \end{array} \right) \leqslant \left( \begin{array}{l} {{y}} + \varepsilon \\ \varepsilon - {{y}} \\ \end{array} \right) \\ {{{\alpha }}^ + },\;{{{\alpha }}^ - }\; \geqslant 0,\;\;\;{{\xi }} \geqslant 0 \end{array} \right. \end{array}$ (11)

 ${{{K}}_{ij}} = k({{{x}}_i},{{{x}}_j}) = {\rm{exp}}\left\{ {\frac{{ - {{\left\| {{{{x}}_i} - {{{x}}_j}} \right\|}^2}}}{{2{\sigma ^2}}}} \right\}。$

2 基于 1范数的回归模型辨识

 ${y_k} = g({{{x}}_k}),\;\;\;\;\;\;k = 1,2, \cdots ,N$

 $\mathop {\sup }\limits_{{{{x}}_k} \in {{S}}} \left| {f({{{x}}_k}) - g({{{x}}_k})} \right| < \eta \;\;\;\;\;\;\;\;\;\;\forall k$

 ${e_k} = {y_k} - f({{{x}}_k})\;\;\;\;\;\;\forall k$ (12)

 $\mathop {\min }\limits_{{{{x}}_k} \in Z} \left| {{y_k} - f({{{x}}_k})} \right|\;\;\;\;\;\;\forall k$ (21)

$Z$ 表示整个输入数据集。显然，这是一个最小（min）优化问题。在式（6）描述的回归模型情况下，式（12）的最小化可通过两个阶段完成：1）核函数中的核宽度 $\sigma$ 的参数寻优，通常采用经典的交叉验证或其他方法来实现，其详细过程在本文中不再讨论；2）式（6）的参数确定可通过min优化问题求解，即

 ${\bf{\beta }} = \arg \;\mathop {\min }\limits_{{\bf{\beta }}\;,\;{{{x}}_k} \in Z} \;\;\left| {{y_k} - \sum\limits_{i = 1}^N {{\beta _i}\exp \left( {\frac{{ - {{\left\| {{{{x}}_i} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right)} - b} \right|$
3 最优下边界回归模型辨识

 $\varGamma = \{ g:{{S}} \to {{R}^1}|\;g({{x}}) = {g_{{\rm{nom}}}}({{x}}) + \Delta g({{x}}),\;{{x}} \in {{S}}\}$

${g_{{\rm{nom}}}}$ 为标称函数，不确定性 $\Delta g({{z}})$ 满足 $\mathop {\sup }\limits_{{{x}} \in {{S}}} |\Delta g({{x}})| \leqslant \gamma$ $\gamma \in {R}$ 。现考虑来自函数簇 $\varGamma$ 的成员函数 $g$ ${{x}} \in {{R}^d}$ ，对应输入 ${{x}}$ 上的测量输出 ${{Y}} = \{ {y_1},{y_1}, \cdots , {y_N}\}$ ，即 ${y_k} = g({{\bf{x}}_k})$ $g \in \varGamma$ ${{{x}}_k} \in {{S}}$ $k = 1,2, \cdots ,N$ 。LBRM建模的思想是，在满足如下约束条件(14)的条件下，建模下界回归模型 $f({{{x}}_k})$

 $f({{{x}}_k}) \leqslant g({{{x}}_k})\;\;\;\;\forall {{{x}}_k} \in {{S}}$ (14)

 $f({{x}},{{\beta }},b) = \sum\limits_{k = 1}^N {{\beta _k}\exp \left( {\frac{{ - {{\left\| {{{x}} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right)} + b$

 $\mathop {\min }\limits_{f,\;\;{{{x}}_k} \in S} \;\;\sum\limits_{k = 1}^N {({y_k} - f({{{x}}_k}))} \;\;\;{\rm{s}}.{\rm{t}}.\;\;\;{y_k} - f({{{x}}_k})\geqslant 0$ (15)

 $\begin{array}{c} \;\;\;\min :\;\;\;\;\;\;\lambda \; = \displaystyle\sum\limits_{k = 1}^N {{\lambda _k}} \\ \left\{ \begin{array}{l} {y_k} - \displaystyle\sum\limits_{i = 1}^N {\beta _i^{}\exp \left( {\dfrac{{ - {{\left\| {{{{x}}_i} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right)} - {b_{}} \leqslant {\lambda _k},\;\;\;\;k = 1,2, \cdots ,N \\ {y_k} - \displaystyle\sum\limits_{i = 1}^N {\beta _i^{}\exp \left( {\dfrac{{ - {{\left\| {{{{x}}_i} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right)} - \beta _i^{} \geqslant 0\;\;\;k = 1,2, \cdots ,N \\ {\lambda _k} \geqslant 0 \end{array} \right. \end{array}$ (16)

 $\begin{array}{c} \;\;\;\min :\;\;\;C\displaystyle\sum\limits_{k = 1}^N {{\xi _k}} + \displaystyle\sum\limits_{i = 1}^N {(\alpha _k^ + + \alpha _k^ - )} + \displaystyle\sum\nolimits_{k = 1}^N {{\lambda _k}} + b \\ \left\{ \begin{array}{l} \displaystyle\sum\limits_{k = 1}^m {(\alpha _k^ + - \alpha _k^ - )\exp \left\{ {\dfrac{{ - {{\left\| {{{x}} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right\}} + b - {y_k} \leqslant \varepsilon + {\xi _k}, \\ {y_k} - \displaystyle\sum\limits_{k = 1}^m {(\alpha _k^ + - \alpha _k^ - )\exp \left\{ {\dfrac{{ - {{\left\| {{{x}} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right\} - } b \leqslant \varepsilon + {\xi _k}, \\ {y_k} - \displaystyle\sum\limits_{i = 1}^N {(\alpha _k^ + - \alpha _k^ - )\exp \left( {\dfrac{{ - {{\left\| {{{{x}}_i} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right)} - b \leqslant {\lambda _k}, \\ \displaystyle\sum\limits_{i = 1}^N {(\alpha _k^ + - \alpha _k^ - )\exp \left( {\dfrac{{ - {{\left\| {{{{x}}_i} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right)} + b - {y_k} \leqslant 0, \\ \;\;\;\;\;\;{\xi _k} \geqslant 0,\;\;{\lambda _k} \geqslant 0,\;\;k = 1,2, \cdots ,N \\ \end{array} \right. \\ \end{array}$ (17)

 $\begin{array}{c} \min \;\;\;\;\;\;{{{c}}^{\rm{T}}}\left( \begin{array}{l} {{\alpha }}_{}^ + \\ {{\alpha }}_{}^ - \\ \;{{\xi }} \\ \;{{{\lambda }}_{}} \\ \;{b_{}} \\ \end{array} \right) \\ {\rm{s}}.{\rm{t}}.\;\;\;\;\;\left\{ \begin{array}{l} \left( \begin{array}{l} \;{{K}}\;\;\; - {{K}}\;\;\; - {{I}}\;\;\;{{Z}}\;\;\;{{E}} \\ - {{K}}\;\;\;\;{{K}}\;\;\;\; - {{I}}\;\;\;{{Z}}\;\;\;{{E}} \\ - {{K}}\;\;\;\;{{K}}\;\;\;\;\;\;{{Z}}\;\; - {{I}}\;\;{{E}} \\ \;{{K}}\;\;\; - {{K}}\;\;\;\;\;{{Z}}\;\;\;{{Z}}\;\;\;{{E}} \\ \end{array} \right) \cdot \left( \begin{array}{l} {{\alpha }}_{}^ + \\ {{\alpha }}_{}^ - \\ \;{{\xi }} \\ \;{{{\lambda }}_{}} \\ \;{b_{}} \\ \end{array} \right) \leqslant \left( \begin{array}{l} {{y}} + \varepsilon \\ \varepsilon - {{y}} \\ \; - {{y}} \\ \;\;\;\;{{y}} \\ \end{array} \right) \\ {{\alpha }}_{}^ + ,\;{{\alpha }}_{}^ - \; \geqslant 0,\;{{\xi }} \geqslant 0,\;\;0 \leqslant {\lambda _k} \leqslant 1 \\ \end{array} \right. \\ \end{array}$ (18)

${{{K}}_{ij}} = {{K}}({{{x}}_i},{{{x}}_j}) = \exp \left\{ {\dfrac{{ - {{\left\| {{{{x}}_i} - {{{x}}_j}} \right\|}^2}}}{{2{\sigma ^2}}}} \right\}$ $\sigma$ 为可调核参数。显然，应用内点法或单纯性方法可以求解优化问题(18)，进而得到下界回归模型 $f({{x}})$

 $f({{x}}) = \sum\limits_{k = 1}^N {(\alpha _k^ + - \alpha _k^ - )\exp \left( {\frac{{ - {{\left\| {{{x}} - {{{x}}_k}} \right\|}^2}}}{{2{\sigma ^2}}}} \right)} + b$ (19)

4 实验分析

 ${\rm{RMSE}} = \frac{1}{N}\sqrt {\sum\limits_{k = 1}^N {{{\left( {y_k - {\hat y}_k} \right)}^2}} } \;$

 ${\rm{SVs}}\% = \frac{{{N_k}}}{N} \times 100\%$

 $\begin{array}{c} y(t + 1) = \dfrac{{y(t)y(t - 1)[y(t) + 2.5]}}{{1 + {y^2}(t) + {y^2}(t - 1)}} + u(t) + {\rm{noise}} \\ y(0) = y(1) = 0,\;\;\;u(t) = \sin (2\pi t/50) \end{array}$ (20)

LBRM的最优性，除了应用提出方法在辨识精度与稀疏特性之间取其平衡得以体现之外，超参数集的选取对LBRM的稀疏特性也起着至关重要的作用。在实验分析中，超参数集的4种取值主要是基于SVR方法的经验来获取[23]，其中不敏感域 $\varepsilon$ 的取值一般在区间 $[0\;,\;1]$ 之间获取，规则化参数 $\gamma$ 一般选取为 ${2^{{n}}},n = - 5,\; - 4,\; \cdots \;,15$ ，核参数 $\sigma$ 一般从区间 $[0\;,\;10]$ 获取。当超参数集 $(\varepsilon ,\;\gamma ,\;\sigma )$ 选择为 $(0.001,\;1\;000,\;\;5.0)$ 时，应用提出方法获取的最优下边界回归模型（LBRM）如图1所示。

 Download: 图 1 提出方法所建立的最优下界回归模型(核宽度为5.0) Fig. 1 Optimal LBRM constructed by our approach, where σ=5.0

 Download: 图 2 提出方法所对应的逼近误差 Fig. 2 Approximation error of the proposed method
 Download: 图 3 提出方法所建立的最优下界回归模型(核宽度为0.1) Fig. 3 Optimal LBRM constructed by our approach, where σ=0.1
 Download: 图 4 过拟合所对应的逼近误差 Fig. 4 Approximation error of the proposed method when the over-fitting appeared.

 ${f_{{\rm{norm}}}}({{x}}) = \cos {{x}}\sin {{x}}$
 $\Delta f({{x}}) = \tau \cos (8{{x}})$
 $g({{x}}) = {f_{{\rm{norm}}}}({{x}}) + \Delta f({{x}})$ (21)

 Download: 图 5 提出方法所建立的最优下界回归模型(核宽度为10.5) Fig. 5 Optimal LBRM constructed by our approach, where σ=10.5

 Download: 图 6 第k个支持向量所对应的 $\alpha _k^ + - \alpha _k^ -$ 值( $\alpha _k^ + - \alpha _k^ - \geqslant {10^{ - 11}}$ ) Fig. 6 The k-th support vector (SV) corresponding to the values of $\alpha _k^ + - \alpha _k^ -$ ( $\alpha _k^ + - \alpha _k^ - \geqslant {10^{ - 11}}$ )

 $\begin{array}{c} f({{x}}) = - 0.26 \cdot \exp \left( {\dfrac{{ - {{\left\| {{{x}} + 0.691\;69} \right\|}^2}}}{{2 \times {{10.5}^2}}}} \right) \; - \\0.40 \cdot \exp \left( {\dfrac{{ - {{\left\| {{{x}} + 0.70} \right\|}^2}}}{{2 \times {{10.5}^2}}}} \right) - 0.18 \cdot \exp \left( {\dfrac{{ - {{\left\| {{{x}} + 0.37} \right\|}^2}}}{{2 \times {{10.5}^2}}}} \right) \; -\\ 0.18 \cdot \exp \left( {\dfrac{{ - {{\left\| {{{x}} + 0.23} \right\|}^2}}}{{2 \times {{10.5}^2}}}} \right) - 0.05 \cdot \exp \left( {\dfrac{{ - {{\left\| {{{x}} + 0.22} \right\|}^2}}}{{2 \times {{10.5}^2}}}} \right) \; +\\ 0.14 \cdot \exp \left( {\dfrac{{ - {{\left\| {{{x}} + 0.37} \right\|}^2}}}{{2 \times {{10.5}^2}}}} \right) + 0.25 \cdot \exp \left( {\dfrac{{ - {{\left\| {{{x}} + 0.39} \right\|}^2}}}{{2 \times {{10.5}^2}}}} \right) \; + \\0.29 \cdot \exp \left( {\dfrac{{ - {{\left\| {{{x}} + 0.46} \right\|}^2}}}{{2 \times {{10.5}^2}}}} \right) \end{array}$

 Download: 图 7 提出方法所建立的最优下界回归模型(核宽度为4.5) Fig. 7 Optimal LBRM constructed by our approach, where σ=4.5

5 结束语

 [1] HAN Honggui, GUO Yanan, QIAO Junfei. Nonlinear system modeling using a self-organizing recurrent radial basis function neural network[J]. Applied soft computing, 2018, 71: 1105-1116. DOI:10.1016/j.asoc.2017.10.030 (0) [2] FRAVOLINI M L, NAPOLITANO M R, DEL CORE G, et al. Experimental interval models for the robust fault detection of aircraft air data sensors[J]. Control engineering practice, 2018, 78: 196-212. DOI:10.1016/j.conengprac.2018.07.002 (0) [3] FANG Shengen, ZHANG Qiuhu, REN Weixin. An interval model updating strategy using interval response surface models[J]. Mechanical systems and signal processing, 2015, 60−61: 909-927. (0) [4] LEUNG F H F, LAM H K, LING S H, et al. Tuning of the structure and parameters of a neural network using an improved genetic algorithm[J]. IEEE transactions on neural networks, 2003, 14(1): 79-88. DOI:10.1109/TNN.2002.804317 (0) [5] 赵文清, 严海, 王晓辉. BP神经网络和支持向量机相结合的电容器介损角辨识[J]. 智能系统学报, 2019, 14(1): 134-140. ZHAO Wenqing, YAN Hai, WANG Xiaohui. Capacitor dielectric loss angle identification based on a BP neural network and SVM[J]. CAAI transactions on intelligent systems, 2019, 14(1): 134-140. (0) [6] 刘道华, 张礼涛, 曾召霞, 等. 基于正交最小二乘法的径向基神经网络模型[J]. 信阳师范学院学报(自然科学版), 2013, 26(3): 428-431. LIU Daohua, ZHANG Litao, ZENG Zhaoxia, et al. Radial basis function neural network model based on orthogonal least squares[J]. Journal of Xinyang Normal University (natural science edition), 2013, 26(3): 428-431. DOI:10.3969/j.issn.1003-0972.2013.03.030 (0) [7] 刘道华, 张飞, 张言言. 一种改进的RBF神经网络对县级政府编制预测[J]. 信阳师范学院学报(自然科学版), 2016, 29(2): 265-269. LIU Daohua, ZHANG Fei, ZHANG Yanyan. A prediction for the preparation of county government based on improved RBF neural networks[J]. Journal of Xinyang Normal University (natural science edition), 2016, 29(2): 265-269. DOI:10.3969/j.issn.1003-0972.2016.02.027 (0) [8] HAN Honggui, GE Luming, QIAO Junfei. An adaptive second order fuzzy neural network for nonlinear system modeling[J]. Neurocomputing, 2016, 2014: 837-847. (0) [9] LI Fanjun, QIAO Junfei, HAN Honggui, et al. A self-organizing cascade neural network with random weights for nonlinear system modeling[J]. Applied soft computing, 2016, 42: 184-193. DOI:10.1016/j.asoc.2016.01.028 (0) [10] HAN Honggui, QIAO Junfei. Hierarchical neural network modeling approach to predict sludge volume index of wastewater treatment process[J]. IEEE transactions on control systems technology, 2013, 21(6): 2423-2431. DOI:10.1109/TCST.2012.2228861 (0) [11] VAPNIK V N. An overview of statistical learning theory[J]. IEEE transactions on neural networks, 1999, 10(5): 988-999. DOI:10.1109/72.788640 (0) [12] 唐波, 彭友仙, 陈彬, 等. 基于BP神经网络的交流输电线路可听噪声预测模型[J]. 信阳师范学院学报(自然科学版), 2015, 28(1): 136-140. TANG Bo, PENG Youxian, CHEN Bin, et al. Audible noise prediction model of ac power lines based on BP neural network[J]. Journal of Xinyang Normal University (natural science edition), 2015, 28(1): 136-140. DOI:10.3969/j.issn.1003-0972.2015.01.033 (0) [13] HAO Peiyi. Interval regression analysis using support vector networks[J]. Fuzzy sets and systems, 2009, 160(17): 2466-2485. DOI:10.1016/j.fss.2008.10.012 (0) [14] HAO Peiyi. Possibilistic regression analysis by support vector machine[C]//Proceedings of 2011 IEEE International Conference on Fuzzy Systems. Taipei, China, 2011: 889−894. (0) [15] 石磊, 侯丽萍. 基于改进PSO算法参数优化的模糊支持向量分类机[J]. 信阳师范学院学报(自然科学版), 2013, 26(2): 288-291. SHI Lei, HOU Liping. Parameter optimization of fuzzy support vector classifiers based on the improved PSO[J]. Journal of Xinyang Normal University (natural science edition), 2013, 26(2): 288-291. DOI:10.3969/j.issn.1003-0972.2013.02.031 (0) [16] SOUZA L C, SOUZA R M C R, AMARAL G J A, et al. A parametrized approach for linear regression of interval data[J]. Knowledge-based systems, 2017, 131: 149-159. DOI:10.1016/j.knosys.2017.06.012 (0) [17] DE A LIMA NETO E, DE A T DE CARVALHO F. An exponential-type kernel robust regression model for interval-valued variables[J]. Information sciences, 2018, 454−455: 419-442. (0) [18] BASAK D, PAL S, PATRANABIS D C. Support vector regression[J]. Neural information processing-letters and reviews, 2007, 11(10): 203-224. (0) [19] SMOLA A J, SCHÖLKOPF B. A tutorial on support vector regression[J]. Statistics and computing, 2004, 14(3): 199-222. DOI:10.1023/B:STCO.0000035301.49549.88 (0) [20] SOARES Y M G, FAGUNDES R A A. Interval quantile regression models based on swarm intelligence[J]. Applied soft computing, 2018, 72: 474-485. DOI:10.1016/j.asoc.2018.04.061 (0) [21] LU Zhao, SUN Jing, BUTTS K. Linear programming SVM-ARMA2K with application in engine system identification[J]. IEEE transactions on automation science and engineering, 2011, 8(4): 846-854. DOI:10.1109/TASE.2011.2140105 (0) [22] RIVAS-PEREA P, COTA-RUIZ J. An algorithm for training a large scale support vector machine for regression based on linear programming and decomposition methods[J]. Pattern recognition letters, 2013, 34(4): 439-451. DOI:10.1016/j.patrec.2012.10.026 (0) [23] HSU C W, CHANG C C, LIN C J. A practical guide to support vector classification. Technical report. Department of Computer Science, National Taiwan University (2003)[EB/OL](2010-4-15). http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf. (0)