«上一篇
 文章快速检索 高级检索

 智能系统学报  2019, Vol. 14 Issue (5): 953-958  DOI: 10.11992/tis.201808004 0

引用本文

HUANG Qingkang, SONG Kaitao, LU Jianfeng. Application of the loss balance function to the imbalanced multi-classification problems[J]. CAAI Transactions on Intelligent Systems, 2019, 14(5): 953-958. DOI: 10.11992/tis.201808004.

文章历史

Application of the loss balance function to the imbalanced multi-classification problems
HUANG Qingkang , SONG Kaitao , LU Jianfeng
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China
Abstract: The traditional classification algorithms generally require a balanced distribution of various categories in datasets. However, the traditional classification algorithms often encounter an imbalanced class distribution in real life. The existing data- and classifier-level methods that attempt to solve this problem based on different perspectives exhibit some disadvantages, including the selection of parameters that have to be handled carefully and additional computing power because of repeated sampling. To solve these disadvantages, a method that can adaptively maintain the loss balance of examples in a mini-batch is proposed. This algorithm uses a dynamic loss-learnt function to adjust the loss ratio of each sample based on the information present in the label in every mini-batch, thereby achieving a balanced total loss for each class. The experiments conducted using the caltech101 and ILSVRC2014 datasets denote that this algorithm can effectively reduce the computational cost, improve the classification accuracy, and avoid the overfitting risk of the model that can be attributed to the oversampling method.
Key words: imbalanced learning    imbalanced data classification    imbalanced multi-classification    loss balance    classification algorithm for imbalanced data    imbalanced dataset    F1 measure    convolutional neural networks    deep learning

1 损失平衡函数 1.1 交叉熵损失函数

 ${\rm{CE}}(\theta ) = - \sum\limits_{i = 1}^n {\mathop y\nolimits_i } \log \overline {\mathop y\nolimits_i }$ (1)

1.2 改进的损失平衡函数

 ${\rm{PE}}(\theta ) = - \sum\limits_{i = 1}^n {\mathop y\nolimits_i } \log \frac{1}{{\left| t \right|}}\overline {\mathop y\nolimits_i }$ (2)
 ${\rm{LE}}(\theta ) = - \sum\limits_{i = 1}^n {\mathop y\nolimits_i } \frac{1}{{\left| t \right|}}\log \overline {\mathop y\nolimits_i }$ (3)

 Download: 图 1 小批量内各类别在损失函数中占据的比重 Fig. 1 The loss proportion of each class in a mini-batch
2 实验过程与结果分析 2.1 数据集及预处理

2.2 实验结果分析

 Download: 图 2 不同算法在各数据集上正确率及其标准差 Fig. 2 The accuracy and standard deviation of different algorithms for each dataset

 Download: 图 3 不同算法在数据集ILSVRC PART2上各阶段正确率 Fig. 3 The accuracy of different algorithms in each epoch with respect to the ILSVRC PART2 dataset
3 结束语

 [1] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444. DOI:10.1038/nature14539 (0) [2] GU Jiuxiang, WANG Zhenhua, KUEN J, et al. Recent advances in convolutional neural networks[J]. Pattern recognition, 2018, 77:354–377. (0) [3] JEATRAKUL P, WONG K W, FUNG C C. Using misclassification analysis for data cleaning[C]//Proceedings of International Workshop on Advanced Computational Intelligence and Intelligent Informatics. Tokyo, Japan, 2009: 297−302. (0) [4] BATISTA G E A P A, PRATI R C, MONARD M C. A study of the behavior of several methods for balancing machine learning training data[J]. ACM SIGKDD explorations newsletter, 2004, 6(1): 20-29. DOI:10.1145/1007730 (0) [5] XIAO Jianxiong, HAYS J, EHINGER K A, et al. SUN database: Large-scale scene recognition from abbey to zoo[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, USA, 2010: 3485−3492. (0) [6] GRZYMALA-BUSSE J W, GOODWIN L K, GRZYMALA-BUSSE W J, et al. An approach to imbalanced data sets based on changing rule strength[M]//PAL S K, POLKOWSKI L, SKOWRON A. Rough-Neural Computing. Berlin, Heidelberg: Springer, 2004: 543−553. (0) [7] JAPKOWICZ N, STEPHEN S. The class imbalance problem: A systematic study[J]. Intelligent data analysis, 2002, 6(5): 429-449. DOI:10.3233/IDA-2002-6504 (0) [8] MORENO-TORRES J G, HERRERA F. A preliminary study on overlapping and data fracture in imbalanced domains by means of Genetic Programming-based feature extraction[C]//Proceedings of the 201010th International Conference on Intelligent Systems Design and Applications. Cairo, Egypt, 2014: 501−506. (0) [9] WANG K J, MAKOND B, CHEN Kunhuang, et al. A hybrid classifier combining SMOTE with PSO to estimate 5-year survivability of breast cancer patients[J]. Applied soft computing, 2014, 20: 15-24. DOI:10.1016/j.asoc.2013.09.014 (0) [10] CHAWLA N V, BOWYER K W, HALL L O, et al. SMOTE: synthetic minority over-sampling technique[J]. Journal of artificial intelligence research, 2002, 16(1): 321-357. (0) [11] KOPLOWITZ J, BROWN T A. On the relation of performance to editing in nearest neighbor rules[J]. Pattern recognition, 1981, 13(3): 251-255. DOI:10.1016/0031-3203(81)90102-3 (0) [12] CATENI S, COLLA V, VANNUCCI M. A method for resampling imbalanced datasets in binary classification tasks for real-world problems[J]. Neurocomputing, 2014, 135: 32-41. DOI:10.1016/j.neucom.2013.05.059 (0) [13] ELKAN C. The foundations of cost-sensitive learning[C]//Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence. San Francisco, USA, 2001: 973−978. (0) [14] ZHOU Zhihua, LIU Xuying. Training cost-sensitive neural networks with methods addressing the class imbalance problem[J]. IEEE transactions on knowledge and data engineering, 2006, 18(1): 63-77. DOI:10.1109/TKDE.2006.17 (0) [15] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 1: 2999-3007. (0) [16] GOODFELLOW I, BENGIO Y, COURVILLE A. Deep learning[M]. Cambridge, massachusetts: MIT press, 2016: 218−227. (0) [17] LI Feifei, FERGUS R, PERONA P. Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories[J]. Computer vision and image understanding, 2007, 106(1): 59-70. DOI:10.1016/j.cviu.2005.09.012 (0) [18] RUSSAKOVSKY O, DENG Jia, SU Hao, et al. ImageNet large scale visual recognition challenge[J]. International journal of computer vision, 2015, 115(3): 211-252. DOI:10.1007/s11263-015-0816-y (0) [19] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas, NV, USA, 2016: 770−778. (0) [20] ESPÍNDOLA R P, EBECKEN N F F. On extending F-measure and G-mean metrics to multi-class problems[M]//ZANASI A, BREBBIA C A, EBECKEN N F F. Data Mining VI Data Mining, Text Mining and Their Business Applications. Southampton: WIT Press, 2005, 25−34. (0)