基于去噪自编码器的镜像极限学习机设计

doi:10.13438/j.cnki.jdzk.2020.05.005

摘要/Abstract

摘要：针对极限学习机在高维度、含噪声数据集中需要大量隐含层节点来保证分类性能的问题，设计了镜像极限学习机.该算法使用伪逆法确定输入权值，随机生成输出权值和偏置，在对数据进行分类时，它仅需极少的隐含层节点.为了提升镜像极限学习机的分类性能和抗噪性，将它与去噪自编码器相结合.利用去噪自编码器对输入数据进行特征提取，并将提取到的特征作为镜像极限学习机的输入数据，再进行网络训练.在无噪和含噪声的MNIST，Fashion MNIST，Rectangles和Convex数据集中，将基于去噪自编码器的镜像极限学习机与ELM，PCA-ELM，SAA-2和DAE-ELM作对比实验，结果表明，基于去噪自编码器的镜像极限学习机的综合性能最优，用于分类的网络隐含层节点数最少.

关键词： 镜像, 极限学习机, 深度学习, 去噪自编码器, 特征提取

Abstract: A mirror extreme learning machine (MELM) is designed aiming at the problem that a large number of hidden layer nodes are needed to guarantee the classification performance of the extreme learning machine(ELM) in the high-dimensional and noisy data set. The MELM uses the pseudo-inverse method to determine the input weights, and randomly generates output weights and biases. When classifying data, it requires only a few hidden layer nodes. In order to improve the classification performance and noise immunity of the MELM, it is combined with a denoising autoencoder (DAE). The DAE is used to extract features from the input data, and the extracted features are used as the input data of the MELM, and finally the network training is triained. In the noiseless and noisy MNIST, Fashion MNIST, Rectangles and Convex data sets, the MELM based on the DAE is compared with the algorithms ELM, PCA-ELM, SAA-2 and DAE-ELM. The experimental results show the superiority of the algorithm: the MELM based on the DAE uses the least number of hidden nodes in the network for classification, and the overall performance is the best.

Key words: mirror, extreme learning machine, deep leaning, denoising autoencoder, feature extraction

龙求青, 廖柏林, 印煜民. 基于去噪自编码器的镜像极限学习机设计[J]. 吉首大学学报（自然科学版）, 2020, 41(5): 19-25.

LONG Qiuqing, LIAO Bolin, YIN Yumin. Mirror Extreme Learning Machine Based on Denoising Autoencoder[J]. Journal of Jishou University(Natural Sciences Edition), 2020, 41(5): 19-25.

参考文献

[1] HUANG Guangbin, ZHU Qinyu, SIEW CHEE KHEONG. Extreme Learning Machine: A New Learning Scheme of Feedforward Neural Networks[C]∥ Proceedings of 2004 IEEE International Joint Conference on Neural Networks. Budapest: IEEE, 2004: 985-990.
[2] 丁李,曾水玲,熊涛.基于BP神经网络的湘西民间方块苗文特征提取方法[J].吉首大学学报(自然科学版),2018,39(5):14-18.
[3] 徐睿,梁循,齐金山,等.极限学习机前沿进展与趋势[J].计算机学报, 2019,42(7):1640-1670.
[4] CHYZHYK DARYA, SAVIO ALEXANDRE, GRANA MANUEL. Computer Aided Diagnosis of Schizophrenia on Resting State FMRI Data by Ensembles of ELM[J]. Neural Networks, 2015, 68:23-33.
[5] HUANG Zhiyong, YU Yuanlong, GU JASON, et al. An Efficient Method for Traffic Sign Recognition Based on Extreme Learning Machine[J]. IEEE Transactions on Cybernetics, 2016, 47(4): 920-933.
[6] 王光华,李素梅,朱丹,等.极端学习机在立体图像质量客观评价中的应用[J].光电子·激光,2014,25(9):1837-1842.
[7] KASUN LIYANAARACHCHI LEKAMALAGE CHAMARA, ZHOU Hongming, HUANG Guangbin. Representational Learning with ELMs for Big Data[J]. Intelligent Systems, IEEE, 2013, 28(6): 31-34.
[8] TANG Jiexiong, DENG Chenwei, HUANG Guangbin. Extreme Learning Machine for Multilayer Perceptron[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(4): 809-821.
[9] 来杰,王晓丹,李睿,等.基于去噪自编码器的极限学习机[J].计算机应用,2019,39(6):1619-1625.
[10] HUANG Guangbin, ZHU Qinyu, SIEW CHEE KHEONG. Extreme Learning Machine: Theory and Applications[J]. Neurocomputing, 2006, 70(1-3): 489-501.
[11] HUANG Guangbin, ZHOU Hongming, DING Xiaojian, et al. Extreme Learning Machine for Regression and Multiclass Classification[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2012, 42(2): 513-529.
[12] FENG Guorui, HUANG Guangbin, LIN Qingping, et al. Error Minimized Extreme Learning Machine with Growth of Hidden Nodes and Incremental Learning[J]. IEEE Transactions on Neural Networks, 2009, 20(8): 1352-1357.
[13] LIANG Nanying, HUANG Guangbin, SARATCHANDRAN P, et al. A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks[J]. IEEE Transactions on Neural Networks, 2006, 17(6): 1411-1423.
[14] LI Shuai, YOU Zhuhong, GUO Hongliang, et al. Inverse-Free Extreme Learning Machine with Optimal Information Updating[J]. IEEE Transactions on Cybernetics, 2015, 46(5): 1229-1241.
[15] HUANG Zhiyong, YU Yuanlong,GU JASON, et al. An Efficient Method for Traffic Sign Recognition Based on Extreme Learning Machine[J]. IEEE Transactions on Cybernetics, 2016, 47(4): 920-933.
[16] CAO Jiuwen, LIN Zhiping, HUANG Guangbin. Composite Function Wavelet Neural Networks with Extreme Learning Machine[J]. Neurocomputing, 2010, 73(7-9): 1405-1416.
[17] WEN Hui, FAN Hongguang, XIEWeixin, et al. Hybrid Structure-Adaptive RBF-ELM Network Classifier[J]. IEEE Access, 2017, 5: 16539-16554.
[18] DENG Jun, FRUHHOLZ SASCHA, ZHANG Zixing, et al. Recognizing Emotions from Whispered Speech Based on Acoustic Feature Transfer Learning[J]. IEEE Access, 2017, 5: 5235-5246.
[19] AKUSOK ANTON, BJORK KAJ-MIKAEL, MICHE YOAN, et al. High-Performance Extreme Learning Machines: A Complete Toolbox for Big Data Applications[J]. IEEE Access, 2015, 3: 1011-1025.
[20] WANG Xinying, HAN Min. Online Sequential Extreme Learning Machine with Kernels for Nonstationary Time Series Prediction[J]. Neurocomputing, 2014, 145(5): 90-97.
[21] ZHAI Junhai, XU Hongyu, WANG Xizhao. Dynamic Ensemble Extreme Learning Machine Based on Sample Entropy[J]. Soft Computing, 2012, 16(9): 1493-1502.
[22] ZHANG Yunong, YU Xiaotian, GUO Dongsheng, et al. Weights and Structure Determination of Multiple-Input Feed-Forward Neural Network Activated by Chebyshev Polynomials of Class 2 via Cross-Validation[J]. Neural Computing and Applications, 2014, 25(7-8): 1761-1770.
[23] WANG Guanghua, LI Sumei,ZHU Dan, et al. Application of Extreme Learning Machine in Objective Stereoscopic Image Quality Assessment[J]. Journal of Optoelectronics· Laser, 2014, 25(9): 1837-1842.
[24] 范亚超,罗天健,周昌乐.基于降噪自编码器特征学习的作者识别及其在《西游记》诗词上的应用[J].厦门大学学报(自然科学版),2018,57(6):884-889.