Mirror Extreme Learning Machine Based on Denoising Autoencoder

doi:10.13438/j.cnki.jdzk.2020.05.005

Abstract

Abstract: A mirror extreme learning machine (MELM) is designed aiming at the problem that a large number of hidden layer nodes are needed to guarantee the classification performance of the extreme learning machine(ELM) in the high-dimensional and noisy data set. The MELM uses the pseudo-inverse method to determine the input weights, and randomly generates output weights and biases. When classifying data, it requires only a few hidden layer nodes. In order to improve the classification performance and noise immunity of the MELM, it is combined with a denoising autoencoder (DAE). The DAE is used to extract features from the input data, and the extracted features are used as the input data of the MELM, and finally the network training is triained. In the noiseless and noisy MNIST, Fashion MNIST, Rectangles and Convex data sets, the MELM based on the DAE is compared with the algorithms ELM, PCA-ELM, SAA-2 and DAE-ELM. The experimental results show the superiority of the algorithm: the MELM based on the DAE uses the least number of hidden nodes in the network for classification, and the overall performance is the best.

Key words: mirror, extreme learning machine, deep leaning, denoising autoencoder, feature extraction

LONG Qiuqing, LIAO Bolin, YIN Yumin. Mirror Extreme Learning Machine Based on Denoising Autoencoder[J]. Journal of Jishou University(Natural Sciences Edition), 2020, 41(5): 19-25.

References

[1] HUANG Guangbin, ZHU Qinyu, SIEW CHEE KHEONG. Extreme Learning Machine: A New Learning Scheme of Feedforward Neural Networks[C]∥ Proceedings of 2004 IEEE International Joint Conference on Neural Networks. Budapest: IEEE, 2004: 985-990.
[2] 丁李,曾水玲,熊涛.基于BP神经网络的湘西民间方块苗文特征提取方法[J].吉首大学学报(自然科学版),2018,39(5):14-18.
[3] 徐睿,梁循,齐金山,等.极限学习机前沿进展与趋势[J].计算机学报, 2019,42(7):1640-1670.
[4] CHYZHYK DARYA, SAVIO ALEXANDRE, GRANA MANUEL. Computer Aided Diagnosis of Schizophrenia on Resting State FMRI Data by Ensembles of ELM[J]. Neural Networks, 2015, 68:23-33.
[5] HUANG Zhiyong, YU Yuanlong, GU JASON, et al. An Efficient Method for Traffic Sign Recognition Based on Extreme Learning Machine[J]. IEEE Transactions on Cybernetics, 2016, 47(4): 920-933.
[6] 王光华,李素梅,朱丹,等.极端学习机在立体图像质量客观评价中的应用[J].光电子·激光,2014,25(9):1837-1842.
[7] KASUN LIYANAARACHCHI LEKAMALAGE CHAMARA, ZHOU Hongming, HUANG Guangbin. Representational Learning with ELMs for Big Data[J]. Intelligent Systems, IEEE, 2013, 28(6): 31-34.
[8] TANG Jiexiong, DENG Chenwei, HUANG Guangbin. Extreme Learning Machine for Multilayer Perceptron[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(4): 809-821.
[9] 来杰,王晓丹,李睿,等.基于去噪自编码器的极限学习机[J].计算机应用,2019,39(6):1619-1625.
[10] HUANG Guangbin, ZHU Qinyu, SIEW CHEE KHEONG. Extreme Learning Machine: Theory and Applications[J]. Neurocomputing, 2006, 70(1-3): 489-501.
[11] HUANG Guangbin, ZHOU Hongming, DING Xiaojian, et al. Extreme Learning Machine for Regression and Multiclass Classification[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2012, 42(2): 513-529.
[12] FENG Guorui, HUANG Guangbin, LIN Qingping, et al. Error Minimized Extreme Learning Machine with Growth of Hidden Nodes and Incremental Learning[J]. IEEE Transactions on Neural Networks, 2009, 20(8): 1352-1357.
[13] LIANG Nanying, HUANG Guangbin, SARATCHANDRAN P, et al. A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks[J]. IEEE Transactions on Neural Networks, 2006, 17(6): 1411-1423.
[14] LI Shuai, YOU Zhuhong, GUO Hongliang, et al. Inverse-Free Extreme Learning Machine with Optimal Information Updating[J]. IEEE Transactions on Cybernetics, 2015, 46(5): 1229-1241.
[15] HUANG Zhiyong, YU Yuanlong,GU JASON, et al. An Efficient Method for Traffic Sign Recognition Based on Extreme Learning Machine[J]. IEEE Transactions on Cybernetics, 2016, 47(4): 920-933.
[16] CAO Jiuwen, LIN Zhiping, HUANG Guangbin. Composite Function Wavelet Neural Networks with Extreme Learning Machine[J]. Neurocomputing, 2010, 73(7-9): 1405-1416.
[17] WEN Hui, FAN Hongguang, XIEWeixin, et al. Hybrid Structure-Adaptive RBF-ELM Network Classifier[J]. IEEE Access, 2017, 5: 16539-16554.
[18] DENG Jun, FRUHHOLZ SASCHA, ZHANG Zixing, et al. Recognizing Emotions from Whispered Speech Based on Acoustic Feature Transfer Learning[J]. IEEE Access, 2017, 5: 5235-5246.
[19] AKUSOK ANTON, BJORK KAJ-MIKAEL, MICHE YOAN, et al. High-Performance Extreme Learning Machines: A Complete Toolbox for Big Data Applications[J]. IEEE Access, 2015, 3: 1011-1025.
[20] WANG Xinying, HAN Min. Online Sequential Extreme Learning Machine with Kernels for Nonstationary Time Series Prediction[J]. Neurocomputing, 2014, 145(5): 90-97.
[21] ZHAI Junhai, XU Hongyu, WANG Xizhao. Dynamic Ensemble Extreme Learning Machine Based on Sample Entropy[J]. Soft Computing, 2012, 16(9): 1493-1502.
[22] ZHANG Yunong, YU Xiaotian, GUO Dongsheng, et al. Weights and Structure Determination of Multiple-Input Feed-Forward Neural Network Activated by Chebyshev Polynomials of Class 2 via Cross-Validation[J]. Neural Computing and Applications, 2014, 25(7-8): 1761-1770.
[23] WANG Guanghua, LI Sumei,ZHU Dan, et al. Application of Extreme Learning Machine in Objective Stereoscopic Image Quality Assessment[J]. Journal of Optoelectronics· Laser, 2014, 25(9): 1837-1842.
[24] 范亚超,罗天健,周昌乐.基于降噪自编码器特征学习的作者识别及其在《西游记》诗词上的应用[J].厦门大学学报(自然科学版),2018,57(6):884-889.