Optimizing the echo state network with a binary particle swarm optimization algorithm

作     者：Wang, Heshan Yan, Xuefeng 

作者机构：E China Univ Sci & Technol Minist Educ Key Lab Adv Control & Optimizat Chem Proc Shanghai 200237 Peoples R China 

出 版 物：《KNOWLEDGE-BASED SYSTEMS》 (知识库系统)

年 卷 期：2015年第86卷第Sep.期

页      面：182-193页

核心收录：

学科分类：08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

主　　题：Echo state network Reservoir computing Artificial neural network Binary particle swarm optimization algorithm Time-series prediction 

摘      要：The echo state network (ESN) is a novel and powerful method for the temporal processing of recurrent neural networks. It has tremendous potential for solving a variety of problems, especially real-valued, time-series modeling tasks. However, its complicated topologies and random reservoirs are difficult to implement in practice. For instance, the reservoir must be large enough to capture all data features given that the reservoir is generated randomly. To reduce network complexity and to improve generalization ability, we present a novel optimized ESN (O-ESN) based on binary particle swarm optimization (BPSO). Because the optimization of output weights connection structures is a feature selection problem and PSO has been used as a promising method for feature selection problems, BPSO is employed to determine the optimal connection structures for output weights in the O-ESN. First, we establish and train an ESN with sufficient internal units using training data. The connection structure of output weights, i.e., connection or disconnection, is then optimized through BPSO with validation data. Finally, the performance of the O-ESN is evaluated through test data. This performance is demonstrated in three different types of problems, namely, a system identification and two time-series benchmark tasks. Results show that the O-ESN outperforms the classical feature selection method, least angle regression (LAR) method in that its architecture is simpler than that of LAR. (C) 2015 Elsevier BM. All rights reserved.