Extreme Learning Machine for Multilayer Perceptron Based on Multi-swarm Particle Swarm Optimization for Variable Topology

Extreme Learning Machine for Multilayer Perceptron(H-ELM) is a newly developing learning algorithm for the generalized multiple hidden layer feed-forward neural networks, of which training architecture is structurally divided into two separate phases: 1) unsupervised hierarchical feature representation and 2) supervised feature classification. However, due to its hand-designed network structure, researchers need to spend a lot of effort on adjusting the structure, which is an error-prone process. To solve the issue, in this paper, a novel fully connected neural network architecture search method based on particle swarm optimization (PSO) algorithm is proposed for H-ELM. The proposed algorithm framework is divided into two main parts: 1) Architecture search based on PSO algorithm and 2) Weight analysis based on H-ELM. The novelties of the paper are as follows: 1) Optimizing the structure of fully connected neural networks by using multi-swarm particle swarm optimization algorithms, and improve it so that the structures of different hidden layer numbers can learn from each other. 2) Minimum principle of structure: Minimize the total number of nodes in the resulting network while ensuring the accuracy of network evaluation. A large number of experiments on various widely used classification datasets show that the algorithm could achieve higher accuracy with more compact network structure than the optimal results in the randomly generated structures.