Hybrid MPSO-CNN: Multi-level Particle Swarm optimized hyperparameters of Convolutional Neural Network

Recent advances in swarm inspired optimization algorithms have shown its extensive acceptance in solving a wide range of different real-world problems. Particle Swarm Optimization (PSO) is one of the most explored nature-inspired population-based stochastic optimization algorithm. In this paper, a Multi-level Particle Swarm Optimization (MPSO) algorithm is proposed to find the architecture and hyperparameters of the Convolutional Neural Network (CNN) simultaneously. This automated learning will reduce the overhead of human experts to find these parameters through manual analysis and experiments. The proposed solution uses multiple swarms at two levels. The initial swarm at level-1 optimizes architecture and multiple swarms at level-2 optimize hyperparameters. The proposed method has used sigmoid like inertia weight to adjust the exploration and exploitation property of particles and avoid the PSO algorithm to prematurely converge into a local optimum solution. In this paper, we have explored an approach to suggest the best well-conditioned CNN architecture and its hyperparameters using MPSO in a specified search space. The complexity and performance of MPSO-CNN will depend on the dimension of the search space. The experimental results on 5 benchmark datasets of MNIST, CIFAR-10, CIFAR-100, Convex Sets, and MDRBI have demonstrated one more effective application of PSO in learning a deep neural architecture.