Performance assessment and exhaustive listing of 500+ nature-inspired metaheuristic algorithms

Metaheuristics are popularly used in various fields, and they have attracted much attention in the scientific and industrial communities. In recent years, the number of new metaheuristic names has been continuously growing. Generally, the inventors attribute the novelties of these new algorithms to inspirations from either biology, human behaviors, physics, or other phenomena. In addition, these new algorithms, compared against basic versions of other metaheuristics using classical benchmark problems, show competitive performances. However, many new metaheuristics are not rigorously tested on challenging benchmark suites and are not compared with state-of-the-art metaheuristic variants. Therefore, in this study, we exhaustively tabulate more than 500 metaheuristics. In particular, several representative metaheuristics are introduced from two aspects, namely, the inspirational source and the essential operators for generating solutions. To comparatively evaluate the performance of the state-of-the-art and newly proposed metaheuristics, 11 newly proposed metaheuristics (generally with high numbers of citations) and 4 state-of-the-art metaheuristics are comprehensively compared on the CEC2017 benchmark suite. For fair comparisons, a parameter tuning tool named irace is used to automatically configure the parameters of all 15 algorithms. In addition, whether these algorithms have a search bias to the origin (i.e., the center of the search space) is investigated. All the experimental results are analyzed by several nonparametric statistical methods, including the Bayesian rank-sum test, Friedman test, Wilcoxon signed-rank test, critical difference plot and Bayesian signed-rank test. Moreover, the convergence, diversity, and the trade-off between exploration and exploitation of these 15 algorithms are also analyzed. The results show that the performance of the newly proposed EBCM algorithm performs similarly to the 4 compared algorithms and has the same properties and behaviors, such as convergence, diversity, exploration and exploitation trade-offs, in many aspects. However, the other 10 recent metaheuristics are less efficient and robust than the 4 state-of-the-art metaheuristics. The performance of all 15 of the algorithms is likely to deteriorate due to certain transformations, while the 4 state-of-the-art metaheuristics are less affected by transformations such as the shifting of the global optimal point away from the center of the search space. It should be noted that, except EBCM, the other 10 new algorithms are inferior to the 4 state-of-the-art algorithms in terms of convergence speed and global search ability on CEC 2017 functions. Moreover, the other 10 new algorithms are rougher (i.e., present in their behavior with high oscillations) in terms of the trade-off between exploitation and exploration and population diversity compared with the 4 state-of-the-art algorithms. Finally, several important issues relevant to the metaheuristic research area are discussed and some potential research directions are suggested.