Metaheuristic-based ensemble learning: an extensive review of methods and applications

Abstract Ensemble learning has become a cornerstone in various classification and regression tasks, leveraging its robust learning capacity across disciplines. However, the computational time and memory constraints associated with almost all-learners-based ensembles necessitate efficient approaches. Ensemble pruning, a crucial step, involves selecting a subset of base learners to address these limitations. This study underscores the significance of optimization-based methods in ensemble pruning, with a specific focus on metaheuristics as high-level problem-solving techniques. It reviews the intersection of ensemble learning and metaheuristics, specifically in the context of selective ensembles, marking a unique contribution in this direction of research. Through categorizing metaheuristic-based selective ensembles, identifying their frequently used algorithms and software programs, and highlighting their uses across diverse application domains, this research serves as a comprehensive resource for researchers and offers insights into recent developments and applications. Also, by addressing pivotal research gaps, the study identifies exploring selective ensemble techniques for cluster analysis, investigating cutting-edge metaheuristics and hybrid multi-class models, and optimizing ensemble size as well as hyper-parameters within metaheuristic iterations as prospective research directions. These directions offer a robust roadmap for advancing the understanding and application of metaheuristic-based selective ensembles.