Fitness function evaluations: A fair stopping condition?

It has become acceptable practice to use only a limit on the number of fitness function evaluations (FEs) as a stopping condition when comparing population-based optimization algorithms, irrespective of the initial number of candidate solutions. This practice has been advocated in a number of competitions to compare the performance of population-based algorithms, and has been used in many articles that contain empirical comparisons of algorithms. This paper advocates the opinion that this practice does not result in fair comparisons, and provides an abundance of empirical evidence to support this claim. Empirical results are obtained from application of a standard global best particle swarm optimization (PSO) algorithm with different swarm sizes under the same FE computational limit, on a large benchmark suite.