Bi-objective multi-mode resource-constrained multi-project scheduling using combined NSGA II and Q-learning algorithm

Multi-mode resource-constrained multi-project scheduling problem (MRCMPSP) plays a pivotal role in project management, serving as a critical component in production management for Engineering-to-Order manufacturing companies to enhance productivity, reduce costs, and minimize project completion time. This paper investigates the challenging problem of a bi-objective MRCMPSP, considering resource and finish time constraints, and develops a mathematical model to reduce project cycles and achieve better resource load balancing. By leveraging on the independent nature of start time selection for each activity, which aligns with the characteristics of a Markov decision process, we propose a two-layer iterative algorithm that combines the Nondominated Sorting Genetic Algorithm II (i.e., NSGA II) and Q-learning algorithm to solve the model effectively. Hence, the NSGA II algorithm generates mode combinations, while its fitness function employs the Q-learning algorithm to search for optimal activity time selections within each mode. We verify the performance superiority of the proposed algorithm by conducting a comparative analysis against classical approaches, encompassing classical NSGA II, Particle Swarm Optimization, and Ant Colony Optimization algorithms. Furthermore, this study's experimental results therefore unequivocally demonstrate the effectiveness of our algorithm in achieving optimized project scheduling outcomes.