Effect of update rule transition triggered by Q-learning algorithm in evolutionary prisoner's dilemma game involving extortion

Abstract Most studies have shown that the heterogeneity of update rules has an important impact on evolutionary game dynamics. In the meanwhile, Q-learning algorithm has gained attention and extensive study in evolutionary games. Therefore, a mixed stochastic evolutionary game dynamic model involving extortion strategy is constructed by combining imitation and aspiration-driven updating rules. During the evolution of the model, individuals will use the Q-learning algorithm which is a typical self-reinforcement learning algorithm to determine which update rule to adopt. Herein, through numerical simulation analyses, it is found that the mixed stochastic evolutionary game dynamic model affected by the Q-learning algorithm ensures the survival of cooperators in the grid network. Moreover, the cooperators cannot form a cooperation cluster in the grid network but will form a chessboard-like distribution with extortioners to protect cooperators from the invasion of defectors. In addition, a series of results show that, before the evolution turns into steady state, our model increases the number of nodes utilizing the average aspiration-driven update rule, thereby promoting the emergence of chessboard-like distribution. Overall, our study may provide some interesting insights into the development of cooperative behavior in the real world.