Optimizing quantum annealing schedules with Monte Carlo tree search enhanced with neural networks

Quantum annealing is a practical approach to approximately implement the adiabatic quantum computational model in a real-world setting. The goal of an adiabatic algorithm is to prepare the ground state of a problem-encoded Hamiltonian at the end of an annealing path. This is typically achieved by driving the dynamical evolution of a quantum system slowly to enforce adiabaticity. Properly optimized annealing schedules often considerably accelerate the computational process. Inspired by the recent success of deep reinforcement learning such as DeepMind’s AlphaZero, we propose a Monte Carlo tree search (MCTS) algorithm and its enhanced version boosted by neural networks—which we name QuantumZero (QZero)—to automate the design of annealing schedules in a hybrid quantum–classical framework. Both the MCTS and QZero algorithms perform remarkably well in discovering effective annealing schedules even when the annealing time is short for the 3-SAT examples considered in this study. Furthermore, the flexibility of neural networks allows us to apply transfer-learning techniques to boost QZero’s performance. We demonstrate in benchmark studies that MCTS and QZero perform more efficiently than other reinforcement learning algorithms in designing annealing schedules. Quantum annealers are computational models implemented on quantum hardware that can efficiently solve combinatorial optimization problems. Annealing schedules with enhanced performance can be discovered with a Monte Carlo tree search algorithm and an enhanced version incorporating value and policy neural networks—as inspired by DeepMind’s AlphaZero.