Feedforward beta control in the KSTAR tokamak by deep reinforcement learning

In this work, we address a new feedforward control scheme of the normalized beta (βN) in tokamak plasmas, using the deep reinforcement learning (RL) technique. The deep RL algorithm optimizes an artificial decision-making agent that adjusts the discharge scenario to obtain the given target βN, from the state-action-reward sets explored by trials and errors of itself in the virtual tokamak environment. The virtual environment for the RL training is constructed with the LSTM network that imitates the plasma responses by external actuator controls, which is trained from 5-year KSTAR experimental data. Then, the RL agent experiences tons of discharges with different actuator controls in the LSTM simulator, and its internal parameters are optimized in the direction of maximizing the reward. We analyze a series of KSTAR experiments conducted with the RL-determined scenarios to validate the feasibility of the beta control scheme in a real device. We discuss the successes and limitations of the feedforward beta control by RL, and suggest our future works about it.