A Deep Q-Network for robotic odor/gas source localization: Modeling, measurement and comparative study

Abstract Robotic odor/gas source localization is a widely studied field, but most of the existing works are about rule-based algorithms. In this paper, the Deep Q-Network algorithm is applied to solve the odor source localization problem. An odor hits distribution model is proposed to model the odor concentration distribution in indoor environments, taking the dispersion by airflow, the odor molecular random walk, and the obstacles into account. The Deep Q-Network takes the stacked historic measurement data as the input and outputs the expected cumulative future reward of actions of the robots. The network is trained through 35,000 repeated episodes of random odor source localization tasks. The Deep Q-Network method is evaluated under four different environment settings in a simplified simulator and compared with two widely used odor source localization algorithms. The evaluation results demonstrate the advantages of the proposed algorithm. The algorithm is also validated in more complex indoor environments.