Flexible needle puncture path planning for liver tumors based on deep reinforcement learning

Abstract Objective. Minimally invasive surgery has been widely adopted in the treatment of patients with liver tumors. In liver tumor puncture surgery, an image-guided ablation needle for puncture surgery, which first reaches a target tumor along a predetermined path, and then ablates the tumor or injects drugs near the tumor, is often used to reduce patient trauma, improving the safety of surgery operations and avoiding possible damage to large blood vessels and key organs. In this paper, a path planning method for computer tomography (CT) guided ablation needle in liver tumor puncture surgery is proposed. Approach. Given a CT volume containing abdominal organs, we first classify voxels and optimize the number of voxels to reduce volume rendering pressure, then we reconstruct a multi-scale 3D model of the liver and hepatic vessels. Secondly, multiple entry points of the surgical path are selected based on the strong and weak constraints of clinical puncture surgery through multi-agent reinforcement learning. We select the optimal needle entry point based on the length measurement. Then, through the incremental training of the double deep Q-learning network (DDQN), the transmission of network parameters from the small-scale environment to the larger-scale environment is accomplished, and the optimal surgical path with more optimized details is obtained. Main results. To avoid falling into local optimum in network training, improve both the convergence speed and performance of the network, and maximize the cumulative reward, we train the path planning network on different scales 3D reconstructed organ models, and validate our method on tumor samples from public datasets. The scores of human surgeons verified the clinical relevance of the proposed method. Significance. Our method can robustly provide the optimal puncture path of flexible needle for liver tumors, which is expected to provide a reference for surgeons’ preoperative planning.