Surgical GAN: Towards real-time path planning for passive flexible tools in endovascular surgeries

Automatic surgical path planning of the passive flexible tool encounters a prohibitive challenge, typically in endovascular surgery (ES). The key problem is that unstructured surgical environment and tools' unpredictable motion is hard to be explicitly modeled. We propose a generative adversarial networks (GAN)-based framework (defined as surgical GAN) towards automatic guidewire path planning in real time for ES. A novel GAN architecture is proposed by combining convolutional neural networks (CNN) and long short-term memory networks (LSTM), which extracts and fuses the spatial features in medical images and temporal features of historical tool path as the conditional information. It inputs the surgical state information and continuously outputs the local future path of the guidewire tip. A training dataset (3.5*105 samples) is collected under laboratory conditions with 10 surgeons. Effects of different CNN architectures and path planning length on network performance are investigated. User experiments, with the tasks delivering the guidewire tip inside a vascular model (endovascular evaluator) from the aortic arch into the left common carotid artery (LCCA), left subclavian artery (LSCA), or brachiocephalic trunk, are conducted with 10 novice surgeons in an operating room. The results shows significant improvement of a path planning accuracy with surgical GAN compared with baseline networks (from 46.2%–69.78%) and the non-rigid registration method (72.94%). Results of user experiments demonstrate an overall better task performance with the guidance of planned surgical path. Collectively, surgical GAN can achieve real-time path planning of the guidewire in simulated ES, and holds great potential for novice training and robotic ES autonomy.