A Patient-Specific Mixed-Reality Visualization Tool for Thoracic Surgical Planning

Identifying small lung lesions during minimally invasive thoracic surgery can be challenging. We describe 3-dimensional mixed-reality visualization technology that may facilitate noninvasive nodule localization.A software application and medical image processing pipeline were developed for the Microsoft HoloLens to incorporate patient-specific data and provide a mixed-reality tool to explore and manipulate chest anatomy with a custom-designed user interface featuring gesture and voice recognition.A needs assessment between engineering and clinical disciplines identified the potential utility of mixed-reality technology in facilitating safe and effective resection of small lung nodules. Through an iterative process, we developed a prototype employing a wearable headset that allows the user to (1) view a patient's original preoperative imaging; (2) manipulate a 3-dimensional rendering of that patient's chest anatomy including the bronchial, osseus, and vascular structures; and (3) simulate lung deflation and surgical instrument placement.Mixed-reality visualization during surgical planning may facilitate accurate and rapid identification of small lung lesions during minimally invasive surgeries and reduce the need for additional invasive preoperative localization procedures.