End-to-end 3D neuroendoscopic video reconstruction for robot-assisted ventriculostomy

Purpose: Navigating deep-brain structures in neurosurgery, especially under deformation from CSF egress, remains challenging due to the limitations of current robotic systems relying on rigid registration. This study presents the initial steps towards vision-based navigation leveraging Neural Radiance Fields (NeRF) to enable 3D neuroendoscopic reconstruction on the Robot-Assisted Ventriculoscopy (RAV) platform. Methods: An end-to-end 3D reconstruction and registration method using posed images was developed and integrated with the RAV platform. The hyperparameters for training the dual-branch network were first identified. Further experiments were conducted to evaluate reconstruction accuracy using projected error (PE) while varying the volume density threshold parameter. Results: A 3D volume was reconstructed using a simple linear trajectory for data acquisition with 300 frames and corresponding camera poses. The density volume threshold was varied to obtain an optimal value of 96.55 percentile, with a corresponding PE of 0.65 mm. Conclusions: Initial methods for end-to-end neuroendoscopic video reconstruction were developed in phantom studies. Experiments identified the optimal parameters, yielding a geometrically accurate reconstruction along with fast network convergence runtime of < 30 s. The method is highly promising for future clinical translation in realistic neuroendoscopic scenes. Future work will also develop a direct surface-to-volume registration method for improving reconstruction accuracy and runtime.