Vision-based and marker-less surgical tool detection and tracking: a review of the literature

In recent years, tremendous progress has been made in practice for example with Minimally Invasive Surgery (MIS). To overcome challenges coming from deported eye-to-hand manipulation, robotic and computer-assisted systems have been developed. Having real-time knowledge of the pose of tools with respect to the camera and underlying anatomy is a key ingredient for such systems. In this paper, we present a review of the literature dealing with vision-based and marker-less tool detection. This paper includes three primary contributions: (1) identification and analysis of data-sets used for developing and testing algorithms, (2) in-depth comparison of tool methods from the feature extraction process to the model learning strategy and highlight existing shortcomings, and (3) analysis of validation techniques employed to obtain performance results and establish comparison between tool detectors. The papers included in the review were selected through PubMed and Google Scholar searches using the keywords: surgical tool detection, surgical tool tracking, surgical instrument detection and surgical instrument tracking limiting results to the year range 2000 2015. Our study shows that despite significant progress over the years, the lack of established tool data-sets, and reference format for performance assessment and method ranking is preventing faster improvement.