MISNeR: Medical Implicit Shape Neural Representation for Image Volume Visualisation

Abstract Three‐dimensional visualisation of mesh reconstruction of the medical images is commonly used for various clinical applications including pre / post‐surgical planning. Such meshes are conventionally generated by extracting the surface from volumetric segmentation masks. Therefore, they have inherent limitations of staircase artefacts due to their anisotropic voxel dimensions. The time‐consuming process for manual refinement to remove artefacts and/or the isolated regions further adds to these limitations. Methods for directly generating meshes from volumetric data by template deformation are often limited to simple topological structures, and methods that use implicit functions for continuous surfaces, do not achieve the level of mesh reconstruction accuracy when compared to segmentation‐based methods. In this study, we address these limitations by combining the implicit function representation with a multi‐level deep learning architecture. We introduce a novel multi‐level local feature sampling component which leverages the spatial features for the implicit function regression to enhance the segmentation result. We further introduce a shape boundary estimator that accelerates the explicit mesh reconstruction by minimising the number of the signed distance queries during model inference. The result is a multi‐level deep learning network that directly regresses the implicit function from medical image volumes to a continuous surface model, which can be used for mesh reconstruction from arbitrary high volume resolution to minimise staircase artefacts. We evaluated our method using pelvic computed tomography (CT) dataset from two public sources with varying z‐axis resolutions. We show that our method minimised the staircase artefacts while achieving comparable results in surface accuracy when compared to the state‐of‐the‐art segmentation algorithms. Furthermore, our method was 9 times faster in volume reconstruction than comparable implicit shape representation networks.