SwinCell: a transformer-based framework for dense 3D cellular segmentation

Abstract Segmentation of three-dimensional (3D) cellular images is fundamental for studying and understanding cell structure and function. However, 3D cellular segmentation is challenging, particularly for dense cells and tissues. This challenge arises mainly from the complex contextual information within 3D images, anisotropic properties, and the sensitivity to internal cellular structures, which often lead to miss-segmentation. In this work, we introduce SwinCell, a 3D transformer-based framework that leverages Swin-transformer for flow prediction and effectively distinguishes individual cell instances in 3D. We demonstrate the broad utility of the SwinCell in the segmentation of nuclei, colon tissue cells, and dense cultured cells. SwinCell strikes a balance between maintaining detailed local feature recognition and understanding broader contextual information. Tested extensively with both public and in-house 3D cell imaging datasets, SwinCell shows superior performance in segmenting dense cells in 3D, making it a powerful 3D segmentation tool for cellular analysis that could expedite research in cell biology and tissue engineering.