Artificial intelligence-based tissue segmentation and cell identification in multiplex-stained histological endometriosis sections

Abstract STUDY QUESTION How can we best achieve tissue segmentation and cell counting of multichannel-stained endometriosis sections to understand tissue composition? SUMMARY ANSWER A combination of a machine learning-based tissue analysis software for tissue segmentation and a deep learning-based algorithm for segmentation-independent cell identification shows strong performance on the automated histological analysis of endometriosis sections. WHAT IS KNOWN ALREADY Endometriosis is characterized by the complex interplay of various cell types and exhibits great variation between patients and endometriosis subtypes. STUDY DESIGN, SIZE, DURATION Endometriosis tissue samples of eight patients of different subtypes were obtained during surgery. PARTICIPANTS/MATERIALS, SETTING, METHODS Endometriosis tissue was formalin-fixed and paraffin-embedded before sectioning and staining by (multiplex) immunohistochemistry. A 6-plex immunofluorescence panel in combination with a nuclear stain was established following a standardized protocol. This panel enabled the distinction of different tissue structures and dividing cells. Artificial intelligence-based tissue and cell phenotyping were employed to automatically segment the various tissue structures and extract quantitative features. MAIN RESULTS AND THE ROLE OF CHANCE An endometriosis-specific multiplex panel comprised of PanCK, CD10, α-SMA, calretinin, CD45, Ki67, and DAPI enabled the distinction of tissue structures in endometriosis. Whereas a machine learning approach enabled a reliable segmentation of tissue substructure, for cell identification, the segmentation-free deep learning-based algorithm was superior. LIMITATIONS, REASONS FOR CAUTION The present analysis was conducted on a limited number of samples for method establishment. For further refinement, quantification of collagen-rich cell-free areas should be included which could further enhance the assessment of the extent of fibrotic changes. Moreover, the method should be applied to a larger number of samples to delineate subtype-specific differences. WIDER IMPLICATIONS OF THE FINDINGS We demonstrate the great potential of combining multiplex staining and cell phenotyping for endometriosis research. The optimization procedure of the multiplex panel was transferred from a cancer-related project, demonstrating the robustness of the procedure beyond the cancer context. This panel can be employed for larger batch analyses. Furthermore, we demonstrate that the deep learning-based approach is capable of performing cell phenotyping on tissue types that were not part of the training set underlining the potential of the method for heterogenous endometriosis samples. STUDY FUNDING/COMPETING INTEREST(S) All funding was provided through departmental funds. The authors declare no competing interests. TRIAL REGISTRATION NUMBER N/A.