Reasoning discriminative dictionary-embedded network for fully automatic vertebrae tumor diagnosis

Fully automatic vertebrae tumor diagnosis (FAVTD) means using an end-to-end network to directly perform vertebrae recognition and tumor diagnosis from MRI images. FAVTD is clinically crucial for tumor screening and treatment, which helps prevent further metastasis and save the patients' lives. However, FAVTD has not yet been fully attempted due to the challenges raised by tumor appearance variability as well as MRI image field of view (FOV) and/or characteristics diversity. We propose a REasoning DiscriminativE diCtIonary-embeDded nEtwork (RE-DECIDE) to tackle the challenges in FAVTD. RE-DECIDE contains an elaborated enhanced-supervision recognition network (ERN) and a self-adaptive reasoning diagnosis network (SRDN). ERN is implemented in a feed-forward dictionary learning manner, which encodes each vertebra by the sparse codes and uses the sparse projections of the vertebrae coordinates onto multiple observation axes for supervision. ERN thus provides multiple sparse encodings of all vertebrae (and their ground truths) to enhance supervision, which reinforces the discrimination of different vertebrae and thus improves recognition performance. SRDN first highlights the most informative feature in the recognized vertebrae based on an attention mechanism. It then performs feature interaction, i.e., exchanges features of different vertebrae based on the graph reasoning mechanism. A reasoning controlling strategy is designed to prompt feature interaction in vertebrae with the same diagnosis labels and meanwhile reduces that in vertebrae with different labels, which avoids over-smoothing and improves diagnosis performance. RE-DECIDE is trained and evaluated using a challenging dataset consisting of 600 MRI images; the evaluation results show that RE-DECIDE achieves high performance in both recognition (accuracy: 0.940) and diagnosis (AUC: 0.947) tasks.