Effectively fusing clinical knowledge and AI knowledge for reliable lung nodule diagnosis

Clinicians typically use semantic features to judge the malignant status of nodules, while artificial intelligence systems (AI) tend to extract unknown features to diagnose nodules. The former relies on clinical knowledge, while the latter explores AI knowledge. Although many studies indicate that fusing clinical and AI knowledge can help computer-aided diagnosis (CAD) systems improve diagnostic accuracy and gain clinician approval, how to effectively fuse them is still an open question. This paper proposes a simple and effective pipeline (abbreviated as CKAK), which fuses clinical and AI knowledge at both feature and decision levels for accurate lung nodule malignancy classification and semantic attributes characterization. The feature-level fusion can retain rich information in high-dimensional features and improve the model’s accuracy; the decision-level fusion can provide some interpretability for the model’s decision-making process, which is expected in clinical applications. Specifically, the proposed CKAK consists of two sequential stages: (i) the initial prediction stage (IPS); and (ii) the prediction refine stage (PRS). The IPS predicts eight radiologist-interpreted semantic attributes and an initial malignancy diagnosis in parallel. Then, these results are fed to the subsequent PRS to refine the diagnosis further by fully fusing them at feature and decision levels. Besides, to enhance the ability of feature learning, we propose a novel scale-aware feature extraction block (SAFE). It integrates multi-scale contextual features with a lightweight Transformer rather than adding or concatenating them roughly. Extensive experiments at the LIDC-IDRI data set show that the proposed CKAK can achieve superior benign-malignant classification accuracy with minor radiologist-interpreted semantic scores error, meeting the need for a reliable CAD system.