Feature Robustness and Diagnostic Capabilities of Convolutional Neural Networks Against Radiomics Features in Computed Tomography Imaging

Introduction Radiomics and deep learning algorithms such as convolutional neural networks (CNNs) are increasingly used for radiological image classification and outcome prediction. One of the main challenges is to create robustness against technical alterations. Both methods initially extract specific imaging features, which are then used as input for machine learning algorithms or in an end-to-end fashion for outcome prediction. For radiomics features, it has previously been shown that differences in image acquisition parameters can cause variability in feature values, making them irreproducible. However, it remains unclear how these technical variations influence feature values extracted by a CNN. Therefore, the aim of this study was to compare the robustness of CNN features versus radiomics features to technical variations in image acquisition parameters. An additional retrospective analysis was performed to show the in vivo capabilities of these features compared with classical radiomics features in a tumor differentiation task. Materials and Methods Imaging phantoms were scanned twice on 3 computed tomography scanners from 2 different manufactures with varying tube voltages and currents. Phantoms were segmented, and features were extracted using PyRadiomics and a pretrained CNN. After standardization the concordance correlation coefficient (CCC), mean feature variance, feature range, and the coefficient of variant were calculated to assess feature robustness. In addition, the cosine similarity was calculated for the vectorized activation maps for an exemplary phantom. For the in vivo comparison, the radiomics and CNN features of 30 patients with hepatocellular carcinoma (HCC) and 30 patients with hepatic colon carcinoma metastasis were compared. Results In total, 851 radiomics features and 256 CNN features were extracted for each phantom. For all phantoms, the global CCC of the CNN features was above 98%, whereas the highest CCC for the radiomics features was 36%. The mean feature variance and feature range was significantly lower for the CNN features. Using a coefficient of variant ≤0.2 as a threshold to define robust features and averaging across all phantoms 346 of 851 (41%) radiomics features and 196 of 256 (77%) CNN features were found to be robust. The cosine similarity was greater than 0.98 for all scanner and parameter variations. In the retrospective analysis, 122 of the 256 CNN (49%) features showed significant differences between HCC and hepatic colon metastasis. Discussion Convolutional neural network features were more stable compared with radiomics features against technical variations. Moreover, the possibility of tumor entity differentiation based on CNN features was shown. Combined with visualization methods, CNN features are expected to increase reproducibility of quantitative image representations. Further studies are warranted to investigate the impact of feature stability on radiological image-based prediction of clinical outcomes.