Breast mass classification with transfer learning based on scaling of deep representations

Abstract Ultrasound (US) imaging is widely used to help radiologists in diagnosing breast cancer. In this work, we propose a deep learning based approach to breast mass classification in US. Transfer learning with convolutional neural networks (CNNs) is commonly used to develop object recognition models in medical image analysis. The most widely used fine-tuning techniques aim to modify weights of pre-trained networks to address target medical problems. However, fine-tuning can be difficult when the number of trainable parameters of the pre-trained network is large and the available medical data are scarce. To address this issue, we propose a novel transfer learning technique based on deep representation scaling (DRS) layers, which are inserted between the blocks of a pre-trained CNN to enable better flow of information in the network. During network training, we only update the parameters of the DRS layers in order to adjust the pre-trained CNN to process breast mass US images. We present that the DRS based approach greatly reduces the number of trainable parameters, and achieves better or comparable performance to the standard transfer learning techniques. The proposed DRS layer method combined with the standard fine-tuning techniques achieved excellent breast mass classification performance, with area under the receiver operating characteristic curve of 0.955 and accuracy of 0.915.