Digital Diagnosis of Hand, Foot, and Mouth Disease Using Hybrid Deep Neural Networks

Hand, Foot and Mouth Disease (HFMD) is a highly contagious paediatric disease showing up symptoms like fever, diarrhoea, oral ulcers and rashes on the hands and foot, and even in the mouth. This disease has become an epidemic with several outbreaks in many Asian-Pacific countries with the basic reproduction number R0 > 1. HFMD's diagnosis is very challenging as its lesion pattern may appear quite similar to other skin diseases such as herpangina, aseptic meningitis, and poliomyelitis. Therefore, clinical symptoms are essential besides skin lesion's pattern and position for precise diagnose of this disease. A deep learning-based HFMD detection system can play a significant role in the digital diagnosis of this disease. Various machine learning and deep learning architectures have been proposed for skin disease diagnosis and classification. However, these models are limited to the image classification problem. The diagnosis of similar appearing skin diseases using the image classification approach may result in misclassification or misdiagnosis of the disease. Parallel integration of clinical symptoms and images can improve disease diagnosis and classification performance. However, no deep learning architecture has been developed to diagnose HFMD disease from images and clinical data. This paper has proposed a novel Hybrid Deep Neural Networks integrating Multi-Layer Perceptron (MLP) network and Convolutional Neural Network into a single framework for the diagnosis of HFMD using the integrated features from clinical and image data. The proposed Hybrid Deep Neural Networks is particularly a multi branched model comprising of Multi-Layer Perceptron (MLP) network in the first branch to extract the clinical features and the modified pre-trained CNN architecture: MobileNet or NasNetMobile in the second branch to extract the features from skin disease lesion images. The features learnt from both the branches are merged to form an integrated feature from clinical data and images, which is fed to the subsequent classification network. We conducted several experiments employing image data only, clinical data only and both sources of data. The analyses compared and evaluated the performance of a typical MLP model and CNN model with our proposed Hybrid Deep Neural Networks. The novel approach promotes the existing image classification model and clinical symptoms based disease classification model, particularly the MLP model. From the cross-validated experiments, the results reveal that the proposed Hybrid Deep Neural Networks can diagnose the disease 99%-100% accurately.