Convolutional Neural Network Approach for South African Sign Language Recognition and Translation

Sign language is a natural, visually oriented, and non-verbal communication channel for the deaf and dumb community. However, not everyone understands sign language, particularly, individuals outside of the deaf-dumb community. This challenge has been addressed through the development of automatic sign language recognition (SLR) systems. Various SLR applications have been developed for English, Indian, Korean, Turkish, Arabic, and other sign languages. However, few studies have been conducted on South African SLR due to the lack of publicly available sign language datasets. In addition, the existing South African SLR systems face challenges in being conducted efficiently as a result of special equipment such as wearable data gloves needed for hand gesture recognition and light illumination complexity background challenges. This paper applies deep learning-based convolutional neural networks (CNNs) for South African SLR and classification. In this work, the CNN model was trained on 12420 images of 26 static South African sign language alphabets and 4050 validation datasets using the Gaussian blurring combined with adaptive threshold pre-processing techniques. The proposed model is embedded with a Google Translate application program interface (API) that translates the signed output into various South African official languages to ensure that sign language can be understood in various languages beyond English. The obtained results and comparative analysis demonstrate the efficiency of the proposed model with a weighted average of 98% accuracy, precision, recall, and F1-score outperforming the existing models in the literature.