End-to-end Gesture Recognition Framework for the Identification of Allergic Rhinitis Symptoms

Human Gesture Recognition (HGR) using smart wearable IoT devices has emerged as a new field in human-centered computing regarding various domains. Though there are many research works related to data processing methodologies and Neural Networks architectures in this field, a lack of research on how to efficiently identify and interpret the AI models’ exports into human gestures is observed. This paper proposes an innovative end-to-end approach of how to solve and evaluate effectively a major part of HGR problems in a real-world scenario, in real-time. This is achieved with the effective utilization of data processing methods, the adoption, and extension of a cutting-edge Deep Learning model architecture, as well as the introduction and implementation in practice of innovative methods, both for interpretation and evaluation, that increase the trustworthiness of the model’s predictions.As a case study, we deployed the introduced pipeline into a real-world scenario of gestures’ identification and classification regarding allergic symptoms. We adopted multidisciplinarity by collaborating with recognized allergists that validated the whole approach in real patients via two pilot phases. As a result, by delivering a real-world application of our approach, we achieved a superior performance concerning the reliability of the pipeline, being 91.6% in our laboratory pilot phase and 81.4% in patients’ pilot data. Lastly, it is worth mentioning here that our framework can be employed in most HGR problems with minor modifications in data processing and learning procedure configuration.