Sensor Fusion of Leap Motion Controller and Flex Sensors Using Kalman Filter for Human Finger Tracking

In our daily life, we, human beings use our hands in various ways for most of our day-to-day activities. Tracking the position, orientation, and articulation of human hands has a variety of applications including gesture recognition, robotics, medicine and health care, design and manufacturing, and art and entertainment across multiple domains. Out of the various tracking methods, vision-based tracking is an efficient and widely used method. Several devices have been developed by researchers and engineers to track objects using vision. The leap motion (LM) controller is one such device. However, visual tracking is an equally complex and challenging task due to several factors like higher dimensional data from hand motion, higher speed of operation, and self-occlusion. This paper puts forth a novel method for tracking the fingertips of human hand using two distinct sensors and combining their data by sensor fusion technique. The proposed method is tested using standard human hand gestures, and the results are discussed. Finally, a soft robotic gripper was operated remotely based on LM hand tracking and the proposed sensor fusion method.