Dexterity Operation, Multi-Modal Perception and Tactile Force Interaction of a Bionic Soft Hand

Existing soft hand mostly focus on mechanism and material innovation, as well as biological behavior imitation, which have problems of fewer sensing abilities and unfriendly human-robot interaction (HRI). In this paper, we propose a novel bionic soft hand (BSH) with dexterity operation, multi-modal perception and bidirectional HRI. For dexterity operation, we first designed and fabricated a dual-joint bellows (DJB) soft finger, and then, five fingers were integrated into BSH with inertial measurement units (IMUs), contact pressure sensors, curvature sensors and air pressure sensors distributed appropriately. For multi-modal perception, the control and perception system were established to drive BSH in pneumatic mode and process the multi-modal sensor information to perform robust perception capabilities using support vector machine (SVM), extreme learning machine (ELM) and BP neural network (BPNN). For bidirectional HRI, a data glove with force reproduction was utilized to perform posture mapping and tactile force mapping. The experimental results verify that the proposed BSH has a wide workspace range and dexterous operation ability, and can accurately recognize human posture and object properties. Also, hand posture mapping and tactile force mapping have been demonstrated Note to Practitioners —This work was motivated by the research status of existing soft hands that lack sensing abilities and friendly HRI, and aimed to develop a bionic soft hand (BSH) with dexterity operation, multi-modal perception and tactile force interaction. We increased the degrees of freedom of BSH by adopting a bellows structure to build a two-joint soft finger, enhanced the perception ability of BSH by utilizing multi-modal sensors and multiple machine learning algorithms, performed posture mapping and tactile force mapping for bidirectional HRI by designing a novel data glove with multi-modal sensors and a variable stiffness jamming layer. Experimental results verify that the proposed BSH has a wide workspace range and dexterous operation ability, and can accurately recognize human posture and object properties by multi-modal sensors. Future work will focus on imitating the joint structure of human hand and make BSH be closer to the dexterity and perception performance of human hand. Also introducing humanbrain-machine interface is a meaningful challenge.