Enhanced robotic tactile perception with spatiotemporal sensing and logical reasoning for robust object recognition

Since tactile sensing provides rich and delicate sensations, touch-based object recognition has attracted public attention and has been extensively developed for robots. However, robotic grasping recognition in real-life scenarios is highly challenging due to the complexity of real-life objects in shapes, sizes, and other details, as well as the uncertainty of real grabs in orientations and locations. Here, we propose a novel robotic tactile sensing method, utilizing the spatiotemporal sensing of multimodal tactile sensors acquired during hand grasping to simultaneously perceive multi-attributes of the grasped object, including thermal conductivity, thermal diffusivity, surface roughness, contact pressure, and temperature. Multimodal perception of thermal attributes (thermal conductivity, diffusivity, and temperature) and mechanical attributes (roughness and contact pressure) greatly enhance the robotic ability to recognize objects. To further overcome the complexity and uncertainty in real-life grasping recognition, inspired by human logical reasoning “from easy to hard” in solving puzzles, we propose a novel cascade classifier using multilayered long short-term memory neural networks to hierarchically identify objects according to their features. With the enhanced multimodal perception ability of tactile sensors and the novel cascade classifier, the robotic grasping recognition achieves a high recognition accuracy of 98.85% in discriminating diverse garbage objects, showing excellent generalizability. The proposed spatiotemporal tactile sensing with logical reasoning strategy overcomes the difficulty of robotic object recognition in complex real-life scenes and facilitates its practical applications in our daily lives.