GelTip tactile sensor for dexterous manipulation in clutter

Tactile sensing is an essential capability for robots that carry out dexterous manipulation tasks. While cameras, Lidars, and other remote sensors can assess a scene globally and instantly, tactile sensors can reduce their measurement uncertainties and gain information about the local physical interactions between the in-contact objects and the robot, which are often not accessible via remote sensing. Tactile sensors can be grouped into two main categories: electronic tactile skins and camera-based optical tactile sensors. The former are slim and can be fitted to different body parts, whereas the latter assume a more prismatic shape and have much higher sensing resolutions, offering a good advantage for being used as robotic fingers or fingertips. One of such optical tactile sensors is our GelTip sensor, which is shaped as a finger and can sense contacts on any location of its surface. As such, the GelTip sensor is able to detect contacts from all directions, like a human finger. To capture these contacts, it uses a camera installed at its base to track the deformations of the opaque elastomer that covers its hollow, rigid, and transparent body. Thanks to this design, a gripper equipped with GelTip sensors is capable of simultaneously monitoring contacts happening inside and outside its grasp closure. Experiments carried out using this sensor demonstrate how contacts can be localized, and more importantly, the advantages, and even possibly a necessity, of leveraging all-around touch sensing in dexterous manipulation tasks in clutter where contacts may happen at any location of the finger. In particular, experiments carried out in a Blocks World environment show that the detected contacts on the fingers can be used to adapt planned actions during the different moments of the reach-to-grasp motion. All the materials for the fabrication of the GelTip sensor can be found at https://danfergo.github.io/geltip/ .