A Head-Mounted Assistive Device for Visually Impaired People with Warning System from Object Detection and Depth Estimation

Abstract People with visual impairment use white cane as their traditional method for perceiving the surroundings. However, the utilization of a cane is limited by its length and orientation. In Thailand, the obstacles on paveway in daily life are not located only on the floor but also above knee level which sometimes could be harmful to pedestrians, especially blind people. A head-mounted assistive device is developed to be an enhancement used with a cane for the visually impaired to comprehend their environment both lower and higher the knee level. The assistive device is designed to be compact and light-weight. It could also send the tactile feedback as a warning from vibration motors mounted on the device. To generate a warning signal, YOLOv4 is used to detect the location of obstacles and depth map from the stereo camera is used to estimate the distance mapping into 4 defined ranges: dangerous, very close, close and fine. The results indicate that the head-mounted assistive device has the ability to perceive obstacles locating farther than 0.9 m. The prediction returned 9.23%, 14.63% and 7.86% error when estimating the depth of obstacles at 1.3 m., 2.8 m. and 4.2 m. respectively. The average execution time for the device to return the command controlling vibration motors is 0.13 second and the maximum estimated time for the motor to send the haptic feedback is 1.05 second.