Visual-Inertial Image-Odometry Network (VIIONet): A Gaussian process regression-based deep architecture proposal for UAV pose estimation

• An end-to-end architecture is designed that makes UAV pose estimation based on optical flow-inertial-visual features. • Inertial numeric data is converted to inertial image to extract the features of inertial data with Inception-v3. • The noise of the IMU data is reduced by the Savitzky–Golay filter. • Probabilistic based Gaussian Process Regression is applied for final pose estimation. This study estimates the pose of Unmanned Aerial Vehicle (UAV) through artificial intelligence-based approaches and combines visual-inertial information in a different way than previous studies. For an effective fusion, the inertial data between both frames is normalized after denoising with the Savitzky-Golay technique and finally converted from numerical value to image. To strengthen these inertial image features with the change of motion between two frames, frames of Optical Flow (OF) are obtained and OF frames are combined with inertial images. Simultaneously, a parallel thread combines this OF frame with two consecutive raw frames. After features are extracted from inertial and camera data via Inception-v3, these features are fused and actual UAV poses are estimated via Gaussian Process Regression (GPR). Thanks to the smoothing process applied to these estimated values, a more stable pose estimation is provided. This proposed method is applied to the EuRoC dataset and our dataset produced in the Gazebo environment. The pose estimation results reveal that the proposed method has high performance compared to many previous studies.