Fast Recurrent Field Transforms for Optical Flow on Edge GPUs

Abstract Optical flow reflects motion information in videos, serving as a crucial visual cue. However, high computational demands make dense optical flow estimation challenging to deploy on mobile platforms. We address this by introducing EdgeFlow, an optical flow network designed explicitly for edge GPUs. To enable efficient feature extraction, we propose a novel Low Memory Access Cost Net (LMAC-Net). Furthermore, we design a Convolutional Gated Core (CGC) for optical flow update, capable of obtaining high-quality features at low computational cost. During training and inference, we decouple the upsampling procedure. Inference speed is increased by only upsampling the final update outputs, reducing unnecessary computations. Our model demonstrates significant efficiency, utilizing only 52% of the parameters required by the baseline model. In generalization tests conducted on the Sintel dataset, EdgeFlow outperforms the baseline by achieving an 13% reduction in error rate. Additionally, the processing speed of EdgeFlow deployed on the Jetson Xavier NX platform is 7.5 times faster than the baseline. Finally, after using TensorRT to accelerate the model (with 3 recurrent updates), the inference speed is 98.67 FPS at 640×480 resolution. Source code is available at https://github.com/sklibra/EdgeFlow/tree/master.