Joint instancewise and instance-union fusion for improving motion detection algorithms

Motion detection (MD) is a fundamental step in many advanced computer vision applications, but the various complex challenges in real surveillance videos lead to some false positives and false negatives in the detection results of traditional MD algorithms. Therefore, joint instancewise and instance-union fusion for improving MD algorithms, in which an instance segmentation model is combined with a traditional MD algorithm. are proposed to address this problem. First, for each input frame (indexed by t), the MD algorithm produces a binary mask Mt, and the instance segmentation model produces the specific categories of binary instance masks (BIMs). Second, according to the instance confidence, BIMs are divided into high-quality binary instance masks (HBIMs) and low-quality binary instance masks (LBIMs). Then instancewise fusion of HBIMs with Mt and instance-union fusion of LBIMs with Mt are used to generate a high-quality foreground segmentation mask DtH and a low-quality foreground segmentation mask DtL, respectively. Finally, the bitwise logic addition operation of DtH and DtL produces a more accurate foreground segmentation result than Mt, called Dt. The experimental results show that our proposed method with visual background extractor and YOLACT++ processes at a resolution of 320 × 240 videos at 30 frames per second. For the Changedetection.net -2014, SBM-RGBD, and labeled and annotated sequences for integral evaluation of segmentation algorithms datasets, the highest overall F-measure of our experimental results with our proposed method are 0.8454, 0.8094, and 0.8939, respectively, surpassing state-of-the-art unsupervised MD methods.