Cross-scene foreground segmentation with supervised and unsupervised model communication

Abstract In this paper 1 , we investigate cross-scene video foreground segmentation via supervised and unsupervised model communication. Traditional unsupervised background subtraction methods often face the challenging problem of updating the statistical background model online. In contrast, supervised foreground segmentation methods, such as those that are based on deep learning, rely on large amounts of training data, thereby limiting their cross-scene performance. Our method leverages segmented masks from a cross-scene trained deep model (spatio-temporal attention model (STAM), pyramid scene parsing network (PSPNet), or DeepLabV3+) to seed online updates for the statistical background model (CPB), thereby refining the foreground segmentation. More flexible than methods that require scene-specific training and more data-efficient than unsupervised models, our method outperforms state-of-the-art approaches on CDNet2014, WallFlower, and LIMU according to our experimental results. The proposed framework can be integrated into a video surveillance system in a plug-and-play form to realize cross-scene foreground segmentation.