Automatic and real-time green screen keying

Green screen keying has always been an essential and fundamental part of film and television special effects. In the actual shooting process, captured green screen images vary significantly due to the comprehensive influence of lighting, shooting angle, green cloth material, characters, etc. In order to obtain visually pleasing effects, traditional methods usually require professionals to adjust the corresponding parameters with lots of workload for different images, which is inefficient. Meanwhile, there are tedious steps in dealing with the green spill problem. In this paper, we propose a deep learning-based green screen keying method which is automatic, effective, and real-time. Firstly, we create a green screen dataset that contains not only alpha and foreground maps but also samples with green spill phenomenon and a large number of distinct green screen backgrounds. Secondly, we propose an end-to-end network that can automatically tackle green screen keying and green spill removal problems. Our method only takes a single image as input without user interaction and estimates alpha and foreground simultaneously. Extensive experiments clearly demonstrate the superiority of our proposed method. Moreover, our method achieves approximately 75fps on 720P videos (1280 $$\times $$ 720) and 25fps on 1080P videos (1920 $$\times $$ 1080), which can be considered real-time for many applications.