Key Frame Extraction from Videos Based on SIFT and Structural Similarity

With the advancement of high-resolution cameras, a large number of videos are captured using mobile devices. Extracting meaningful information from such videos is important for video summarization, fast archival, and storage. Key frames are the basic elements of videos that represent the salient features of a video stream. Due to the large volume, complex structure, and redundancy in the frames, key frame extraction is applied as a pre-processing work for analyzing any video content and compressing a video. In this work, a new key frame extraction technique is presented that uses the scale-invariant feature transform (SIFT) along with local features of the image like frame difference, and structural similarity (SSIM) of two consecutive frames. Candidate key frames are computed independently using the SSIM and SIFT techniques. Using suitable thresholds on SSIM and SIFT-based matching techniques, the key frames are computed. The qualitative and quantitative analysis has been carried out to evaluate the proposed algorithm on a state-of-the-art video dataset, and the experimental results show the efficacy of the algorithm proposed.