An Attention-based Audio-visual Fusion Method for Short Video Classification

The short multimedia video has become one of the most representative products in the new media era. Because short videos are highly time-sensitive, it is essential to manage and classify them efficiently. Currently, the primary approaches for short video classification are to extract the image features and then make judgments to complete the classification. However, audio information, crucial for classification tasks, is often discarded or used separately. To this end, we propose an attention-based audio-visual fusion method for short video classification. In this method, the attention module calculates the magnitude of the influence of image and audio information on the classification result, and the image and audio information are fused for the short video classification. The experimental results on different short multimedia video datasets demonstrate that the proposed attention-based audio-visual fusion method is effective and can significantly improve the classification accuracy of short videos.