Hybrid Attention and Motion Constraint for Anomaly Detection in Crowded Scenes

Crowds often appear in surveillance videos in public places, from which anomaly detection is of great importance to public safety. Since the abnormal cases are rare, variable and unpredictable, autoencoders with encoder and decoder structures using only normal samples have become a hot topic among various approaches for anomaly detection. However, since autoencoders have excessive generalization ability, they can sometimes still reconstruct abnormal cases very well. Recently, some researchers construct memory modules under normal conditions and use these normal memory items to reconstruct test samples during inference to increase the reconstruction errors for anomalies. However, in practice, the errors of reconstructing normal samples with the memory items often increase as well, which makes it still difficult to distinguish between normal and abnormal cases. In addition, the memory-based autoencoder is usually available only in the specific scene where the memory module is constructed and almost loses the prospect of cross-scene applications. We mitigate the overgeneralization of autoencoders from a different perspective, namely, by reducing the prediction errors for normal cases rather than increasing the prediction errors for abnormal cases. To this end, we propose an autoencoder based on hybrid attention and motion constraint for anomaly detection. The hybrid attention includes the channel attention used in the encoding process and spatial attention added to the skip connection between the encoder and decoder. The hybrid attention is introduced to reduce the weight of the feature channels and regions representing the background in the feature matrix, which makes the autoencoder features more focused on optimizing the representation of the normal targets during training. Furthermore, we introduce motion constraint to improve the autoencoder’s ability to predict normal activities in crowded scenes. We conduct experiments on real-world surveillance videos, UCSD, CUHK Avenue, and ShanghaiTech datasets. The experimental results indicate that the prediction errors of the proposed method for frequent normal crowd activities are smaller than those of other approaches, which increases the gap between the prediction errors for normal frames and the prediction errors for abnormal frames. In addition, the proposed method does not depend on a specific scene. Therefore, it balances good anomaly detection performance and strong cross-scene capability.