Intelligent video anomaly detection and classification using faster RCNN with deep reinforcement learning model

Abstract Recently, intelligent video surveillance applications have become essential in public security by the use of computer vision technologies to investigate and understand long video streams. Anomaly detection and classification are considered a major element of intelligent video surveillance. The aim of anomaly detection is to automatically determine the existence of abnormalities in a short time period. Deep reinforcement learning (DRL) techniques can be employed for anomaly detection, which integrates the concepts of reinforcement learning and deep learning enabling the artificial agents in learning the knowledge and experience from actual data directly. With this motivation, this paper presents an Intelligent Video Anomaly Detection and Classification using Faster RCNN with Deep Reinforcement Learning Model, called IVADC-FDRL model. The presented IVADC-FDRL model operates on two major stages namely anomaly detection and classification. Firstly, Faster RCNN model is applied as an object detector with Residual Network as a baseline model, which detects the anomalies as objects. Besides, deep Q-learning (DQL) based DRL model is employed for the classification of detected anomalies. In order to validate the effective anomaly detection and classification performance of the IVADC-FDRL model, an extensive set of experimentations were carried out on the benchmark UCSD anomaly dataset. The experimental results showcased the better performance of the IVADC-FDRL model over the other compared methods with the maximum accuracy of 98.50% and 94.80% on the applied Test004 and Test007 dataset respectively.