A critical review of the techniques used for anomaly detection of HTTP-based attacks: taxonomy, limitations and open challenges

Intrusion Detection Systems (IDSs) and Web Application Firewalls (WAFs) offer a crucial layer of defense that allows organizations to detect cyberattacks on their web servers. Academic research overwhelmingly suggests using anomaly detection techniques to improve the performance of these defensive systems. However, analyzing and comparing the wide range of solutions in the scientific literature is challenging since they are typically presented as isolated (unrelated) contributions, and their results cannot be generalized. We believe that this impairs the industry’s adoption of academic results and the advancement of research in this field. This paper aims to shed light on the literature on anomaly-based detection of attacks that use HTTP request messages. We define a novel framework for anomaly detection based on six data processing steps grouped into two sequential phases: preprocessing and classification. Based on this framework, we provide a taxonomy and critical review of the techniques surveyed, emphasizing their limitations and applicability. Future approaches should take advantage of the syntax and semantics of the Uniform Resource Locator (URL), be scalable, and address their obsolescence. These aspects are frequently overlooked in the literature and pose a significant challenge in the current era of web services. For better comparability, authors should use adequate public datasets, follow a thorough methodology, and use appropriate metrics that fully show the pros and cons of the approach.