Netspot: A Simple Intrusion Detection System with Statistical Learning

Machine learning is nowadays increasingly used in cyber-security. While intrusion detection was mainly based on human expertise in the 1990s, learning models to predict attacks are now built from data. However, a large part of the developed learning algorithms hitherto has missed real-world issues, making them unpractical. Indeed, many supervised algorithms described in the literature have been trained and tuned only on the KDD99 dataset. Besides, these algorithms are often static and are unable to automatically adapt for detecting attacks depending on the network traffic. Consequently, we are far from detecting zero-day or more general Advanced Persistent Threats (APT) since only pre-registered and well-characterized attacks can be catched. Some recent systems use unsupervised ML algorithms, but the resulting tools are overly complex: many ML components are stacked with various tuning parameters, usually making the results hard to interpret. And finally, a strong ML/DM expertise is required to set up these systems on real networks. We present netspot, a very simple network intrusion detection system (NIDS) powered by SPOT, a recent streaming statistical anomaly detector. This statistical test uses Extreme Value Theory, which is a powerful method for detecting anomalies. Unlike all the previous works, it is not an end-to-end solution aimed to detect all cyber-attacks with packet resolution. It is rather a module providing a behavioral information which can be integrated in a more general monitoring system. netspot is simple: it has few (simple) parameters, it adapts along time to the monitored network and it is as fast as current rule-based methods. But most importantly, it is able to detect realworld cyber-attacks, making it a credible practical anomaly-based NIDS.