An Unsupervised Approach for False Alarm Filtering in Rule-based NIDS

Abstract

To detect malicious activities in a network, intrusion detection systems are used. Even though these solutions are widely deployed for this purpose they have one serious shortcoming which is the huge amount of false alarms that they are generating. Different measures are taken to tackle this problem such as manually changing the settings of the intrusion detection systems. However, this is an infeasible approach for organisations since a network is changing regularly and specialists that have good knowledge of both the environment and the solution are required. The existing unsupervised approaches cannot be implemented as fully automated solutions because of the need to tune hyper-parameters. Additionally,the implementation of the existing solutions is complicated since often multiple models and the constant updating thereof is required which is a computationally intensive process considering the selected algorithms. In this work, the possibilities to reduce the false alarms in an automated manner are investigated. This is done by applying unsupervised anomaly detection techniques on the resulting alert data to distinguish regular alarms from high priority ones. Real alert data is collected from a network of a large organisation and an additional synthetically generated data set is used to evaluate the proposed approach. Four unsupervised anomaly detection algorithms are chosen to model the regular alerts. These are Local Outlier Factor (LOF), Isolation Forest (IF), Histogram-based Outlier Score(HBOS) and Cluster-based Local Outlier Factor (CBLOF). We show that this approach can greatly reduce the false alarms in real environments. By adding noise to the data we evaluate the performance of the models and propose a method that can be used to determine when the model needs to be retrained. This is done by deriving a metric that is used to trigger the system to automatically retrain on the most recent historic data. This is necessary in order to make the system automated and adaptable to changes in the network.