A Comprehensive Comparative Benchmark of AI and XAI Techniques for CICIDS2017-Based Intrusion Detection

Abstract

With the increasing traffic on the networks and the complexity of threats, the machine learning algorithm-based Intrusion Detection Systems (IDS) have been developed. But it is actually yet a crucial problem to achieve both high detection accuracy and transparency of the model. In this paper, 12 AI classifiers ranging from traditional ML models (Decision Tree, Random Forest, XGBoost) to advanced boosting models (LightGBM, AdaBoost, CatBoost), and three XAI techniques (SHAP, LIME and TreeExplainer) are studied on the CICIDS2017 dataset and execution time analysis is the focus of this paper. The results show that the best performance is taken by the ensemble tree based models, with LightGBM being able to reach near perfect accuracy rate (99.55%) and Decision Tree is the fastest model (3 ms) for inference. The XAI analysis shows that while LIME takes 2.09 seconds for 10 samples, SHAP TreeExplainer for XGBoost only needs 0.24 seconds for explaining 500 samples. Our results are also compared to recent state‑of-the‑art works to confirm that our results are in line with previous ones and to shed light on some important future research avenues such as the high computational complexity of XAI for models other than trees, and the necessity for imbalance sensitive metrics. A unified benchmark and repeatable code that can inform researchers and practitioners of the deployment of AI‑based IDS with XAI in real world cybersecurity operations.