Data-Driven Cybercrime Intelligence For Identifying Threat Patterns, Systemic Vulnerabilities, And Cybersecurity Resilience

Abstract

As cyberattacks become more sophisticated and numerous, it is essential to have intelligent frameworks that can identify unusual network activities and help to build a cybersecurity resilient system. This research introduces a novel data-driven cybercrime intelligence framework that combines Variational Autoencoder anomaly detection and LightGBM classification. In this study, a novel data-driven cybercrime intelligence framework is proposed, which is based on the combination of Variational Autoencoder anomaly detection and LightGBM classification. This proposed gated architecture is verified on the publicly available CICIDS2017 network attack traffic which includes attack traffic from various network types and benign traffic. The methodology involves the following data preprocessing, feature normalization, exploratory data analysis, unsupervised feature learning using variational autoencoder, sample routing based on confidence and attack classification using LightGBM. The results of the experiments showed that accuracy of the VAE model was 94.02% with ROC-AUC of 95.42% and the standalone lightGBM model resulted in accuracy of 99.82%, balanced accuracy of 99.63%, macro F1 score of 96.93%, ROC-AUC of 99.99%, and log loss of 0.0104. In the proposed gated framework, accuracy was found to be 94.86% and weighted F1-score was 94.13% while 89.73% of the samples were taken directly from the VAE, 10.27% needed refinement using LightGBM. The novelty of this work is that it combines the unsupervised representation learning with intelligent routing based on confidence, thus reducing the amount of computation while ensuring a good detection performance. The proposed framework provides an efficient, scalable, and interpretable solution for cyber threat intelligence, which can help to detect anomalies promptly and enhance the cybersecurity resilience in today's network environment.