Protocol-Aware Anomaly Detection for Transport and Network Layer Attacks: A Comprehensive Review and Intelligent Framework Design
DOI:
https://doi.org/10.51483/IJAIML.6.8s.2026.781-797Keywords:
Intrusion Detection System (IDS), Cyber Threat Intelligence, Protocol-Aware Anomaly Detection, TCP Flooding Attacks, UDP Amplification Attacks, ICMP Abuse Detection, Lightweight Deep Learning, CNN-GRU Hybrid Model, Real-Time Network Security, Au-tomated MitigationAbstract
The threats are more complex in modern networks such as IoT systems, cloud infras-tructures, and distributed corporate networks. Among certain transport and network based protocols like TCP, UDP, and ICMP, have been used in flooding, amplification and spoof-ing attacks; in particular, it is the transport layer that is the primary target. Traditional intrusion detection systems generally depend on signature-based or flow-level analysis, which can leave out details of protocol-level anomalies. This paper introduces a protocol-aware anomaly detection framework based on packet-level analysis instead of aggregated traffic flows. Specifically, this system combines feature extraction from TCP, UDP, and ICMP headers with a lightweight hybrid deep learning model architecture using CNN and GRU architectures. This design allows spatial- and temporal-scale detection of network traffic patterns.
