Mvar-Tef: A Modern Transformer-Based Model For Intrusion Detection Over An Industrial Environment

Abstract

Determining an unobserved signal as normal or abnormal is a critical task in many real-world applications, known as time series irregularity identification. Some dense learning models, like Transformers, are typically used to study a regular representation of data to distinguish between regular and unusual signals. Though no pure Transformer-based technique exists for temporal sequence anomaly identification, some investigations have recently used Transformers to analyze time series information. This study introduces the Multi-variate Transformer encoding function (MVar-TEF), a novel Transformer model and its successful training approach for improved feature learning. In particular, we introduce two-stage techniques to enhance our model's recognition capability. The first phase is random sample initialization, in which we create a pool of random feature to cover up random portions of the samples. As a result, our model can learn how to depict typical data. Phase two is an exclusive prediction based on the variation approach: To appropriately simulate the exclusive and uncertain elements in the first stage, we suggest a novel refining procedure that provides feedback. This work offers an empirical analysis by determining the usefulness of the prediction procedure, which reliably produces more signals that resemble normality. According to extensive testing on various datasets, MVar-TEF performs considerably better in time series anomaly identification than other modern techniques.