Embedding Instability Score for Detecting Backdoor Attacks in NLP Models

Abstract

Data poisoning attacks secretly tamper with training data to plant hidden backdoors in machine learning models, causing targeted misbehavior without degrading clean performance. Standardard accuracy checks fail to expose these corruptions, creating a major security risk. To solve this problem, this paper introduces the Embedding Instability Score (EIS), a novel detection framework that explicitly measures embedding value shifts by calculating the layer-wise Jensen-Shannon Divergence (JSD) between the pairwise distance distributions of clean and poisoned model embeddings. EIS is tested across three transformer architectures (BERT, DistilBERT, and RoBERTa) and three NLP datasets namely IMDB, Yelp Polarity, and SST-2 under poison rates of 1%, 2%, 5%, and 10%. While poisoned models maintain clean accuracy with a maximum drop of 0.0089%, with Attack Success Rates near 100%, existing defenses like STRIP, ONION, and Spectral Signatures show limited effectiveness of near random guessing (0.5 – 0.7). In contrast, EIS achieves an detection performance with an AUC of 1.000, a 100% Detection Rate, and a 0% False Positive Rate (FPR). By isolating changes in embeddings in intermediate layers, EIS effectively mitigates the L12 paradox. It also establishes embedding space analysis as a robust defense against backdoor attacks.