Student Engagement Prediction in E-Learning Environments Using Attention-Based Transformers

Abstract

Data regarding student interactions were gathered from Learning Management Systems that included login logs, assignment submissions, quizzes, attendance, and interactions in Discussion and Video. Data was preprocessed by mean imputation, Min-Max normalization, and one-hot encoding, before computing engagement scores by feature engineering the behavior. Using the Adam optimizer and a train/valid/test split of 70/15/15, an Attention-Based Transformer architecture with positional encoding, multi-head self-attention modules, and feed-forward layers is created and trained. To evaluate and compare the performance of the proposed model, the accuracy, precision, recall, F1-Score, RMSE, and MAE metrics were used and applied to the baseline models (LR, RF, SVM, RNN, and LSTM). The proposed transformer model achieved the lowest prediction error (RMSE 0.121 and MAE 0.103), accuracy 95.8%, precision 95.1%, recall 94.5%, and F1-Score 94.8% compared to all other models. It achieved an accuracy of 95.4%, which was 4.6%, 7.3% and 14.2% better than LSTM, RNN, and Logistic Regression, respectively. The accuracy of the classification of engagements was 96.7%, 94.8%, and 93.5% for high, moderate, and low levels of engagement, respectively. Loss was steadily reduced from 0.462 to 0.109 as training went on for 50 epochs. The ablation study verified that the multi-head attention and positional encoding are the most important parts, as their deletion resulted in a decrease in accuracy of up to 10%. The proposed Attention-Based Transformer framework is capable of effectively capturing temporal dependencies and sequential behavior, which shows its superior performance in predicting student engagement in e-learning environments. The framework offers educators an effective early intervention and personalized learning support tool and scalable solution for adaptive educational platforms.