Multimodal Deep Learning for Automated Anemia Screening and Hemoglobin Estimation Using Conjunctival Images

Abstract

Anemia is a condition affecting around 25% of the world's population but still undiagnosed in low-resourced countries due to limited blood testing facilities in such regions. Conjunctival images were captured using a smartphone to train a multi-modality deep-learning model that detects anemia and estimates hemoglobin level non-invasively. MM-AnemiaNet utilizes the convolutional feature extractor (here EfficientNet-B3) in association with the bright and colored conjunctival images as well as a lightweight MLP model that takes the patient's age, gender, and symptoms as inputs. This means that in MLP networks, cross-modal attention fusion methods are applied. The training of the model was based on the binary model for the diagnosis of the presence of anemia and the continuous model for hemoglobin estimation. The trained model performed better than its competitors by achieving a very high classification accuracy of 92.6% as well as 0.913F1 score, 0.96 AUC, and 0.98g/dl average error regarding 4218 conjunctival images from 1406 subjects. The ablation analysis shows that illumination normalization and cross-modal attention fusion increases the model’s statistical performance, whereas calibration analysis indicates that multi-task training enhances the probability calibration parameter compared to classification.