Early Detection of Cardiovascular Disease Through Symptom-Based Text Mining: A Pure Natural Language Processing Approach Using the MIMIC-III Public Dataset

Abstract

Despite being reduced, cardiovascular disease (CVD) is still the leading cause of death in the world, and misrecognition of early warning signs, which includes chest discomfort, shortness of breath, palpitations and lack of energy, persist leading to poor outcomes. Although there are techniques for imaging and biomarker-based screening, these methods are expensive, unavailable for many, and the first and most common source of information about a patient's condition is the free-text clinical narrative in primary-care settings and in low-resource areas. This paper suggests a novel image-free,text-based, non-invasive method for early detection of the Cardiovascular Diseases (CVD) based entirely on Natural Language Processing (NLP) methods used on the unstructured Clinical Notes (CNs). We use the publicly available MIMIC-III critical care database to model a two-stage pipeline, a Symptom Extraction and Weighting (SEW) algorithm that extracts and weights cardiovascular symptom mentions from free text using a lexicon-based named entity recognition approach and Term Frequency-Inverse Document Frequency (TF-IDF) statistics, and a Risk Scoring and Classification (RSC) algorithm that combines the weighted symptom evidence in free text via a guideline-based clinical weight table, and uses a sigmoid-based decision function to output a meaningful Low/High/Moderate CVD risk class. Both algorithms are intentionally formulated in a simple closed-form mathematical way to enforce a low-calculability cost, to make the pipeline fully interpretable and without the need for GPU infrastructure for deployment. The Proposed pipeline achieved competitive accuracy of 87.6%, an AUROC of 0.889, and an F1-score of 0.85, and has significantly lower inference time and memory usage, not only when compared to the considerably heavier transformer-based baselines, but also and most notably, when compared to the results on the held-out test set of 8,432 discharge summaries and progress notes on MIMIC-III. The results suggest that the well-designed and purely statistical/rule based NLP pipelines are still worthwhile and explainable for early and low-cost, symptom-based cardiovascular risk screening in practice.