A SMOTE-Optimized RoBERTa Stacking Framework for Multiclass Cardiovascular Risk Prediction

Abstract

Predicting multiclass cardiovascular risk from the diagnostic text data is complex because of class imbalance, weak feature representation, and reliance on individual classifiers. The current research aims to develop an improved system that combines RoBERTa-enriched embeddings, SMOTE, Optuna optimization, and stacking ensemble learning techniques. For the experiment, 21,430 samples from the PTB-XL dataset were used. Each record is labeled according to the five possible classes: NORM, MI, STTC, CD, and HYP, divided into train and test parts with 80/20 ratio. The diagnostic text is produced from SCP statements, and then it is represented with the concatenation of 768-dimensional CLS embedding and 768-dimensional mean-pooled RoBERTa embedding to form 1536-dimensional vectors. SMOTE is used for class balancing, and Optuna optimizes XGBoost and LightGBM models. The Results show that the framework provides 99.68% accuracy, precision, recall, and F1-score, along with 100% weighted ROC-AUC. In addition to this, the number of inaccurate predictions made by the existing WPSA-DRF framework was 243, while it became 1.