Near-instantaneous Cardiovascular Event Prediction Using Multimodal Deep Learning
Maytham Kareem Naeem Al-Hasooni, Amenah Y. Abdzaid, Noor. H Hadi, and Hassan Falah Fakhruldeen
This study introduces a new perspective into deep learning in the light of a multimodal approach: cardiovascular events can be predicted, using real-time data of physiological signals in collaboration with metadata related to the patient. Electronic Health Records (EHR) are digital versions of patients’ medical histories, while Multilayer Perceptron (MLP) and Convolutional Neural Network (CNN) are deep learning architectures designed for processing structured data and spatial/temporal patterns, respectively. A hybrid neural network model is designed that allows taking, as input from the CNN, the 12-lead ECG signals, while an MLP processes patient demographic and clinical features. It is designed to simultaneously process temporal ECG patterns and static patient characteristics for all-rounded cardiovascular risk assessment. In this work, our dataset consisted of 17,441 ECG recordings per patient, each being a 12-channel signal sampled on 500-time points and patient metadata like age, sex, and weight. Our architecture has two specialised components: the proposed SignalCNN to process the waveforms including two convolutional layers with batch normalization and dropout as regularization and MetaMLP processing patient metadata. These combined features are then fed into a classifier to enable multi-label prediction of five common cardiovascular conditions. The model yielded very promising results and performed very robustly with an overall validation accuracy of 85.19% after 15 epochs of training. The training was improving smoothly for both training and validation metrics, while the validation loss decreased from 0.4298 to 0.3484, which is indicative of good generalization. The model was very stable in its
training without showing any hint of overfitting thanks to strategic dropout and batch normalization. This work will contribute to cardiovascular healthcare with a real-time, automated system that can be used for the early detection of cardiac events. The approach is multimodal, offering more nuanced predictions by including instantaneous
physiological signals, together with patient-specific factors. This may enable earlier and more accurate clinical assessment of cardiovascular risk.