Increasing interest in wearable devices has led to an associated rise in the volume of electrocardiogram (ECG) data being collected outside of controlled clinical environments. These data are more susceptible to noise, meaning accurate assessment of signal quality is particularly important. However, there is limited research on the consistency of publicly-available signal quality indices (SQI) for ECGs. We evaluate the performance of several publicly available SQIs by assessing the quality of synthetic ECG signals with varying categories and levels of noise.
We used an existing framework to generate realistic ECG signals. We examined the impact of independently changing heart rate, power line interference, white noise, and motion artifacts on the outputs of four SQIs. ECG signals were generated at the threshold of acceptable and unacceptable outputs from each SQI across four categories of noise. The 16 signals were then evaluated by a cardiologist based on four specific criteria. SQI outputs (acceptable/unacceptable) were compared against the cardiologist's assessment.
The four SQIs were inconsistent with each other. For instance, the most conservative SQI determined an ECG to be unacceptable if heart rate exceeded 155 bpm, whereas the least conservative deemed heart rates up to 925 bpm to be acceptable. SQIs frequently disagreed with the cardiologist assessment. When the cardiologist was asked if the ECG could be used to ‘estimate a plausible heart rate', the SQIs agreed with the cardiologist in between 9/16 and 15/16 cases. When asked if the ECG was ‘clinically useful', agreement was much lower – between 4/16 and 10/16.
These results suggest that the SQIs tested here have limited suitability for clinical applications beyond identifying ECG signals with acceptable quality for heart rate extraction. Findings from this study highlight the need for users to critically evaluate the outputs of these SQIs and assess their suitability for comprehensive clinical application.