Aim: Electrocardiographic (ECG) monitoring is becoming increasingly commonplace, with modern ambulatory devices allowing long-term, continuous capture from a broad population. To develop robust algorithms for automated diagnosis and characterisation of medical conditions such as ventricular tachycardia (VT), researchers require high-quality annotations. These annotations typically need to be provided by human experts. Currently there is a lack of freely available, high-quality software to enable collaborative, human annotation of physiological waveforms such as ECG.
Method: We developed a software platform to support collaborative, expert annotation of physiological waveforms. The software enables experts to quickly annotate waveform records using a standard web browser. The software is simple to install, following best practice in Python packaging, and it offers a range of features, including: user management and task customization; a programmatic interface for data import and export; and a leaderboard for annotation progress tracking. Using the platform, we carried out a pilot study to assess the quality of 1,980 VT alarms from several commercial hospital monitors. Four expert cardiac clinicians were recruited and randomly assigned batches of alarms to annotate.
Results: Our pilot project demonstrated the utility of the annotation software and provided important feedback for improvement. Interestingly, of the 1,980 VT alarms in the source data, only 23% were considered "true” VT events by the human experts.
Conclusion: We developed a flexible, generalizable, web-based platform to enable multiple users to collaboratively annotate physiological waveforms. In a pilot study using the platform, we found that a significant proportion of VT alarms in commercial platforms were false positives.