Aims: The aim of this work is to implement a network using local attention to solve the task of multi-class classification on the PhysioNet/Computing in Cardiology Challenge 2021 dataset. Most previous work in this challenge processed the 12 leads separately, and cumulated the outputs towards the end of the network. However, this approach is not effective as inter-lead features are not utilised at the early stages of the network. Also, processing all the leads with the same convolution may introduce noise and prevent effective model training as the wave morphologies are different between the ECG leads. To combat this, we introduce a novel local lead-attention to learn embeddings across a single lead as well as multiple leads.
Methods: We start by projecting the ECG signals using a shallow strided depthwise separable convolution block without overlap between the leads. The projected embeddings are then processed using the novel local lead-attention layers to learn lead-specific and variant features. To address the quadratic complexity of self-attention, we limit the receptive field of attention to only neighborhood values. The use of depthwise convolutional layers and local self-attention makes the model efficient and light-weight with only 2.4M parameters.
Results: Our model achieves competitive scores on the PhysioNet/CinC Challenge 2021 dataset using a transformer-based network. Our results show that the proposed network using local lead-attention can effectively process ECG signals for multi-class classification, which can learn lead-specific and variant features at all stages of the network.
Conclusion: In this paper, we have presented a novel approach using local lead-attention for ECG processing. This work opens up new avenues for attention-based models in ECG signal processing and can also be used as an effective feature extractor in future work.