Paper is not Dead: Automated Conversion and Analysis of Printed ECG Using Object Detection, Attention, and Siamese Neural Networks

Jan Pavlus1, Kristyna Pijackova2, Veronika Vicianova1, Petr Nejedly3, Filip Plesinger1
1Institute of Scientific Instruments of the CAS, 2Institute of Scientific Instruments of the CAS, v. v. i., 3Institute of Scientific Instruments of the Czech Academy of Science


Abstract

This study presents the ISIBrno-AIMT team's approach to addressing the George B. Moody PhysioNet Challenge 2024. The solution devised for the challenge's digitalization task involves a sequential application of three neural networks: lead detection, classification, and digitalization. For lead detection, a pre-trained Faster R-CNN with a ResNet50 backbone is utilized, finetuned to the task. A simple convolutional neural network (CNN) is employed for lead classification, trained to categorize cropped lead names. Digitalization of the leads is performed by a neural network comprising 2D convolutional layers and gated recurrent units (GRU). The pipeline initially identifies bounding boxes encompassing lead signals and their corresponding names. Subsequently, the lead names are cropped and classified, while the lead signals are extracted using the detected bounding boxes and subsequently digitalized by the third network. In the final step, the lead names and signals are linked via bounding box intersection, completing the digitalization process. In this task, our team achieved a score of -10.18 SNR.

The classification task is addressed through the utilization of the winning model from the PhysioNet/Computing in Cardiology Challenge 2021 (2021 model), which integrates CNN layers and an attention mechanism. In this article, we introduce two methodologies to tackle this task. The initial approach involves utilizing digitalized signals obtained from the preceding task, which are then subjected to pathology classification using a pretrained and fine-tuned 2021 model. Conversely, the second approach substitutes the CNN component of the 2021 model with twelve Siamese digitalization models from the first task. Here, the final layer of the digitalization model is extracted, and embeddings from the GRU layer are harnessed for pathology classification through the integration of an attention mechanism. We did not submit an implementation of this task in the unofficial phase.