physionet-logo-kp6The PhysioNet/CinC Challenge

– 2024 –

Digitization and Classification of ECG Images

2024 Challenge Summary

For the past 24 years, PhysioNet and Computing in Cardiology have co-hosted a series of annual challenges, now called the George B. Moody PhysioNet Challenges, to tackle clinically interesting questions that are either unsolved or not well-solved.

The goal of the 2024 Challenge is to develop algorithms for digitizing and classifying electrocardiograms (ECGs) captured from images or paper printouts.

Competition Background

While planning CinC 2000, local hosts Roger Mark and George Moody proposed to organize an activity that would make effective use of their newly-established PhysioNet web site to stimulate rapid progress on an unsolved problem of practical clinical significance. A timely contribution of data made it possible to create the first PhysioNet/CinC Challenge, which attracted the attention of more than a dozen teams to the subject of detecting sleep apnea from the ECG. Their efforts were broadly successful, they discussed their findings at CinC 2000, and an annual tradition was born.

PhysioNet offers free access via the web to large and growing collections of recorded physiologic signals and related open-source software. Originally established under the auspices of the NIH’s National Center for Research Resources, PhysioNet has been funded since September 2007 under a cooperative agreement with the NIH’s National Institute of Biomedical Imaging and Bioengineering (NIBIB), and with the NIH’s National Institute of General Medical Sciences (NIGMS).

In complementary ways, PhysioNet and Computing in Cardiology catalyze and support scientific communication and collaboration between basic and clinical scientists. The annual meetings of CinC are gatherings of researchers from many nations and disciplines, bridging the geographic and specialty chasms that separate understanding from practice, while PhysioNet provides on-line data and software resources that support collaborations of basic and clinical researchers throughout the year. The annual PhysioNet/CinC Challenges seek to provide stimulating yet friendly competitions, while at the same time offering both specialists and non-specialists alike opportunities to make progress on significant open problems whose solutions may be of profound clinical value. The use of shared data provided via PhysioNet makes it possible for participants to work independently toward a common objective. At CinC, participants can make meaningful results-based comparisons of their methods; lively and well-informed discussions are the norm at scientific sessions dedicated to these challenges. Discovery of the complementary strengths of diverse approaches to a problem when coupled with deep understanding of that problem frequently sparks new collaborations and opportunities for further study.

A new challenge topic is announced each year on the Challenge page at PhysioNet. The PhysioNet team assembles and posts the raw materials needed to begin work. Usually, these raw materials consist of a collection of data to be analyzed; the analyses are provided for a subset of the data (the “learning set”) in each case, and the challenge is to analyze the remaining data (the “test set”).

What will be the topic of the next challenge? It might be image analysis, or simulation, or forecasting…. An ideal challenge problem is interesting, clinically important, and possible to study using available materials that have not been widely circulated previously. Moreover, there must be an objective way to evaluate the quality of a challenge entry (for an analysis problem, this usually means there must be a known set of correct analyses of the data, i.e., a “gold standard” against which entries can be compared). You are invited to submit your ideas for future challenge topics to the Challenge organizer, Gari Clifford; suggestions accompanying relevant data are particularly welcome.

PhysioNet/CinC Challenge Awards

Challenges are open to all. An important milestone for participants is the deadline for submitting abstracts for CinC, which is 15 April each year. Those wishing to qualify as official entrants, with eligibility for awards, must submit an abstract describing their work as well as an entry for scoring by about one week before the abstract deadline. A limited number of revised entries may be submitted between the abstract deadline and the final challenge deadline in early September. Eligibility for awards also requires participants to present their work in a scientific session of CinC. See the Challenge pages on PhysioNet for deadlines and rules for this year’s competition.

Most Challenges are presented as two events (often a narrowly-defined question and a more general one), or sometimes more.

An award is offered for the best solution obtained by any eligible participant.  To qualify for this prize, the solution must include computer code in open-source format.  Each entry should contain a file that defines the specific open-source license under which the software is available. See the Challenge Web page for more details on the problem and the submission process for each year’s challenge.  The winners receive their awards during the final plenary session on Wednesday afternoon. Follow the links below for details about previous challenges.

PhysioNet/CinC Challenge Award Winners

2024: Digitization and Classification of ECG Images

Overall Challenge Compétition:

  • Digitization Winner 1st : Felix Krones, Ben Walker, Terry Lyons, Adam Mahdi. Combining Hough Transform and Deep Learning Approaches to Reconstruct ECG Signals From Printouts.
  • Digitization Winner 2nd : Hong-Cheol Yoon, Dong-Kyu Kim, Hyun-Seok Kim, Woo-Young Seo, Chang-Hoe Heo, Sung-Hoon Kim. Segmentation-based Extraction of Key Components from ECG Images: A Framework for Precise Classification and Digitization.
  • Digitization Winner 3rd : Mathilde A Verlyck, Joshua R Dillon, Stephen A Creame, Debbie Zhao. A Modular and Open-Source Python Implementation for Fully Automated Digitization of Paper Electrocardiograms Using Deep Learning.
  • Classification Winner 1st : Felipe M. Dias, Estela Ribeiro, Quenaz B.Soares, Jose E. Krieger, Marco A. Gutierrez. Image-Based Electrocardiogram Classification using Pre-trained ConvNext. 
  • Classification Winner 2nd : Chun-Ti Chou, Sergio Gonzalez. Dual Deep Learning System to Digitize and Classify 12-lead ECGs from Scanned Images.
  • Classification Winner 3rd : Hong-Cheol Yoon, Dong-Kyu Kim, Hyun-Seok Kim, Woo-Young Seo, Chang-Hoe Heo, Sung-Hoon Kim. Segmentation-based Extraction of Key Components from ECG Images: A Framework for Precise Classification and Digitization.

Hackathon Competition:

  • Winner Digitization : Samer Jammoul, Abdullatif Hassan, Emily Zhang, Philip Warrick, Jonathan Afilalo.
  • Winner Classification : Sara Summerton, Nicola Dinsdale, Tuija Leinonen, George Searle, Matti Kaisti, David C Wong.

Best Challenge Oral:

  • Mathilde A Verlyck, Joshua R Dillon, Stephen A Creame, Debbie Zhao. A Modular and Open-Source Python Implementation for Fully Automated Digitization of Paper Electrocardiograms Using Deep Learning.

Best Challenge Poster:

  • Rafael Silva, Yingyu Yang, Maelis Morier, Safaa Al-Ali, Maxime Sermesant. YOUR-Lead: YOLO and U-Net for Reconstruction of ECG Lead Signals.

Best Team Name :

  • Sara Summerton, Nicola Dinsdale, Tuija Leinonen, George Searle, Matti Kaisti, David C Wong.

Best title :

  • Clinton Mwangi, David Warutumo, Paul Bett, Mary Kariuki. Heart Disease Classification Using EfficientNetB5 with Three-Dimensional Scaled Electrocardiogram Images.

Best preprint (by deadline) :

  • Amaan Kazi, Kelvin Nguyen, Varun Sendilraj, Shadi Manafi Avari, Sasan Esfahani, Zaniar Ardalan, Saman Parvaneh. Fusion of Deep Learning and Rule-Based Techniques for Enhanced Paper-Based ECG Digitization.

2023: Predicting Neurological Recovery from Coma After Cardiac Arrest

Overall Challenge Compétition:

  • Winner 1st : Morteza Zabihi, Alireza Chaman Zar, Pulkit Grover, Eric Rosenthal. HyperEnsemble Learning from Multimodal Biosignals to Robustly Predict Functional Outcome after Cardiac Arrest
  • Winner 2nd : Dong-Kyu Kim, Hong-Cheol Yoon, Hyun-seok Kim, Woo-Yeong Seo, Sung-Hoon Kim. Predicting Neurological Outcome After Cardiac Arrest Using a Pretrained Model with Electroencephalography Augmentation
  • Winner 3rd : Hongliu Yang and Ronald Tetzlaff. Model Ensembling for Predicting Neurological Recovery after Cardiac Arrest: Top-down or Bottom-up?

Hackathon Competition:

  • Winner 1st : Dong-Kyu Kim, Hongcheol Yoon and Hyun-Seok Kim.
  • Winner 2nd : Richard Hohmuth, Marc Goettling and Maurice Rohr

Best Challenge Oral:

  • Charlotte Maschke. Functional outcome prediction after cardiac arrest using machine learning and network dynamics of resting-state electroencephalography

Best Challenge Poster:

  • Inês Sampaio. Predicting Comatose Patient’s Outcome Using Brain Functional Connectivity with a Random Forest Model

Best Team Name :

  • Philip Hempel, Philip Zaschke, Miriam Goldammer, Nicolai Spicher. Fusion of Features with Neural Networks for Prediction of Secondary Neurological Outcome After Cardiac Arrest

Best title :

  • Mostafa Moussa, Hessa Alfalahi, Mohanad Alkhodari, Leontios Hadjileontiadis, Ahsan Khandoker. Random Forest and Attention-Based Networks in Quantifying Neurological Recovery

Best preprint (by deadline) :

  • Felix Krones, Benjamin Walker, Guy Parsons, Adam Mahdi, Terry Lyons. Multimodal deep learning approach to predicting neurological recovery from coma after cardiac arrest

2022: Heart Murmur Detection from Phonocardiogram Recordings

Overall Challenge Competition:

  • Co-Winner 1st (murmurs category) : Andrew McDonald, Mark JF Gales and Anurag Agarwal
  • Co-Winner 1st (murmurs category) : Yujia Xu, Xinqi Bao, Hak-Keung Lam and Ernest Nlandu Kamavuako
  • Winner 3rd (murmurs category) : Jungguk Lee, Taein Kang, Narin Kim, Soyul Han, Hyejin Won, Wuming Gong and Il-Youp Kwak
  • Winner 1st (outcomes category) : Andrew McDonald, Mark JF Gales and Anurag Agarwal
  • Winner 2nd (outcomes category) : Yale Chang, Luoluo Liu and Corneliu Antonescu
  • Winner 3rd (outcomes category) : Zaria Imran, Ethan Grooby, Chiranjibi Sitaula, Vinayaka Malgi, Sunil Aryal and Faezeh Marzbanrad

Hackathon Competition:

  • N/A

Best Challenge Oral:

  • Sofia M. Monteiro, Ana Fred and Hugo P. Silva

Best Challenge Poster: 

  • Hui Lu, Julia Beatriz Yip, Tobias Steigleder, Stefan Grießhammer, Naga Venkata Sai Jitin Jami, Bjoern Eskofier, Christoph Ostgathe and Alexander Koelpin

Best Team Name : 

  • Sara Summerton, Danny Wood, Darcy Murphy, Oliver Redfern, Matt Benatan, Matti Kaisti and David Wong

2021: Varying Dimensions in Electrocardiography

Overall Challenge Competition:

  • Winner 1st (all leads category) : ISIBrno-AIMT: Petr Nejedly, Adam Ivora, Radovan Smisek, Ivo Viscor, Zuzana Koscova, Pavel Jurak, Filip Plesinger, Classification of ECG using Ensemble of Residual CNNs with Attention Mechanism
  • Winner 2nd (all leads category) : DSAIL_SNU: Hyeongrok Han, Seongjae Park, Seonwoo Min, Hyun-Soo Choi, Eunji Kim, Hyunki Kim, Sangha Park, Jinkook Kim, Junsang Park, Junho An, Kwanglo Lee, Wonsun Jeong, Sangil Chon, Kwonwoo Ha, Myungkyu Han, Sungroh Yoon, Towards High Generalization Performance on Electrocardiogram Classification
  • Winner 3rd (all leads category) : NIMA: Nima Wickramasinghe and Mohamed Athif, Multi-label Cardiac Abnormality Classification from Electrocardiogram using Deep Convolutional Neural Networks
  • Winner 1st (three-lead category) : ISIBrno-AIMT: Petr Nejedly, Adam Ivora, Radovan Smisek, Ivo Viscor, Zuzana Koscova, Pavel Jurak, Filip Plesinger, Classification of ECG using Ensemble of Residual CNNs with Attention Mechanism
  • Winner 2nd (three-lead category) : DSAIL_SNU: Hyeongrok Han, Seongjae Park, Seonwoo Min, Hyun-Soo Choi, Eunji Kim, Hyunki Kim, Sangha Park, Jinkook Kim, Junsang Park, Junho An, Kwanglo Lee, Wonsun Jeong, Sangil Chon, Kwonwoo Ha, Myungkyu Han, Sungroh Yoon, Towards High Generalization Performance on Electrocardiogram Classification
  • Winner 3rd (three-lead category) : NIMA: Nima Wickramasinghe and Mohamed Athif, Multi-label Cardiac Abnormality Classification from Electrocardiogram using Deep Convolutional Neural Networks
  • Winner 1st (two-lead category) :