physionet-logo-kp6The PhysioNet/CinC Challenge


– 2023 –

Predicting Neurological Recovery from Coma After Cardiac Arrest

2023 Challenge Summary

For the past 23 years, PhysioNet and Computing in Cardiology have co-hosted a series of annual challenges, now called the George B. Moody PhysioNet Challenges, to tackle clinically interesting questions that are either unsolved or not well-solved.

The goal of the 2023 Challenge is to use longitudinal EEG and ECG recordings to predict good and poor patient outcomes after cardiac arrest.

We ask participants to develop and deploy an open-source algorithm that can use basic clinical information and EEG and ECG recordings to predict the level of neurological recovery for cardiac arrest patients who present to the hospital in a coma. The winner of the Challenge will be the team whose algorithm achieves the highest prediction performance in the hidden test set.

Competition Background

While planning CinC 2000, local hosts Roger Mark and George Moody proposed to organize an activity that would make effective use of their newly-established PhysioNet web site to stimulate rapid progress on an unsolved problem of practical clinical significance. A timely contribution of data made it possible to create the first PhysioNet/CinC Challenge, which attracted the attention of more than a dozen teams to the subject of detecting sleep apnea from the ECG. Their efforts were broadly successful, they discussed their findings at CinC 2000, and an annual tradition was born.

PhysioNet offers free access via the web to large and growing collections of recorded physiologic signals and related open-source software. Originally established under the auspices of the NIH’s National Center for Research Resources, PhysioNet has been funded since September 2007 under a cooperative agreement with the NIH’s National Institute of Biomedical Imaging and Bioengineering (NIBIB), and with the NIH’s National Institute of General Medical Sciences (NIGMS).

In complementary ways, PhysioNet and Computing in Cardiology catalyze and support scientific communication and collaboration between basic and clinical scientists. The annual meetings of CinC are gatherings of researchers from many nations and disciplines, bridging the geographic and specialty chasms that separate understanding from practice, while PhysioNet provides on-line data and software resources that support collaborations of basic and clinical researchers throughout the year. The annual PhysioNet/CinC Challenges seek to provide stimulating yet friendly competitions, while at the same time offering both specialists and non-specialists alike opportunities to make progress on significant open problems whose solutions may be of profound clinical value. The use of shared data provided via PhysioNet makes it possible for participants to work independently toward a common objective. At CinC, participants can make meaningful results-based comparisons of their methods; lively and well-informed discussions are the norm at scientific sessions dedicated to these challenges. Discovery of the complementary strengths of diverse approaches to a problem when coupled with deep understanding of that problem frequently sparks new collaborations and opportunities for further study.

A new challenge topic is announced each year on the Challenge page at PhysioNet. The PhysioNet team assembles and posts the raw materials needed to begin work. Usually, these raw materials consist of a collection of data to be analyzed; the analyses are provided for a subset of the data (the “learning set”) in each case, and the challenge is to analyze the remaining data (the “test set”).

What will be the topic of the next challenge? It might be image analysis, or simulation, or forecasting…. An ideal challenge problem is interesting, clinically important, and possible to study using available materials that have not been widely circulated previously. Moreover, there must be an objective way to evaluate the quality of a challenge entry (for an analysis problem, this usually means there must be a known set of correct analyses of the data, i.e., a “gold standard” against which entries can be compared). You are invited to submit your ideas for future challenge topics to the Challenge organizer, Gari Clifford; suggestions accompanying relevant data are particularly welcome.

PhysioNet/CinC Challenge Awards

Challenges are open to all. An important milestone for participants is the deadline for submitting abstracts for CinC, which is 15 April each year. Those wishing to qualify as official entrants, with eligibility for awards, must submit an abstract describing their work as well as an entry for scoring by about one week before the abstract deadline. A limited number of revised entries may be submitted between the abstract deadline and the final challenge deadline in early September. Eligibility for awards also requires participants to present their work in a scientific session of CinC. See the Challenge pages on PhysioNet for deadlines and rules for this year’s competition.

Most Challenges are presented as two events (often a narrowly-defined question and a more general one), or sometimes more.

An award is offered for the best solution obtained by any eligible participant.  To qualify for this prize, the solution must include computer code in open-source format.  Each entry should contain a file that defines the specific open-source license under which the software is available. See the Challenge Web page for more details on the problem and the submission process for each year’s challenge.  The winners receive their awards during the final plenary session on Wednesday afternoon. Follow the links below for details about previous challenges.

PhysioNet/CinC Challenge Award Winners

2023: Predicting Neurological Recovery from Coma After Cardiac Arrest

Overall Challenge Compétition:

  • Winner 1st : Morteza Zabihi, Alireza Chaman Zar, Pulkit Grover, Eric Rosenthal. HyperEnsemble Learning from Multimodal Biosignals to Robustly Predict Functional Outcome after Cardiac Arrest
  • Winner 2nd : Dong-Kyu Kim, Hong-Cheol Yoon, Hyun-seok Kim, Woo-Yeong Seo, Sung-Hoon Kim. Predicting Neurological Outcome After Cardiac Arrest Using a Pretrained Model with Electroencephalography Augmentation
  • Winner 3rd : Hongliu Yang and Ronald Tetzlaff. Model Ensembling for Predicting Neurological Recovery after Cardiac Arrest: Top-down or Bottom-up?

Hackathon Competition:

  • Winner 1st : Dong-Kyu Kim, Hongcheol Yoon and Hyun-Seok Kim.
  • Winner 2nd : Richard Hohmuth, Marc Goettling and Maurice Rohr

Best Challenge Oral:

  • Charlotte Maschke. Functional outcome prediction after cardiac arrest using machine learning and network dynamics of resting-state electroencephalography

Best Challenge Poster:

  • Inês Sampaio. Predicting Comatose Patient’s Outcome Using Brain Functional Connectivity with a Random Forest Model

Best Team Name :

  • Philip Hempel, Philip Zaschke, Miriam Goldammer, Nicolai Spicher. Fusion of Features with Neural Networks for Prediction of Secondary Neurological Outcome After Cardiac Arrest

Best title :

  • Mostafa Moussa, Hessa Alfalahi, Mohanad Alkhodari, Leontios Hadjileontiadis, Ahsan Khandoker. Random Forest and Attention-Based Networks in Quantifying Neurological Recovery

Best preprint (by deadline) :

  • Felix Krones, Benjamin Walker, Guy Parsons, Adam Mahdi, Terry Lyons. Multimodal deep learning approach to predicting neurological recovery from coma after cardiac arrest

2022: Heart Murmur Detection from Phonocardiogram Recordings

Overall Challenge Competition:

  • Co-Winner 1st (murmurs category) : Andrew McDonald, Mark JF Gales and Anurag Agarwal
  • Co-Winner 1st (murmurs category) : Yujia Xu, Xinqi Bao, Hak-Keung Lam and Ernest Nlandu Kamavuako
  • Winner 3rd (murmurs category) : Jungguk Lee, Taein Kang, Narin Kim, Soyul Han, Hyejin Won, Wuming Gong and Il-Youp Kwak
  • Winner 1st (outcomes category) : Andrew McDonald, Mark JF Gales and Anurag Agarwal
  • Winner 2nd (outcomes category) : Yale Chang, Luoluo Liu and Corneliu Antonescu
  • Winner 3rd (outcomes category) : Zaria Imran, Ethan Grooby, Chiranjibi Sitaula, Vinayaka Malgi, Sunil Aryal and Faezeh Marzbanrad

Hackathon Competition:

  • N/A

Best Challenge Oral:

  • Sofia M. Monteiro, Ana Fred and Hugo P. Silva

Best Challenge Poster: 

  • Hui Lu, Julia Beatriz Yip, Tobias Steigleder, Stefan Grießhammer, Naga Venkata Sai Jitin Jami, Bjoern Eskofier, Christoph Ostgathe and Alexander Koelpin

Best Team Name : 

  • Sara Summerton, Danny Wood, Darcy Murphy, Oliver Redfern, Matt Benatan, Matti Kaisti and David Wong

2021: Varying Dimensions in Electrocardiography

Overall Challenge Competition:

  • Winner 1st (all leads category) : ISIBrno-AIMT: Petr Nejedly, Adam Ivora, Radovan Smisek, Ivo Viscor, Zuzana Koscova, Pavel Jurak, Filip Plesinger, Classification of ECG using Ensemble of Residual CNNs with Attention Mechanism
  • Winner 2nd (all leads category) : DSAIL_SNU: Hyeongrok Han, Seongjae Park, Seonwoo Min, Hyun-Soo Choi, Eunji Kim, Hyunki Kim, Sangha Park, Jinkook Kim, Junsang Park, Junho An, Kwanglo Lee, Wonsun Jeong, Sangil Chon, Kwonwoo Ha, Myungkyu Han, Sungroh Yoon, Towards High Generalization Performance on Electrocardiogram Classification
  • Winner 3rd (all leads category) : NIMA: Nima Wickramasinghe and Mohamed Athif, Multi-label Cardiac Abnormality Classification from Electrocardiogram using Deep Convolutional Neural Networks
  • Winner 1st (three-lead category) : ISIBrno-AIMT: Petr Nejedly, Adam Ivora, Radovan Smisek, Ivo Viscor, Zuzana Koscova, Pavel Jurak, Filip Plesinger, Classification of ECG using Ensemble of Residual CNNs with Attention Mechanism
  • Winner 2nd (three-lead category) : DSAIL_SNU: Hyeongrok Han, Seongjae Park, Seonwoo Min, Hyun-Soo Choi, Eunji Kim, Hyunki Kim, Sangha Park, Jinkook Kim, Junsang Park, Junho An, Kwanglo Lee, Wonsun Jeong, Sangil Chon, Kwonwoo Ha, Myungkyu Han, Sungroh Yoon, Towards High Generalization Performance on Electrocardiogram Classification
  • Winner 3rd (three-lead category) : NIMA: Nima Wickramasinghe and Mohamed Athif, Multi-label Cardiac Abnormality Classification from Electrocardiogram using Deep Convolutional Neural Networks
  • Winner 1st (two-lead category) : ISIBrno-AIMT: Petr Nejedly, Adam Ivora, Radovan Smisek, Ivo Viscor, Zuzana Koscova, Pavel Jurak, Filip Plesinger, Classification of ECG using Ensemble of Residual CNNs with Attention Mechanism
  • Winner 2nd (two-lead category) : DSAIL_SNU: Hyeongrok Han, Seongjae Park, Seonwoo Min, Hyun-Soo Choi, Eunji Kim, Hyunki Kim, Sangha Park, Jinkook Kim, Junsang Park, Junho An, Kwanglo Lee, Wonsun Jeong, Sangil Chon, Kwonwoo Ha, Myungkyu Han, Sungroh Yoon, Towards High Generalization Performance on Electrocardiogram Classification
  • Winner 3rd (two-lead category) : NIMA: Nima Wickramasinghe and Mohamed Athif, Multi-label Cardiac Abnormality Classification from Electrocardiogram using Deep Convolutional Neural Networks

Hackathon Competition:

  • N/A

Best Challenge Oral:

  • Cristina Gallego Vázquez, Alexander Breuss, Oriella Gnarra, Julian Portmann, Giulia Da Poian, Two will do: Convolutional Neural Network with Asymmetric Loss and Self-Learning Label Correction for Imbalanced Multi-Label ECG Data Classification

Best Challenge Poster: 

  • Hao-Chun Yang, Wan-Ting Hsieh, Trista Pei-Chun Chen, A Mixed-Domain Self-Attention Network for Multilabel Cardiac Irregularity Classification Using Reduced-Lead Electrocardiogram

Best Team Name : 

  • PhysioNauts: Stefano Magni, Chiara Salvi, Andrea Sansonetti, Tiziana Tabiadon, Guadalupe García Isla, Combining a ResNet Model with Handcrafted Temporal Features for ECG Classification with Varying Number of Leads

Best Article Title :

  • Cristina Gallego Vázquez, Alexander Breuss, Oriella Gnarra, Julian Portmann, Giulia Da Poian, Two will do: Convolutional neural network with asymmetric loss, self-learning label correction, and hand-crafted features for imbalanced multi-label ECG data classification

Best Preprint :

  • Maurice Rohr, Filip Plesinger, Veronika Bulkova, Christoph Hoog Antink, Improving Machine Learning Education during the COVID-Pandemic using past Computing in Cardiology Challenges

2020: Classification of 12-lead ECGs 

Overall Challenge Competition:

  • 1st Place: Annamalai Natarajan, Yale Chang, Sara Mariani, Asif Rahman, Gregory Boverman, Shruti Vij, Jonathan Rubin, A Wide and Deep Transformer Neural Network for 12-Lead ECG Classification
  • 2nd Place: Zhibin Zhao, Hui Fang, Samuel D. Relton, Ruqiang Yan, Yuhong Liu, Zhijing Li, Jing Qin, David C. Wong, Adaptive lead weighted ResNet trained with different duration signals for classifying 12-lead ECGs 
  • 3rd Place: Zhaowei Zhu, Han Wang, Tingting Zhao, Yangming Guo, Zhuoyang Xu, Zhuo Liu, Siqi Liu, Xiang Lan, Xingzhi Sun, Mengling Feng, Classification of Cardiac Abnormalities From ECG Signals Using SE-ResNet

Hackathon Competition:

  • 1st Place: N/A
  • 2nd Place: N/A
  • 3rd Place: N/A

Best Challenge Oral:

  • 1st Place: David Assaraf, Jeremy Levy, Janmajay Singh, Armand Chocron and Joachim A. Behar, Classification of 12-lead ECGs using digital biomarkers and representation learning

Best Challenge Poster:

  • 1st Place: Guadalupe García Isla, Rita Laureanti, Valentina Corino and Luca Mainardi, ECG Morphological Decomposition for Automatic Rhythm Identification

2019: Early Prediction of Sepsis from Clinical Data:

Overall Challenge Competition:

  • 1st Place: James Morrill, Andrey Kormilitzin, Alejo Nevado-Holgado, Sumanth Swaminathan, Sam Howison, Terry Lyons, The Signature-based Model for Early Detection of Sepsis from Electronic Health Records in the Intensive Care Unit
  • 2nd Place: John Anda Du, Nadi Sadr, Philip de Chazal, A Comparison of Neural Network Approaches for Sepsis Prediction 
  • 3rd Place: Morteza Zabihi, Sepsis Prediction in Intensive Care Unit Using Ensemble of XGboost Models

Hackathon Competition:

  • 1st Place: Meicheng Yang, Hongxiang Gao, Xingyao Wang, Yuwen Li, Xing Liu, Jianqing Li, Chengyu Liu
  • 2nd Place: Jonathan Rubin, Yale Chang, Saman Parvaneh, Gregory Boverman
  • 3rd Place: John Anda Du, Miquel Alfaras, Naoki Nonaka, Inès Krissaane, Edwar Hernando Macias Toro, Matthieu Scherpf

Best Challenge Oral:

  • 1st Place: Marcus Vollmer, Christian F. Luz, Philipp Sodmann, Bhanu Sinha, Sven-Olaf Kuhn,Time-specific Metalearners for the Early Prediction of Sepsis

Best Challenge Poster:

  • 1st Place: Chloé Pou-Prom, Zhen Yang, Maitreyee Sidhaye, David Dai, Development of a Sepsis Early Warning Indicator

2018: You Snooze, You win:

Official results, as well as a paper describing the Challenge, are available on the Physionet pages. Top scores were achieved by:

  • Matthew Howe-Patterson, Bahareh Pourbabaee, and Frederic Benard (0.54) 
  • Guðni Fannar Kristjansson, Heiðar Már Þráinsson, Hanna Ragnarsdóttir, Bragi Marinósson, Eysteinn Gunnlaugsson, Eysteinn Finnsson, Sigurður Ægir Jónsson, Halla Helgadóttir, and Jón Skírnir Ágústsson (0.45) 
  • Runnan He, Kuanquan Wang, Yang Liu, Na Zhao, Yongfeng Yuan, Qince Li, and Henggui Zhang (0.43)

#An unofficial entry from Hongyang Li and Yuanfang Guan (who unfortunately missed the deadline to submit an abstract) achieved a score of 0.55.

2017:AF classification from a short single lead ECG recording

Morteza Zabihi, Ali Bahrami Rad, Aggelos K. Katsaggelos, Serkan Kiranyaz, Susanna Narkilahti, Moncef GabboujDetection of Atrial Fibrillation in ECG Hand-held Devices Using a Random Forest Classifier

Tomás Teijeiro, Constantino A. García, Paulo Félix, Daniel CastroArrhythmia Classification from the Abductive Interpretation of Short Single-lead ECG Records

Shreyasi Datta, Chetanya Puri, Ayan Mukherjee, Rohan Banerjee, Anirban Dutta Choudhury, Arijit Ukil, Soma Bandyopadhyay, Rituraj Singh, Arpan Pal, Sundeep
KhandelwalA Robust AF Classifier using Time and Frequency Features from Single Lead ECG Signal

Shenda Hong, Meng Wu, Yuxi Zhou, Qingyun Wang, Junyuan Shang, Hongyan Li, Junqing Xie
ENCASE: an ENsemble ClASsifiEr for ECG Classification Using Expert Features and
Deep Neural Networks

2016: Classification of Normal/Abnormal Heart Sound Recordings

1. Cristhian Potes*, Saman Parvaneh, Asif Rahman, Bryan Conroy, Daniel Schulman and John Ames
Hybrid Feature Aggregation for Detection of Abnormal Heart Sound
Philips Research, Cambridge, MA, USA

2. Morteza Zabihi*1, Ali Bahrami Rad2, Serkan Kiranyaz3, Moncef Gabbouj1 and Aggelos K. Katsaggelos4  PhysioNet/CinC Challenge: Normal/Abnormal PCG Classification using an Ensemble of Support Vector Machines
1Tampere University of Technology, Tampere, Finland 2University of Stavanger, Stavang