Objective: Electroencephalography (EEG) has been demonstrated to be a valuable tool for predicting neurological outcomes after cardiac arrest. However its complexity limits timely interpretation. In consequently, we propose a Multi-Modal Conv-Transformer network to accurately and timely assess probability of coma recovery with complex EEG.
Method: Our proposed method is divided into two parts: In pre-processing part, we selected the most recent EEG for 12, 24, 48, and 72 h from each patient. The EEG was segmented into 10-second equal-length slices and filtered using a 5th-order Butterworth bandpass filter (0.5-30 Hz). Then, the time-frequency spectrograms of each channel of EEG as a kind of modal input to the network, were extracted by short-time Fourier transform (STFT). The second part, feature extraction and prediction, is performed by the network model we built. The network is a two-branch structure. In the branch with 1-D signal as input, we used the large kernel convolution structure to extract the spatial-correlation between different polar associations of EEG; in the branch with spectrogram as input, we use the module before the classification layer of ResNet18 as a feature extractor to obtain the spectrogram features. The features output from the two branches are connected and input to the Transformer. By Transformer encoder module captures the long-term dependencies of the EEG signal, allowing the network to learn time-correlation features of the input EEG. Finally the features were input the fully connected layers to predict the results.
Results: Our (CQUPT_FP_mana) method was evaluated using the challenge validation dataset, and we obtained the best challenge metric score of 0.21, 0.30, 0.42, 0.48 corresponding to 12 h, 24 h, 48 h and 72 h in the unofficial stage. Scores of 0.26, 0.40, 0.49 and 0.59 were obtained in local ten-fold cross-validation using only the open challenge training set corresponding to different times.