In this work, we leverage neural networks as transformers to analyze data from multiple channels of EEG (electroencephalogram) recordings. Our model consists of three modules: a channel-wise encoder, a cross-channel decoder, and a multilayer perceptron (MLP) head for classification and regression. The channel-wise encoder consists of 18 transformer blocks, each with multiple self-attention heads. Each transformer block is responsible for independently processing the mel frequency cepstral coefficients (MFCCs) of each channel.
The MFCCs of each channel are tokenized and fed into the corresponding transformer block of the encoder module. Each transformer block uses multiple self-attention heads to learn contextual time and frequency relationships derived from the channel MFCCs. This self-attention head mechanism encapsulates each channel's activity in terms of the spectral and spatial representations (magnitude and phase of MFCC coefficients) of the raw channels. The cross-channel decoder then uses the self-attention maps generated by the transformer blocks of the encoder to learn cross-channel relationships. The decoder examines how different channels' activities relate to one another, allowing us to identify patterns across the entire EEG recording.
Finally, the MLP head processes the decoder output to classify and predict CPC value for the given EEG data. Our model is designed to analyze 5-minute segments of EEG recordings per hour. However, it can also be used to analyze recordings of varying hours, where majority voting is used to combine the predictions from each 5-minute segment per hour and obtain a final classification decision, followed by regression to obtain the corresponding CPC value. Our team, OUS_IVS, failed to have a successful entry during the unofficial phase. However, when tested in a 3-fold cross-validation setup using given training data, our model achieved an average performance of 0.23 (scoring metric).