A Joint-Embedding Predictive Architecture for Multi-Modal Time Series

Simon Böhi1, Volodimir Buchakchiyskiy2, Hélène Corbaz3, Paul Fischer1, Max Krähenmann1, Sidaty El Hadramy1, Philippe C. Cattin1, Ece Özkan Elsen4
1University of Basel, 2University of Base, 3University of Basel, University Hospital of Basel, 4University of Basel


Abstract

Aims: Cognitive impairment increasingly burdens aging populations but is typically diagnosed only after substantial neurological decline. Polysomnography (PSG) captures physiological dynamics during sleep that may reveal early markers of neurological deterioration years before clinical presentation. We propose a self-supervised model capable of extracting prognostic representations from PSG to predict future cognitive impairment diagnoses.

Methods: Our approach adapts the Learnable Joint-Embedding Predictive Architecture (LeJEPA) to multi-channel PSG time series. For pretraining, one-hour windows of raw PSG signals were sampled from overnight recordings. From each window, two global context views (30–60 minutes) and six local views (6–20 minutes) were extracted. The model was trained to align the latent embeddings of local views with their corresponding global views, capturing multi-scale temporal structures without manual annotations. For downstream prediction, recordings were processed using one-hour windows with 50% overlap. We utilized the pretrained encoder to extract features for a lightweight linear classifier. Patient-level risk was determined via majority voting across all windows, improving robustness to recording artifacts. Models were evaluated using leave-one-site-out cross-validation.

Results: Local testing produced an average AUROC of 0.56 ± 0.09 across the three splits. Our submission to the unofficial phase with a preliminary submission using an alternative pipeline achieved an AUROC of 0.477 on the validation server.

Conclusion: Self-supervised predictive pretraining on PSG shows promise for neurological risk prediction, moving beyond handcrafted sleep features toward scalable foundation representations that generalize across heterogeneous clinical recording environments.

Team: Koalalition | Unofficial phase AUROC: 0.477