The heterogeneity of polysomnography (PSG) data across patients, recording sites, and acquisition configurations poses a major challenge to the robustness and generalizability of deep learning models. While foundation models are promising for handling heterogeneous inputs, their limited task-specific specialization may hinder optimal performance on targeted disease prediction tasks. In this study, we investigate whether modality-specific fine-tuning of foundation models can improve prediction of future cognitive impairment. Our approach is developed and evaluated on multi-institutional overnight PSG recordings from the PhysioNet/Computing in Cardiology Challenge 2026 training set. For each physiological modality group (e.g. electromyography), we apply a pretrained PSG foundation model to that group's channels to encode 5 min segments into fixed-length embeddings. For each segment, the modality-specific embeddings are combined into a single multimodal representation and passed to a LSTM-based classification head that predicts future cognitive impairment. By training the encoders and classification head jointly in an end-to-end manner the foundation model encoders are fine-tuned to capture disease and modality specific patterns relevant to future cognitive outcomes. For preliminary evaluation, our (Team REM and Reason) baseline model — an AdaBoost decision tree classifier trained on expert features derived from sleep stage annotations — achieved an AUROC of 0.577 on the leaderboard in the unofficial phase of the PhysioNet/Computing in Cardiology Challenge 2026. We further evaluate performance using 5-fold cross-validation on the training set, stratified by subject, institution, and cognitive impairment label. The baseline achieves an AUROC of 0.510. Preliminary results indicate the need for a multimodal strategy, such as the proposed approach, which can capture relevant information for cognitive impairment prediction. Ongoing work focuses on hyperparameter tuning of our main approach to improve predictive performance and on interpretability analyses of modality-specific contributions to enhance robustness and support clinical insight.