Aims: Heart sounds offer a promising, noninvasive alternative to Right Heart Catheterization for Pulmonary Hypertension screening. We previously developed a 3D-CNN using the beat-wise time-frequency distribution (TFD) of the second heart sound (S2), yielding an AUC of 0.89 on a private dataset (D1). However, when applied on a second dataset (D2), despite comparable training metrics, performance dropped dramatically (AUC = 0.48). This study aims to quantify the domain shift driving this discrepancy in generalizability. Methods: First, we applied transfer learning from D1 to D2, with and without fine-tuning, to determine whether robust features learnt from D1 could transfer to D2. Second, we analyzed GradCAMs, quantifying the input-activation alignment with correlation and Intersection over Union (IoU), to assess the focus of the model on S2 across the experiments. Results: One-shot transfer learning from D1 to D2 outperformed finetuning (AUC 0.66 vs. 0.48), thus highlighting the need for domain adaptation. GradCAMs revealed that, while the model correctly focused on the S2 region in D1, it relied on non-generalizable shortcuts in D2, explaining the performance drop. Both the correlation and the IoU were strongly associated with AUC, proving their value in quantifying detrimental domain shifts.
Conclusions: Our analysis shows and quantifies the vulnerability of DL-based PH detection with respect to domain shifts. This provides strong motivation for integrating domain adaptation techniques to enhance model generalizability, which is fundamental for real-life clinical deployment.