The application of AI solutions to predict critical heart failure endpoints from echocardiography presents a promising avenue for improving patient care and treatment planning. However, the requirement for a substantial amount of labelled data for fully supervised training of deep learning models in medical imaging poses significant challenges, particularly due to the need for skilled medical professionals to annotate image sequences. To address this limitation, our study delves into the potential of self-supervised learning, emphasizing interpretability, robustness, and safety as crucial factors in cardiac imaging analysis. Our investigation focuses on leveraging self-supervised learning on a large unlabeled dataset, enabling the discovery of features applicable to a multitude of downstream tasks. The study employs the DINO and modified STEGO model as a backbone, pre-trained on diverse medical and non-medical data. This approach facilitates the generation of self-segmented outputs, termed "parcels," which identify distinct anatomical sub-regions of the heart. Our findings showcase the exceptional robustness of these self-learned parcels across diverse patient profiles and cardiac cycle phases. Moreover, these parcels offer high interpretability and effectively encapsulate clinically relevant cardiac substructures. Furthermore, we conduct a comprehensive evaluation of the proposed self-supervised approach on publicly available datasets, demonstrating its adaptability to a wide range of requirements. The implications of our results underscore the potential of self-supervised learning in addressing the challenges of labelled data scarcity in medical im-aging. This approach not only offers a pathway for improved cardiac imaging analysis but also holds promise for enhancing the efficiency and interpretability of diagnostic procedures, thereby positively impacting patient care and clinical decision-making.