Heart sound detection is crucial for diagnosing cardiovascular diseases and monitoring cardiac health. However, traditional approaches often rely solely on either audio analysis or electrocardiogram (ECG) interpretation, potentially limiting diagnostic accuracy. To address this, our study proposes a novel approach by integrating audio and ECG data using machine learning techniques. In our experimentation with the PhysioNet 2016 datasets, we employed various models. Utilizing an LSTM model on dataset set_a resulted in an impressive accuracy of 80%, showcasing its efficacy in capturing temporal dependencies within the data. Conversely, applying a CNN model to dataset_f yielded a slightly lower accuracy of 73%, possibly due to the dataset's limited availability. As a next step,, we plan to employ advanced signal processing methods to extract relevant features from both modalities. Subsequently, we will develop a machine-learning model capable of accurately distinguishing between different heart sounds and abnormal ECG patterns. We are also planning to explore the advance state-of-the-art deep learning approaches for improving performance. Our preliminary evaluation results demonstrate promising performance, highlighting the effectiveness of integrating audio and ECG data for heart sound detection. We believe that our research significantly contributes to advancing the field of cardiovascular diagnostics. By integrating advanced deep learning methods with multimodal data, we aim to lay the groundwork for future developments in healthcare monitoring technologies, ultimately improving patient outcomes.