A phonocardiogram (PCG), known as heart sound auscultation, is used to diagnose congenital heart disease (CHD). It is a severe cardiovascular problem that affects children. The diagnosis of CHD through heart sound auscultation requires extensive medical training and understanding. PCG data can be collected through sensors. The quality of PCG data may be challenging to interpret because of the sensor location, a child's developing heart, and the complex and changeable cardiac acoustic environment. Despite substantial breakthroughs in heart sound technology, most diagnostic techniques depend on the shallow structure and conventional segmentation features. This study used 751 patients' PCG signals with an age range of 5 months to 20 years to develop the model using a signal quality method. It proposes a deep learning model that classifies PCG signals to predict heart abnormalities using a one-dimensional Convolution Neural Network (1D-CNN) with residual block. First, the heart sound signal is pre-processed, and its quality is improved by removing the spikes in the signals. After assessing the signal quality, only good quality signals are used as input features of the neural network. The results of the study showed that the residual model was successful in achieving a balance between accuracy, sensitivity, and specificity. The model has a 0.91 accuracy, 0.91 sensitivity, and 0.91 specificity. The Receiver Operating Characteristic (ROC) plot yielded an Area Under Curve (AUC) value of 0.97, and the F1-score was 0.92. The proposed model has required only 0.05s to predict the signal abnormalities. Thus, it can be implemented as a primary screening tool for remote-end physicians by providing cheaper and faster interpretations of PCG signals before referring the cases to specialists. This article also presents in-depth assessments of various model evaluation metrics with demographic information of patients.