Accurate, automatic heart auscultation for diagnosis of cardiovascular diseases has potential to support health outcomes, especially in areas with limited access to effective healthcare. The proposed method for heart murmur detection consists of a decision fusion between two models that takes multiple recordings per patient. The first model is an efficient Convolutional Neural Network (CNN) that takes unsegmented phonocardiogram (PCG) recordings, with Long-Short Term Memory (LSTM) and attention mechanism. Patient oversampling was used to deal with class imbalance. The direct input to the model are the Mel-Frequency Cepstrum Coefficients (MFCCs) extracted from 10 second segments. The second model is a Random Undersampling Boosting (RUSBoost) classifier which takes handcrafted features and innately deals with class imbalance. Before extracting features, the PCG recordings were segmented using a modified Hidden Semi Markov Model (HSMM) with heart rate range adjusted according to the patient's age group. Features representing heart sound quality, temporal, spectral, autocorrelation, wavelet and statistical features were extracted. The handcrafted features were normalised and ranked using Minimum Redundancy Maximum Relevance (mRMR) from 1636 extracted features. The top 50 features and three voting methods were assessed using 10-fold cross validation according to the competition cost function. The most effective voting method was if a murmur is detected in any recording, the patient classifies as having a murmur present, else perform majority voting. In patient-wise 10-fold cross validation, and evaluated using the competition score, the deep learning model scored mean ± standard deviation of 849±380 and the hand-crafted feature method scored 581±80. The submitted entry based on deep learning (CNN-LSTM with attention mechanism) achieved an unofficial score of 1315 (Team: Melbourne_Kangas). The fusion of this method with the feature-based classifier will be submitted to the challenge in the official phase as a cost-effective classifier algorithm for heart murmur detection.