Aims: This study (team name “amc-sh”) aims to detect heart sound murmur in phonocardiogram recordings by PhysioNet Challenge 2022. And we also aim to classify ‘unknown cases’ with signals that are difficult to distinguish between normal and murmur, or with poor signal quality. Methods: In this study, 3163 heart sound recordings collected from 942 patients were used as the training set. Each signal was recorded from different locations: pulmonary valve, aortic valve, mitral valve, tricuspid valve, and other. Our proposed model was designed to learn feature representation of each location separately and then combine them with demographic features. The architecture employs four CNNs and one 2-layered MLP for heart sound signals and demographic data, respectively. Each CNN consists of a convolutional layer, 12 residual blocks and a global average pooling layer in order. The outputs of four CNNs and the outputs of 2-layered MLP networks are concatenated and fed into the fully connected layers. For the CNN input, we preprocessed signals using envelope methods as follows: Homomorphic envelope, Hilbert envelope, Power spectral density, Wavelet envelope, Shannon energy and slope sum. The extracted envelope was combined with the raw signal in the 6-dimensional signal. For the MLP input, age, sex, height, weight, and pregnant information were used as demographic features. Results: Our model achieved 1271 challenging scores for the validation data. For 5-fold cross-validation results of training data set, our methods achieved 1333 challenging scores on average. From cross-validation results, average value of accuracy, F1-score, precision, and recall are 0.79, 0.55, 0.71, and 0.53, respectively. Especially, we achieved 0.55 f1-score for murmur classes, 0.21 f1-score for unknown classes. Conclusions: In this study, we proposed the residual CNN model to detect heart sound murmur by combining envelope method with demographic information.