Automatic methods to identify cardiovascular disorders based on Electrocardiograms (ECGs) have been used for decades and gained special attention with the breakthrough of deep learning methods.This paper proposes a new model based on a Mixture of Experts (MoE) to identify 6 physician-defined clinical labels spanning rhythm and conduction disorders. The MoE framework combines the opinions of multiple models ("experts") to make weighted decisions. It follows the divide-and-conquer principle, dividing the problem space among a select group of neural network subspace "experts" overseen by a gating network. Unlike typical MoE scenarios, where the gating network finds ways to weigh and combine the models, our approach leverages prior knowledge from cardiology specialists, who hierarchically divide the space, enabling us to tailor the MoE architecture to incorporate this information explicitly.The disorders are organized in a 2-level hierarchy, where specialists first separate normal ECGs from those rooted in rhythm and morphological abnormalities. ECGs with abnormalities are further divided into 6 categories. The proposed MoE is trained on a dataset with over 2 million labeled ECGs from the CODE (Clinical Outcomes in Digital Electrocardiography) study. The results in the test set represent the new state-of-the-art in this task. By incorporating a residual CNN to the MoE framework, we improved the weighted mean f-measure of over 0.93 even further, showing the hierarchical approach is more effective than using simple flat classes.