Deep Learning (DL) models for automatic ECG interpretation became widely investigated in recent years. However, their performance varies highly across models and datasets. One of the main reasons is the capability of the DL model to learn spurious correlations between inputs and outcomes that could be present in a dataset. In this study, we proposed a novel training strategy potentially able to force the domain knowledge into a DL model, by complementing, only during training, an end-to-end approach with features known to be relevant for the outcome. We tested the approach for the creation of a DL model tuned to identify myocardial infarction (MI) from the standard 12-lead electrocardiograms (ECGs). Two models were trained: one with standard backpropagation (full model) and the second one (split model) with the proposed approach, on the PTB Diagnostic ECG Database. An explainable AI technique was then used to identify which ECG leads were considered relevant by both models for each MI location, and were compared with guidelines for MI location identification. The validation accuracy was 0.85 and 0.69 for full and split models, respectively. Despite the lower performance achieved with the proposed approach, the number of relevant leads was higher (11 vs 4), suggesting that the domain knowledge was likely percolated into the DL model and made it more robust.