The demand for extensive annotated datasets in ECG interpretation has led to the development of synthetic datasets using generative neural networks. Our study is aimed at assessing the quality of synthetic ECGs generated via a CycleGAN network by means of visual inspection (confidence bands and UMAP 2D plots), GAN-specific evaluation methods (GAN-train and GAN-test scoring), and statistical tests comparing ST segment amplitudes (modified Hotelling T-squared test). To this goal, we utilized a selection of 12-lead ECGs from the PTBXL dataset (available on Physionet) falling under three conditions: normal sinus rhythm, anteroseptal myocardial infarction and inferior myocardial infarction. Through the CycleGAN network we generated synthetic ECGs and compared them with the original ones. The qualitatively analysis, by means of plots, showed that there was a difference in the distributions of real and synthetic data. The GAN-train/test method provided results confirming this conclusion. Lastly, the ST-segments analysis showed distributions which were dissimilar among all the conditions. In conclusion, our work demonstrated that generative networks developed in the context of image processing cannot be simply adapted to augment ECG dataset, and that proper care should be enforced to verify the quality of the generated signals, before utilising such data in applications.