Beat-level ECG arrhythmia classification is benchmarked almost exclusively on small datasets: the majority of deep learning studies rely on MIT-BIH, a 48-recording, single-centre database. Whether these models generalise beyond this dataset is largely unknown. We evaluate this directly, fine-tuning xECG, an xLSTM-based foundation model, on INCART and MIT-BIH (inter-patient split, AAMI classes N, V, S, F, Q) and assessing beat-level predictions at scale on MIMIC-IV-ECG (706,073 recordings).
This exposes two distinct failure modes. First, patch-based models process fixed-length windows, so a single beat may span consecutive patches, causing double-counting. Suppressing consecutive same-class predictions treats contiguous same-class predictions as a single beat, but this heuristic fails at elevated heart rates when the RR interval approaches the patch duration. On MIMIC-IV-ECG, a patch-only model achieves beat-count Pearson r=0.678; under tachycardia (HR>120 bpm), it collapses to r=−0.420. Training R-peak detection jointly with beat classification resolves this: overall r rises to 0.961, tachycardia r to 0.776. On MIT-BIH, the joint model achieves a macro F1 of 0.599, V-class F1 of 0.973, and R-peak F1 of 0.996 (150 ms tolerance).
Second, and independently, distribution shift degrades precision substantially. On CPSC 2018 (6,871 12-lead recordings), converting beat-level predictions to recording-level ventricular ectopic beat detection yields sensitivity 0.930 but a PPV of 0.501. False positives are dominated by right bundle branch block, atrial fibrillation, and premature atrial contractions. On MIMIC-IV-ECG (F1=0.726, sensitivity=0.826, PPV=0.648), 52.5% of false positives contain conduction defects and 25.8% atrial fibrillation. Joint R-peak detection does not address this failure.
These results demonstrate that standard beat-level benchmarks substantially overstate clinical utility and that robust arrhythmia classification requires both reliable beat localisation and training data that reflects real-world rhythm diversity.