Electrocardiogram (ECG) is a widely used signal to diagnose heart health. Experts can detect multiple cardiac abnormalities using the ECG signal. In a clinical setting, 12-lead ECG is mainly used. But using a lower number of leads can make the ECG more pervasive as it can be integrated with wearable devices. At the same time, we need to build systems that can diagnose cardiac abnormality automatically. This work develops Channel self-Attention (CA) based deep neural network to diagnose cardiac abnormality using a different number of ECG leads. Our approach takes care of the temporal and spatial interdependence of multi-lead ECG signals. Our team participates under the name “cardiochallenger” in the PhysioNet/Computing in Cardiology Challenge 2021”. Our method achieves the challenge metric scores of 0.64, 0.64, 0.64, 0.63, and 0.63 for the 12-lead, 6-lead, 4-lead, 3-lead, and 2-lead cases, respectively, on the validation data set.