Sleep apnea syndrome (SAS) is a nocturnal respiratory disorder that often goes undiagnosed and can be associated with long-term cardiovascular complications. In order to overcome the constraints associated with gold standard in-lab polysomnography, alternative screening solutions are currently being developed. In particular, since the respiratory signal can be reconstructed from the electrocardiogram (ECG), the latter is all the more interesting as its recording is easy and non-invasive for the patient. The application of deep learning algorithms using ECGs has been shown to be effective in classifying pathological sleep-related events. In this paper, we propose a novel hybrid architecture to accurately detect apneic episodes on the basis of single-lead ECGs for moderate or severe SAS assessment. First, raw waveforms are processed to remove undesired noises and facilitate their interpretation by the network. Secondly, morphological and temporal components of interest are extracted through convolutional and recurrent blocks, respectively. Additional mechanisms are further integrated in order to enhance the performance of the classification process. Models were trained and validated on the Stanford Technology Analytics and Genomics in Sleep (STAGES) database. Influence of patient phenotype on classification was estimated by comparing the performance between several groups of patients with different clinical information. Overall, a model we have developed shows competitive results compared to best current methods by accurately classifying patients with different degrees of severity with a mean sensitivity, specificity and accuracy of respectively 93.1, 72 and 85.7%. Thus, it offers promising prospects for the early and less constraining diagnosis of sleep apnea.