In this study, we present a deep learning based framework for the identification of Major Depressive disorder (MDD) patients from cardiovascular signals. In this work, multi-modal cardiovascular signals, including electrocardiogram (ECG) and finger photoplethysmography (PPG), are used. The signals were collected from 60 subjects for 10 minutes, out of whom 30 were diagnosed with MDD by a psychiatric, and 30 were healthy. The signals are pre-processed and segmented into 30 seconds segments to be able to perform the identification in half a minute window, which proved to be sufficient in this work. Then, time-frequency analysis is performed on the signals for feature extraction and then a recurrent neural network architecture based on Long Short-Term Memory (LSTM) networks is utilized for the identification of the MDD patients. The results demonstrated a robust performance with an accuracy of 85.7%. This study can be considered an advancement towards the involvement of artificial intelligence tools in the assisted diagnosis and monitoring of mental diseases, and reducing their risk and impact on human daily life.