SleepViTransformer: Patch-based sleep spectrogram transformer for automatic sleep staging

Sleep staging is a crucial aspect of sleep evaluation and disease diagnosis. Numerous automatic schemes have been developed to replace the tedious and expensive task of manual sleep staging. In this paper, we propose SleepViTransformer, a novel scheme that aims to improve the performance of automatic sleep staging. There are three main contributions. (1) A patch-based sleep spectrogram Transformer encoder is proposed for learning more effective feature representations to enhance the automatic sleep staging performance. (2) To reduce the dependence on large amounts of annotated PSG data, cross-modality knowledge from a model pre-trained on image and audio datasets is transferred to SleepViTransformer, significantly improving the model performance. (3) To improve the model robustness under noise and artifacts, a set of PSG augmentations based on the characteristics of the PSG signal is proposed. Experimental results show that SleepViTransformer achieves state-of-the-art performance on four publicly available datasets. On small-scale datasets, SleepViTransformer outperforms the runner-up by 1.8% and 2.5% on SleepEDF-20 and by 1.2% and 1.6% on SleepEDF-78 in terms of accuracy and Cohen’s kappa. SleepViTransformer also performs well on large-scale datasets, outperforming the runner-up by 0.8% and 1.2% on Physio-2018 and 0.4% and 0.2% on SHHS, respectively. The ablation experiments show that both the cross-modality pre-training and PSG augmentation module have positive impacts on improving model performance. To the best of our knowledge, this is the first model to adopt the patch encoding technique from Vision Transformer (ViT) on the sleep PSG spectrogram, showing its eminent potential for PSG signal analysis.