Unsupervised clustering of single-cell RNA sequencing (scRNA-seq) data is essential because it allows us to identify putative cell types. With the rapid growth of scRNA-seq data, it is difficult for traditional learning-based single-cell analysis methods to efficiently and continuously deal with them due to catastrophic forgetting. Inspired by how the human brain learns and remembers, we propose a novel unsupervised continual learning method for single-cell analysis, namely Continual Unsupervised Memory Replay (CUMR). We first employ a novelty detection algorithm to learn and assign pseudo-labels to unlabeled scRNA-seq data. Then, we apply memory-replay-based continual learning networks to achieve unsupervised continual learning of single-cell analysis. Experiments on real datasets show that CUMR has superior performance over other state-of-the-art continual learning methods in single-cell analysis tasks such as cell typing.