The data imbalance limits the stability and accuracy in fault diagnosis of rolling bearings. In general, traditional methods need the necessary features and a large number of labeled data in advance, which requires lots of time and manpower. In this paper, a novel data augmentation method named variational autoencoding generative adversarial networks with deep regret analysis is proposed to improve the fault diagnosis ability. Firstly, an encoder is merged into the generative adversarial networks to learn deep features of real data for the improvement of data generation quality. Secondly, the discriminator is integrated with the deep regret analysis method to avoid mode collapse by imposing the gradient penalty on it. Thirdly, the feature matching module is adopted in the generator to enhance the deep feature and eliminate overfitting. The proposed method is verified to diagnose two rolling bearing datasets. The results denote that the proposed method has better effectiveness and robustness than typical data synthesis based fault diagnosis methods.