Oversampling method is used to solve the class imbalanced issues. Some existing oversampling methods do not well remove noisy samples and avoid synthesizing noisy samples. Therefore, we propose a new oversampling approach based differential evolution on the safe set for highly imbalanced datasets (SS_DEBOHID). SS_DEBOHID firstly uses k-nearest neighbors (kNN) method to learn the safe area of minority; then the DEBOHID oversampling method is used to synthesize new minority samples in the safe area. The advantages of SS_DEBOHID include that (a) it generates samples in the safe area to reduce generation of noisy samples and reduce synthetic samples falling into the classification boundary and majority area; (b) it uses the DEBOHID method to synthesize samples and increase the diversity of samples; (c) the method is suitable for highly imbalanced datasets. The proposed method is compared with 10 methods on 43 highly imbalanced datasets and evaluated on AUC and G_Mean metrics. The experimental results show that SS_DEBOHID obtains more than 30 best performing datasets on KNN, SVM, and DT classifiers in terms of AUC and G_mean, respectively. The proposed method outperforms other methods by 8.07% to 24.34% on average AUC metric and by at least 6.96% and up to 45.37% on average G_mean metric. In addition, we validate the efficiency of SS_DEBOHID on 8 high-dimensional and large sample size datasets. The experimental results show that SS_DEBOHID has better classification performance and robustness.