Ensemble methods give better performance compared to a single machine learning algorithm. Vote is one of the best ensembles. Vote merges predictions from Simple Logistics and the Naive Bayes algorithms in the present work. The paper presents a new ensemble approach – Ensemble Particle Swarm Optimization (EnsPSO). The EnsPSO approach is a combination of (i) Vote, (ii) Correlation based Feature(s) Selection (CFS) method, (iii) PSO algorithm and (iv) random sampling method. The EnsPSO shows better performance results than Vote. The EnsPSO shows higher classification accuracy (96%) as compared to Vote (84%). The performance enhancement of EnsPSO is also proved using ten-fold cross validation on 3 standard datasets.