The main challenge of feature selection is overcoming the curse of dimensionality. In this paper, a new Bacterial Colony Optimization method with Multi-Dimensional Population, abbreviated as BCO-MDP, is presented for feature selection for the purpose of classification. To address the combinational problem associated with feature selection, the population with multiple dimensionalities is represented by subsets of different feature sizes. The population is grouped in terms of ‘Tribes’. The sizes of the feature subsets within a tribe are equal, while the dimensionalities differ when they belong to different tribes to achieve parallel solutions. The features are identified by their contributions to the most promising solutions in the total population and the classification performances of their tribes. A search is then conducted for the optimal feature subsets with varying dimensionalities. The convergence speed can be enhanced by a variety of exchange strategies within and between tribes. The proposed BCO-MDP method is demonstrated to be superior to the binary algorithms in terms of feature size and efficiency, while having a lower computational complexity in comparison to other population-based algorithms with constant dimensionality.