Small-Scale Data Classification Based on Deep Forest

Developing effective and efficient small-scale data classification methods is very challenging in the digital age. Recent researches have shown that deep forest achieves a considerable increase in classification accuracy compared with general methods, especially when the training set is small. However, the standard deep forest may experience over-fitting and feature vanishing in dealing with small sample size. In this paper, we tackle this problem by proposing a skip connection deep forest (SForest), which can be viewed as a modification of the standard deep forest model. It leverages multi-class-grained scanning method to train multiple binary forest from different training sub-dataset of classes to encourage the diversity of ensemble and solve the class-imbalance problem. To expand the diversity of each layer in cascade forest, five different classifiers are employed. Meanwhile, the fitting quality of each classifiers is taken into consideration in representation learning. In addition, we propose a skip connection strategy to augment the feature vector, and use Gradient Boosting Decision Tree (GBDT) as the final classifier to improve the overall performance. Experiments demonstrated the proposed model achieved superior performance than the-state-of-the-art deep forest methods with almost the same parameter.