Imbalanced least squares regression with adaptive weight learning

Least squares regression (LSR) has demonstrated promising performance in various classification tasks owing to its effectiveness and efficiency. However, there are some deficiencies that seriously hinder its application in imbalanced data scenarios. The first is that LSR strongly relies on a balanced class distribution. A severely imbalanced class distribution may seriously damage the effectiveness of the algorithm. Second, the utilized binary label matrix in the conventional LSR model may be too strict to learn a discriminative transformation matrix for imbalanced learning. To address the above issues, in this paper, an adaptive weight learning mechanism and label relaxation constraint are proposed and incorporated into the framework of LSR to tackle the imbalanced classification problem. The weight of each sample can be adaptively obtained according to the original distribution information of the imbalanced data, in which the importance of minority class samples can be better reflected with larger weights. A new label relaxation matrix consisting of the original label matrix and auxiliary matrix is constructed to widen the margins between different classes. Further, we provide an iterative algorithm with fast convergence to solve the resulting optimization problem. Extensive experimental results on diverse binary-class and multi-class imbalanced datasets show that the proposed method outperforms many other state-of-the-art imbalanced learning approaches.