Robust logistic regression with shift parameter estimation

We investigate a shift parameter approach to logistic regression for robust classification. Shift parameter moves margin to the minimum of loss function. For robust estimation, margin-based logistic regression requires its own version of thresholding-type estimate which is different from residual-based regression. We discuss shift parameter estimation desirable to robust classification and propose some penalty functions producing such shift parameter estimates. Comparing to existing robust logistic regression methods requiring non-convex optimization or label transition modelling, our proposal is implemented in a simple alternating optimization: the classifier is obtained as a solution of conventional logistic regression with an offset and shift parameter is individually estimated in a closed form. We discuss some robust properties of the method and demonstrate its performance in linear and nonlinear classification with synthetic and real-world examples.