Multi-Task Gradient Boosting

Gradient Boosting Machines (GBMs) have revealed outstanding proficiency in various machine learning applications, such as classification and regression. Gradient boosting builds a set of regression models in an iterative process, in which at each iteration, a regressor model is trained to reduce a given loss on a given objective. This paper proposes an extension of gradient boosting that can handle multi-task problems, that is, problems in which the tasks share the attribute space but not necessarily the data distribution. The objective of the proposed algorithm is to split the GB process into two phases, one in which the base models learn the multiple interconnected tasks simultaneously, and a second one, in which different models are built to optimize the loss function on each task. The performance of proposed model shows a better overall performance with respect to models that learn the tasks independently and all tasks together in several multi-task regression and classification problems.