Federated Multi-Task Learning: An Overview and Quantitative Evaluation

In recent, federated learning (FL) has emerged as a potential solution for large-scale distributed system networks, owing to the explosive growth of network devices and data. Despite FL’s numerous benefits, it suffers from statistical and system heterogeneity while training machine learning models. In this context, multi-task learning (MTL) is a suitable solution for dealing with the statistical challenges of federated settings. In other words, MTL has the capability to use knowledge from numerous related tasks which can enhance the generalization performance of all tasks. In this study, we provide a survey for federated multi-task learning (FMTL) from the standpoint of MTL techniques that are adopted in FL. More precisely, a classic definition of MTL is provided from the viewpoint of algorithmic modeling, and after that, the use-cases of two widely popular MTL techniques i.e., deep MTL technique and non deep MTL technique in three well-known FL categories: centralized, distributed, and hierarchical are discussed. Finally, several case studies in the form of numerical simulation are presented in this study.