Prototype-Based Decentralized Federated Learning for the Heterogeneous Time-Varying IoT Systems

Federated learning (FL) encounters two major obstacles, namely, the heterogeneity among Internet of Things (IoT) devices hampers both personalized and generalized performance, and the limited communication resources impede the learning efficiency. This article proposes a decentralized FL approach based on prototype representation learning (called DeProFL) and time-varying communication topology. The heterogeneity among devices can arise from varying data distributions and model architectures, leading to local or global gradient drift. To mitigate the effects of heterogeneity, we introduce prototype learning to establish consistent representations of samples and models across devices. Rather than directly propagating the large model weights, which require significant transmission volumes, we propagate prototype representations among local devices to reduce transmission. Given the limited communication resources of IoT and the dynamic wireless environments, we propose a time-varying decentralized FL approach to address these practical constraints. During each global iteration, every device shares its prototype only with adjacent devices for the collaborative learning, aiming to improve system stability and reduce network bandwidth usage. We have theoretically analyzed and verified the convergence of the DeProFL under nonconvex conditions. Extensive experiments have been applied to the benchmark data sets, and the results show that DeProFL outperforms state-of-the-art methods.