Efficient one-off clustering for personalized federated learning

In traditional federated learning such as FedAvg, the associations among clients are often ignored when executing on non-independent or heterogeneously distributed datasets, resulting in unsatisfactory accuracy. Although some previous works on clustered federated learning have been proposed to address such problems, most of them have a polarized problem. When the number of clustering is small, the model performs poorly and fails to accurately capture the distinction between clients. While a large number of clustering times tends to lead to higher communication costs. Therefore, a critical need is to design an efficient clustered federated solution that can both better capture the diversity between local clients and minimize the communication and computation costs. To this end, we propose an efficient one-off clustered federated learning framework called FedEOC. FedEOC exploits the "learning-to-learn" characteristic of meta-learning to enhance the generalization of the model across different clients so that only a small number of iterations are needed for each client to quickly obtain locally adapted weights. Based on the well-initially trained weights on all clients, we can cluster the clients only once to achieve the effect of one-off clustering and multiple-round applying. Additionally, to alleviate the issue of cluster imbalance, FedEOC is equipped with a Decomposition and Consolidation (Dec-Con) mechanism to decompose the clients from the extreme clusters and consolidate them into the most similar ones. The comprehensive experiments conducted on two real-world datasets demonstrate the superior capability of FedEOC from both aspects of accuracy and efficiency.