A Cluster-based Privacy-Enhanced Hierarchical Federated Learning Framework with Secure Aggregation

Traditional machine learning typically requires training datasets on local machines or data centers. However, this approach may raise concerns related to data privacy and security. To address these issues, federated learning was proposed. However, federated learning, which involves a server communicating with multiple client devices, can significantly burden the server. Even when using hierarchical federated learning, there is still a considerable cost associated with communication at intermediate nodes. To further alleviate the communication cost burden on intermediate nodes, the most direct approach is to have each intermediate node select a subset of clients for training and accept their model parameters. However, client training data distributions are not uniform, leading to a state known as Non-Independent and Identically Distributed (Non-lID). Unthinkingly selecting clients for training may result in more imbalanced data selection and bias the model training in specific directions. Therefore, we propose the "post-clustering selection", where clients with similar data distributions are grouped together, and a certain proportion of clients are selected as representatives for training. This approach allows intermediate nodes to reduce communication costs while avoiding the selection of clients with highly imbalanced data distributions. Finally, we integrate differential privacy and secure aggregation to enhance privacy protection and present a framework called 'Cluster-based Privacy-Enhanced Hierarchical Federated Learning Framework with Secure Aggregation (CPE-HFL). From experiments, we reduce the communication volume by up to 29% while maintaining accuracy. Additionally, the accuracy improves more in cases with clustering than those without clustering. The proposed framework can reduce communication costs and effectively protect clients' privacy while maintaining model accuracy.