Personalized Federated Learning Based on Hypernetworks and Attention Mechanism Ensembles for Internet of Things

Abstract

As the demand for data privacy protection continues to grow and the concept of collaborative modeling gains traction, federated learning has emerged as a pivotal distributed learning paradigm in the Internet of Things (IoT) domain. However, the client data held by different institutions often varies significantly in sources and characteristics, which can hinder the efficiency of federated learning model training and increase the risk of personal privacy breaches. To address the challenges of model accuracy degradation and privacy exposure when federated learning is applied to multi-source heterogeneous data, we propose a personalized federated learning strategy that integrates hypernetworks with attention mechanisms. This strategy involves transforming labeled data at the source to protect personal privacy while employing hypernetworks and Transformer-based mechanisms to focus on the personalized information of clients from various institutions. Our proposed approach supports handling heterogeneous data, thereby better meeting the personalized needs of different institutions. Experimental results demonstrate that this framework not only effectively safeguards data privacy but also significantly enhances the performance and generalization capability of federated learning on heterogeneous data. This research offers a novel perspective for developing more adaptable personalized federated learning models, facilitating cross-institutional collaborative research, and providing an innovative model training solution for various IoT devices, balancing the dual requirements of data privacy protection and multi-institutional data sharing.