FedDRL: Trustworthy Federated Learning Model Fusion Method Based on Staged Reinforcement Learning

Abstract

Federated learning facilitates collaborative data analysis among multiple participants while preserving user privacy. However, conventional federated learning approaches, typically employing weighted average techniques for model fusion, confront two significant challenges: 1. The inclusion of malicious models in the fusion process can drastically undermine the accuracy of the aggregated global model. 2. Due to the heterogeneity problem of devices and data, the number of client samples does not determine the weight value of the model. To solve those challenges, we propose a trustworthy model fusion method based on reinforcement learning (FedDRL), which includes two stages. In the first stage, we propose a reliable client selection mechanism to exclude malicious models from the fusion process. In the second stage, we propose an adaptive model fusion method that dynamically assigns weights based on model quality to aggregate the best global models. Finally, we validate our approach against five distinct model fusion scenarios, demonstrating that our algorithm significantly enhanced reliability without compromising accuracy.