FedGP: Correlation-Based Active Client Selection for Heterogeneous Federated Learning

Client-wise heterogeneity is one of the major issues that hinder effective training in federated learning (FL). Since the data distribution on each client may vary dramatically, the client selection strategy can largely influence the convergence rate of the FL process. Active client selection strategies are popularly adopted in recent studies. However, they neglect the loss correlations between the clients and achieve marginal improvement compared to the uniform selection strategy. In this work, we propose FedGP -- a federated learning framework built on a correlation-based client selection strategy, to boost the convergence rate of FL. Specifically, we first model the loss correlations between the clients with a Gaussian Process (GP). To make the GP training practical in the communication-bounded FL process, we develop a GP training method to reduce the communication cost by utilizing the covariance stationarity. Finally, based on the correlations we learned, we derive a client selection strategy with an enlarged reduction of expected global loss in each round. Our experimental results show that compared to the latest active client selection strategy, FedGP can improve the convergence rates by $1.3\sim2.0\times$ and $1.2\sim1.5\times$ on FMNIST and CIFAR-10, respectively.