Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approach

Federated Learning (FL) aims to infer a shared model from private and decentralized data stored by multiple clients. Personalized FL (PFL) enhances the model's fit for each client by adapting the global model to the clients. A significant level of personalization is required for highly heterogeneous clients but can be challenging to achieve, especially when clients' datasets are small. To address this issue, we introduce the PAC-PFL framework for PFL of probabilistic models. PAC-PFL infers a shared hyper-posterior and treats each client's posterior inference as the personalization step. Unlike previous PFL algorithms, PAC-PFL does not regularize all personalized models towards a single shared model, thereby greatly enhancing its personalization flexibility. By establishing and minimizing a PAC-Bayesian generalization bound on the average true loss of clients, PAC-PFL effectively mitigates overfitting even in data-poor scenarios. Additionally, PAC-PFL provides generalization bounds for new clients joining later. PAC-PFL achieves accurate and well-calibrated predictions, as supported by our experiments.

@article{boroujeni2025_2401.08351,
  title={ Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approach },
  author={ Mahrokh Ghoddousi Boroujeni and Andreas Krause and Giancarlo Ferrari Trecate },
  journal={arXiv preprint arXiv:2401.08351},
  year={ 2025 }
}