Collaborative and Efficient Personalization with Mixtures of Adaptors

4 October 2024

Abdulla Jasem Almansoori

Papers citing "Collaborative and Efficient Personalization with Mixtures of Adaptors"

31 / 31 papers shown

Title
Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architectures Yicheng Zhang Zhen Qin Zhaomin Wu Jian Hou Shuiguang Deng 134 3 0 28 Nov 2024
Multi-Task Dense Prediction via Mixture of Low-Rank Experts Yuqi Yang Peng-Tao Jiang Qibin Hou Hao Zhang Jinwei Chen Yue Liu MoE 52 20 0 26 Mar 2024
Federated Learning Can Find Friends That Are Advantageous N. Tupitsa Samuel Horváth Martin Takávc Eduard A. Gorbunov FedML 84 2 0 07 Feb 2024
Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models Fangzhao Zhang Mert Pilanci AI4CE 115 21 0 04 Feb 2024
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 67 27 0 15 Nov 2023
Provably Personalized and Robust Federated Learning Mariel A. Werner Lie He Michael I. Jordan Martin Jaggi Sai Praneeth Karimireddy FedML 57 10 0 14 Jun 2023
Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneity Konstantin Mishchenko Rustem Islamov Eduard A. Gorbunov Samuel Horváth FedML 77 11 0 29 May 2023
PaDPaF: Partial Disentanglement with Partially-Federated GANs Abdulla Jasem Almansoori Samuel Horváth Martin Takáč FedML 43 0 0 07 Dec 2022
Towards Understanding Mixture of Experts in Deep Learning Zixiang Chen Yihe Deng Yue-bo Wu Quanquan Gu Yuan-Fang Li MLT MoE 73 55 0 04 Aug 2022
Federated Learning with Partial Model Personalization Krishna Pillutla Kshitiz Malik Abdel-rahman Mohamed Michael G. Rabbat Maziar Sanjabi Lin Xiao FedML 78 166 0 08 Apr 2022
Federated Multi-Task Learning under a Mixture of Distributions Othmane Marfoq Giovanni Neglia A. Bellet Laetitia Kameni Richard Vidal FedML 105 278 0 23 Aug 2021
Federated Mixture of Experts M. Reisser Christos Louizos E. Gavves Max Welling FedML 71 24 0 14 Jul 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 442 10,328 0 17 Jun 2021
Initialization and Regularization of Factorized Neural Layers M. Khodak Neil A. Tenenholtz Lester W. Mackey Nicolò Fusi 122 57 0 03 May 2021
FedProto: Federated Prototype Learning across Heterogeneous Clients Yue Tan Guodong Long Lu Liu Tianyi Zhou Qinghua Lu Jing Jiang Chengqi Zhang FedML 224 483 0 01 May 2021
FedBN: Federated Learning on Non-IID Features via Local Batch Normalization Xiaoxiao Li Meirui Jiang Xiaofei Zhang Michael Kamp Qi Dou OOD FedML 276 815 0 15 Feb 2021
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning Armen Aghajanyan Luke Zettlemoyer Sonal Gupta 101 563 1 22 Dec 2020
Learning Mixtures of Low-Rank Models Yanxi Chen Cong Ma H. Vincent Poor Yuxin Chen 34 13 0 23 Sep 2020
An Efficient Framework for Clustered Federated Learning Avishek Ghosh Jichan Chung Dong Yin Kannan Ramchandran FedML 68 860 0 07 Jun 2020
Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent Tian Tong Cong Ma Yuejie Chi 66 117 0 18 May 2020
Personalized Federated Learning: A Meta-Learning Approach Alireza Fallah Aryan Mokhtari Asuman Ozdaglar FedML 154 568 0 19 Feb 2020
Sparsified SGD with Memory Sebastian U. Stich Jean-Baptiste Cordonnier Martin Jaggi 74 750 0 20 Sep 2018
Local SGD Converges Fast and Communicates Little Sebastian U. Stich FedML 170 1,063 0 24 May 2018
Measuring the Intrinsic Dimension of Objective Landscapes Chunyuan Li Heerad Farkhoor Rosanne Liu J. Yosinski 84 413 0 24 Apr 2018
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 823 11,909 0 09 Mar 2017
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer Noam M. Shazeer Azalia Mirhoseini Krzysztof Maziarz Andy Davis Quoc V. Le Geoffrey E. Hinton J. Dean MoE 248 2,653 0 23 Jan 2017
Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors Christos Louizos Max Welling BDL 60 257 0 15 Mar 2016
Evolutionary Multimodal Optimization: A Short Survey Ka-Chun Wong OffRL 32 28 0 03 Aug 2015
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.6K 100,386 0 04 Sep 2014
Speeding up Convolutional Neural Networks with Low Rank Expansions Max Jaderberg Andrea Vedaldi Andrew Zisserman 128 1,462 0 15 May 2014
Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization Alexander Rakhlin Ohad Shamir Karthik Sridharan 167 768 0 26 Sep 2011