FL-TAC: Enhanced Fine-Tuning in Federated Learning via Low-Rank, Task-Specific Adapter Clustering

23 April 2024

Papers citing "FL-TAC: Enhanced Fine-Tuning in Federated Learning via Low-Rank, Task-Specific Adapter Clustering"

28 / 28 papers shown

Title
On the Opportunities of Green Computing: A Survey You Zhou Xiujing Lin Xiang Zhang Maolin Wang Gangwei Jiang ... Chao Mou Shuai Han Wuxia Jin Guannan Zhang Xiaodong Zeng 76 10 0 01 Nov 2023
The Expressive Power of Low-Rank Adaptation Yuchen Zeng Kangwook Lee 98 66 0 26 Oct 2023
Towards Building the Federated GPT: Federated Instruction Tuning Jianyi Zhang Saeed Vahidian Martin Kuo Chunyuan Li Ruiyi Zhang Tong Yu Yufan Zhou Guoyin Wang Yiran Chen ALM FedML 85 131 0 09 May 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.5K 13,490 0 27 Feb 2023
On the Effectiveness of Parameter-Efficient Fine-Tuning Z. Fu Haoran Yang Anthony Man-Cho So Wai Lam Lidong Bing Nigel Collier 71 161 0 28 Nov 2022
Towards Efficient Communications in Federated Learning: A Contemporary Survey Zihao Zhao Yuzhu Mao Yang Liu Linqi Song Ouyang Ye Xinlei Chen Wenbo Ding FedML 95 62 0 02 Aug 2022
OPT: Open Pre-trained Transformer Language Models Susan Zhang Stephen Roller Naman Goyal Mikel Artetxe Moya Chen ... Daniel Simig Punit Singh Koura Anjali Sridhar Tianlu Wang Luke Zettlemoyer VLM OSLM AI4CE 375 3,700 0 02 May 2022
SAFARI: Sparsity enabled Federated Learning with Limited and Unreliable Communications Yuzhu Mao Zihao Zhao Meilin Yang Le Liang Yang Liu Wenbo Ding Tian-Shing Lan Xiaoping Zhang FedML 75 22 0 05 Apr 2022
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances Michael Ahn Anthony Brohan Noah Brown Yevgen Chebotar Omar Cortes ... Ted Xiao Peng Xu Sichun Xu Mengyuan Yan Andy Zeng LM&Ro 195 1,987 0 04 Apr 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 900 13,228 0 04 Mar 2022
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models Elad Ben-Zaken Shauli Ravfogel Yoav Goldberg 202 1,244 0 18 Jun 2021
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers Rabeeh Karimi Mahabadi James Henderson Sebastian Ruder MoE 119 492 0 08 Jun 2021
Data-Free Knowledge Distillation for Heterogeneous Federated Learning Zhuangdi Zhu Junyuan Hong Jiayu Zhou FedML 86 663 0 20 May 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 620 4,099 0 18 Apr 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 252 4,313 0 01 Jan 2021
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning Armen Aghajanyan Luke Zettlemoyer Sonal Gupta 110 571 1 22 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 684 41,563 0 22 Oct 2020
An Efficient Framework for Clustered Federated Learning Avishek Ghosh Jichan Chung Dong Yin Kannan Ramchandran FedML 89 870 0 07 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 908 42,520 0 28 May 2020
Federated learning with hierarchical clustering of local updates to improve training on non-IID data Christopher Briggs Zhong Fan Péter András FedML 89 579 0 24 Apr 2020
Survey of Personalization Techniques for Federated Learning V. Kulkarni Milind Kulkarni Aniruddha Pant FedML 230 331 0 19 Mar 2020
Federated Visual Classification with Real-World Data Distribution T. Hsu Qi Matthew Brown FedML 139 202 0 18 Mar 2020
Three Approaches for Personalization with Applications to Federated Learning Yishay Mansour M. Mohri Jae Hun Ro A. Suresh FedML 138 579 0 25 Feb 2020
Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints Felix Sattler K. Müller Wojciech Samek FedML 167 1,011 0 04 Oct 2019
Parameter-Efficient Transfer Learning for NLP N. Houlsby A. Giurgiu Stanislaw Jastrzebski Bruna Morrone Quentin de Laroussilhe Andrea Gesmundo Mona Attariyan Sylvain Gelly 226 4,529 0 02 Feb 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,324 0 11 Oct 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,201 0 20 Apr 2018
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 417 17,653 0 17 Feb 2016