Low-rank finetuning for LLMs: A fairness perspective

Low-rank finetuning for LLMs: A fairness perspective

28 May 2024

Marco Romanelli

Ferdinando Fioretto

Papers citing "Low-rank finetuning for LLMs: A fairness perspective"

14 / 14 papers shown

Title
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection Jiawei Zhao Zhenyu Zhang Beidi Chen Zhangyang Wang A. Anandkumar Yuandong Tian 81 209 0 06 Mar 2024
Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors Ido Amos Jonathan Berant Ankit Gupta 47 28 0 04 Oct 2023
ReLoRA: High-Rank Training Through Low-Rank Updates Vladislav Lialin Namrata Shivagunde Sherin Muckatira Anna Rumshisky BDL 76 111 0 11 Jul 2023
DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation Mojtaba Valipour Mehdi Rezagholizadeh I. Kobyzev A. Ghodsi 63 176 0 14 Oct 2022
A Closer Look at How Fine-tuning Changes BERT Yichu Zhou Vivek Srikumar 39 68 0 27 Jun 2021
Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets Irene Solaiman Christy Dennison 85 224 0 18 Jun 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 373 10,273 0 17 Jun 2021
Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models Tongshuang Wu Marco Tulio Ribeiro Jeffrey Heer Daniel S. Weld 89 249 0 01 Jan 2021
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models Samuel Gehman Suchin Gururangan Maarten Sap Yejin Choi Noah A. Smith 133 1,194 0 24 Sep 2020
A Survey on Text Classification: From Shallow to Deep Learning Qian Li Hao Peng Jianxin Li Congyin Xia Renyu Yang Lichao Sun Philip S. Yu Lifang He VLM 52 343 0 02 Aug 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 385 20,114 0 23 Oct 2019
The Woman Worked as a Babysitter: On Biases in Language Generation Emily Sheng Kai-Wei Chang Premkumar Natarajan Nanyun Peng 271 642 0 03 Sep 2019
Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology Ran Zmigrod Sabrina J. Mielke Hanna M. Wallach Ryan Cotterell 61 281 0 11 Jun 2019
Racial Bias in Hate Speech and Abusive Language Detection Datasets Thomas Davidson Debasmita Bhattacharya Ingmar Weber 94 456 0 29 May 2019