Flexora: Flexible Low Rank Adaptation for Large Language Models

v1v2 (latest)

Flexora: Flexible Low Rank Adaptation for Large Language Models

20 August 2024

Ying Tiffany He

ArXiv (abs)PDF HTML

Papers citing "Flexora: Flexible Low Rank Adaptation for Large Language Models"

14 / 14 papers shown

Title
LoRA Dropout as a Sparsity Regularizer for Overfitting Control Yang Lin Xinyu Ma Xu Chu Yujie Jin Zhibang Yang Yasha Wang Hong-yan Mei 90 26 0 15 Apr 2024
LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning Boyao Wang Xiang Liu Shizhe Diao Renjie Pi Jipeng Zhang Chi Han Tong Zhang 95 53 0 26 Mar 2024
LoRA-drop: Efficient LoRA Parameter Pruning based on Output Evaluation Hongyun Zhou Xiangyu Lu Wang Xu Conghui Zhu Tiejun Zhao Muyun Yang 63 18 0 12 Feb 2024
The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction Pratyusha Sharma Jordan T. Ash Dipendra Kumar Misra LRM 74 90 0 21 Dec 2023
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 504 10,526 0 17 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 605 4,099 0 18 Apr 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 252 4,305 0 01 Jan 2021
PIQA: Reasoning about Physical Commonsense in Natural Language Yonatan Bisk Rowan Zellers Ronan Le Bras Jianfeng Gao Yejin Choi OOD LRM 192 1,847 0 26 Nov 2019
Truncated Back-propagation for Bilevel Optimization Amirreza Shaban Ching-An Cheng Nathan Hatch Byron Boots 111 266 0 25 Oct 2018
Bilevel Programming for Hyperparameter Optimization and Meta-Learning Luca Franceschi P. Frasconi Saverio Salzo Riccardo Grazzi Massimiliano Pontil 179 732 0 13 Jun 2018
Learning to Reweight Examples for Robust Deep Learning Mengye Ren Wenyuan Zeng Binh Yang R. Urtasun OOD NoLa 154 1,431 0 24 Mar 2018
RACE: Large-scale ReAding Comprehension Dataset From Examinations Guokun Lai Qizhe Xie Hanxiao Liu Yiming Yang Eduard H. Hovy ELM 205 1,359 0 15 Apr 2017
Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data Science Randal S. Olson Nathan Bartley Ryan J. Urbanowicz J. Moore 63 530 0 20 Mar 2016
Gradient-based Hyperparameter Optimization through Reversible Learning D. Maclaurin David Duvenaud Ryan P. Adams DD 236 946 0 11 Feb 2015