OLoRA: Orthonormal Low-Rank Adaptation of Large Language Models

3 June 2024

Papers citing "OLoRA: Orthonormal Low-Rank Adaptation of Large Language Models"

12 / 12 papers shown

Title
LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models Zhiqiang Hu Lei Wang Yihuai Lan Wanyu Xu Ee-Peng Lim Lidong Bing Xing Xu Soujanya Poria Roy Ka-wei Lee ALM 110 269 0 04 Apr 2023
Towards a Unified View of Parameter-Efficient Transfer Learning Junxian He Chunting Zhou Xuezhe Ma Taylor Berg-Kirkpatrick Graham Neubig AAML 132 952 0 08 Oct 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 490 10,496 0 17 Jun 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 250 4,299 0 01 Jan 2021
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning Armen Aghajanyan Luke Zettlemoyer Sonal Gupta 101 570 1 22 Dec 2020
Parameter-Efficient Transfer Learning with Diff Pruning Demi Guo Alexander M. Rush Yoon Kim 82 406 0 14 Dec 2020
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions Christopher Clark Kenton Lee Ming-Wei Chang Tom Kwiatkowski Michael Collins Kristina Toutanova 244 1,551 0 24 May 2019
To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks Matthew E. Peters Sebastian Ruder Noah A. Smith 79 437 0 14 Mar 2019
Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks Lei Huang Xianglong Liu B. Lang Adams Wei Yu Yongliang Wang Bo Li ODL 80 230 0 16 Sep 2017
Unitary Evolution Recurrent Neural Networks Martín Arjovsky Amar Shah Yoshua Bengio ODL 75 771 0 20 Nov 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 338 18,651 0 06 Feb 2015
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 183 1,852 0 20 Dec 2013