v1v2 (latest)

DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation

14 October 2022

Papers citing "DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation"

31 / 131 papers shown

Title
Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent Xiaoyan Yu Tongxu Luo Yifan Wei Fangyu Lei Yiming Huang Peng Hao Liehuang Zhu LLMAG 103 28 0 21 Feb 2024
LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models Yifan Yang Jiajun Zhou Ngai Wong Zheng Zhang 73 8 0 18 Feb 2024
QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning Hossein Rajabzadeh Mojtaba Valipour Tianshu Zhu Marzieh S. Tahaei Hyock Ju Kwon Ali Ghodsi Boxing Chen Mehdi Rezagholizadeh 67 10 0 16 Feb 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 116 58 0 15 Feb 2024
Efficient and Scalable Fine-Tune of Language Models for Genome Understanding Huixin Zhan Ying Nian Wu Zijun Zhang ALM 42 1 0 12 Feb 2024
Let Your Graph Do the Talking: Encoding Structured Data for LLMs Bryan Perozzi Bahare Fatemi Dustin Zelle Anton Tsitsulin Mehran Kazemi Rami Al-Rfou Jonathan J. Halcrow GNN 82 69 0 08 Feb 2024
Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models Fangzhao Zhang Mert Pilanci AI4CE 149 21 0 04 Feb 2024
A Comprehensive Survey of Compression Algorithms for Language Models Seungcheol Park Jaehyeon Choi Sojin Lee U. Kang MQ 116 16 0 27 Jan 2024
StyleInject: Parameter Efficient Tuning of Text-to-Image Diffusion Models Mohan Zhou Yalong Bai Qing Yang Tiejun Zhao 49 0 0 25 Jan 2024
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment Lingling Xu Haoran Xie S. J. Qin Xiaohui Tao F. Wang 118 163 0 19 Dec 2023
MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRA Lang Yu Qin Chen Jie Zhou Liang He KELM 103 58 0 19 Dec 2023
Run LoRA Run: Faster and Lighter LoRA Implementations Daria Cherniuk A. Mikhalev Ivan Oseledets AI4CE 37 1 0 06 Dec 2023
Hyperparameter Optimization for Large Language Model Instruction-Tuning C. Tribes Sacha Benarroch-Lelong Peng Lu I. Kobyzev 67 13 0 01 Dec 2023
Towards Better Parameter-Efficient Fine-Tuning for Large Language Models: A Position Paper Chengyu Wang Junbing Yan Wei Zhang Jun Huang ALM 81 4 0 22 Nov 2023
Sparse Low-rank Adaptation of Pre-trained Language Models Ning Ding Xingtai Lv Qiaosen Wang Yulin Chen Bowen Zhou Zhiyuan Liu Maosong Sun 80 70 0 20 Nov 2023
Low-Rank Adaptation for Multilingual Summarization: An Empirical Study Chenxi Whitehouse Fantine Huot Jasmijn Bastings Mostafa Dehghani Chu-Cheng Lin Mirella Lapata 64 8 0 14 Nov 2023
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization Weiyang Liu Zeju Qiu Yao Feng Yuliang Xiu Yuxuan Xue ... Songyou Peng Yandong Wen Michael J. Black Adrian Weller Bernhard Schölkopf 104 72 0 10 Nov 2023
VeRA: Vector-based Random Matrix Adaptation D. J. Kopiczko Tijmen Blankevoort Yuki Markus Asano VLM 83 164 0 17 Oct 2023
Matrix Compression via Randomized Low Rank and Low Precision Factorization R. Saha Varun Srivastava Mert Pilanci 85 23 0 17 Oct 2023
Low-rank Adaptation of Large Language Model Rescoring for Parameter-Efficient Speech Recognition Yu Yu Chao-Han Huck Yang J. Kolehmainen Prashanth Gurunath Shivakumar Yile Gu ... Denis Filimonov Shalini Ghosh A. Stolcke Ariya Rastrow I. Bulyko 103 10 0 26 Sep 2023
QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models Yuhui Xu Lingxi Xie Xiaotao Gu Xin Chen Heng Chang Hengheng Zhang Zhensu Chen Xiaopeng Zhang Qi Tian MQ 75 108 0 26 Sep 2023
Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference Parsa Kavehzadeh Mojtaba Valipour Marzieh S. Tahaei Ali Ghodsi Boxing Chen Mehdi Rezagholizadeh 87 6 0 16 Sep 2023
Delta-LoRA: Fine-Tuning High-Rank Parameters with the Delta of Low-Rank Matrices Bojia Zi Xianbiao Qi Lingzhi Wang Jianan Wang Kam-Fai Wong Lei Zhang 96 47 0 05 Sep 2023
A Survey on Model Compression for Large Language Models Xunyu Zhu Jian Li Yong Liu Can Ma Weiping Wang 139 233 0 15 Aug 2023
Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey Chen Ling Xujiang Zhao Jiaying Lu Chengyuan Deng Can Zheng ... Chris White Quanquan Gu Jian Pei Carl Yang Liang Zhao ALM 169 139 0 30 May 2023
Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models Oana Ignat Zhijing Jin Artem Abzaliev Laura Biester Santiago Castro ... Verónica Pérez-Rosas Siqi Shen Zekun Wang Winston Wu Rada Mihalcea LRM 139 6 0 21 May 2023
Visual Tuning Bruce X. B. Yu Jianlong Chang Haixin Wang Lin Liu Shijie Wang ... Lingxi Xie Haojie Li Zhouchen Lin Qi Tian Chang Wen Chen VLM 174 41 0 10 May 2023
AMR Parsing with Instruction Fine-tuned Pre-trained Language Models Young-Suk Lee Ramón Fernández Astudillo Radu Florian Tahira Naseem Salim Roukos 66 4 0 24 Apr 2023
AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning Han Zhou Xingchen Wan Ivan Vulić Anna Korhonen 81 48 0 28 Jan 2023
Soft Alignment Objectives for Robust Adaptation of Language Generation Michal vStefánik Marek Kadlcík Petr Sojka 77 2 0 29 Nov 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 156 114 0 31 Aug 2022