v1v2 (latest)

DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation

14 October 2022

Papers citing "DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation"

50 / 131 papers shown

Title
NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large Models Cheng Lin Lujun Li Dezhi Li Jie Zou Wei Xue Yike Guo AI4TS 78 8 0 18 Aug 2024
SeLoRA: Self-Expanding Low-Rank Adaptation of Latent Diffusion Model for Medical Image Synthesis Yuchen Mao Hongwei Bran Li Wei Pang G. Papanastasiou G. Yang Chengjia Wang MedIm 72 2 0 13 Aug 2024
BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Models Yupeng Chang Yi-Ju Chang Yuan Wu ALM AI4CE 159 0 0 08 Aug 2024
SARA: Singular-Value Based Adaptive Low-Rank Adaption Jihao Gu Shuai Chen Zelin Wang Yibo Zhang Ping Gong 80 3 0 06 Aug 2024
Accurate and Efficient Fine-Tuning of Quantized Large Language Models Through Optimal Balance Ao Shen Qiang Wang Zhiquan Lai Xionglve Li Dongsheng Li ALM MQ 54 1 0 24 Jul 2024
AI Act for the Working Programmer Holger Hermanns Anne Lauber-Rönsberg Philip Meinel Sarah Sterz Hanwei Zhang 69 1 0 23 Jul 2024
Enhancing Parameter Efficiency and Generalization in Large-Scale Models: A Regularized and Masked Low-Rank Adaptation Approach Yuzhu Mao Siqi Ping Zihao Zhao Yang Liu Wenbo Ding 93 1 0 16 Jul 2024
SwitchCIT: Switching for Continual Instruction Tuning of Large Language Models Xinbo Wu Max Hartman Vidhata Arjun Jayaraman Lav Varshney CLL LRM 110 1 0 16 Jul 2024
Low-Rank Interconnected Adaptation across Layers Yibo Zhong Jinman Zhao Yao Zhou OffRL MoE 114 1 0 13 Jul 2024
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences Nikolaos Dimitriadis Pascal Frossard François Fleuret MoE 255 8 0 10 Jul 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 136 41 0 08 Jul 2024
Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning Arijit Sehanobish Avinava Dubey Krzysztof Choromanski Somnath Basu Roy Chowdhury Deepali Jain Vikas Sindhwani Snigdha Chaturvedi ALM 81 3 0 25 Jun 2024
Word Matters: What Influences Domain Adaptation in Summarization? Yinghao Li Siyu Miao Heyan Huang Yang Gao 87 4 0 21 Jun 2024
Computation-Efficient Semi-Supervised Learning for ECG-based Cardiovascular Diseases Detection Rushuang Zhou Zijun Liu Lei A. Clifton David Clifton Kannie W. Y. Chan Yuan-Ting Zhang Yining Dong 56 1 0 20 Jun 2024
Bayesian-LoRA: LoRA based Parameter Efficient Fine-Tuning using Optimal Quantization levels and Rank Values trough Differentiable Bayesian Gates Cristian Meo Ksenia Sycheva Anirudh Goyal Justin Dauwels MQ 72 5 0 18 Jun 2024
Mixture-of-Subspaces in Low-Rank Adaptation Taiqiang Wu Jiahao Wang Zhe Zhao Ngai Wong 146 27 0 16 Jun 2024
Practical offloading for fine-tuning LLM on commodity GPU via learned sparse projectors Siyuan Chen Zelong Guan Yudong Liu Phillip B. Gibbons Phillip B. Gibbons 45 0 0 14 Jun 2024
ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models Jing Liu Ruihao Gong Mingyang Zhang Yefei He Jianfei Cai Bohan Zhuang MoE 73 0 0 13 Jun 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang Xiaojie Li Zhongzhu Zhou Shuaiwen Leon Song Jianlong Wu Liqiang Nie Guohao Li 110 14 0 07 Jun 2024
Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning Naibin Gu Peng Fu Xiyu Liu Bowen Shen Zheng Lin Weiping Wang 69 10 0 06 Jun 2024
Wings: Learning Multimodal LLMs without Text-only Forgetting Yi-Kai Zhang Shiyin Lu Yang Li Yanqing Ma Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang De-Chuan Zhan Han-Jia Ye VLM 128 10 0 05 Jun 2024
Low-Rank Adaption on Transformer-based Oriented Object Detector for Satellite Onboard Processing of Remote Sensing Images Xinyang Pu Feng Xu 70 3 0 04 Jun 2024
Editing the Mind of Giants: An In-Depth Exploration of Pitfalls of Knowledge Editing in Large Language Models Cheng-Hsun Hsueh Paul Kuo-Ming Huang Tzu-Han Lin Che-Wei Liao Hung-Chieh Fang Chao-Wei Huang Yun-Nung Chen KELM 82 6 0 03 Jun 2024
ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections Massimo Bini Karsten Roth Zeynep Akata Anna Khoreva 69 5 0 30 May 2024
Low-rank finetuning for LLMs: A fairness perspective Saswat Das Marco Romanelli Cuong Tran Zarreen Reza B. Kailkhura Ferdinando Fioretto 70 2 0 28 May 2024
Low-Rank Few-Shot Adaptation of Vision-Language Models Maxime Zanella Ismail Ben Ayed OffRL VLM 98 36 0 28 May 2024
DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distribution Yulong Mao Kaiyu Huang Changhao Guan Ganglin Bao Fengran Mo Jinan Xu 96 17 0 27 May 2024
Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation Shen Yuan Haotian Liu Hongteng Xu 81 5 0 24 May 2024
Sparse Spectral Training and Inference on Euclidean and Hyperbolic Neural Networks Jialin Zhao Yingtao Zhang Xinghang Li Huaping Liu C. Cannistraci 61 1 0 24 May 2024
Spectral Adapter: Fine-Tuning in Spectral Space Fangzhao Zhang Mert Pilanci 70 13 0 22 May 2024
Efficient Multimodal Large Language Models: A Survey Yizhang Jin Jian Li Yexin Liu Tianjun Gu Kai Wu ... Xin Tan Zhenye Gan Yabiao Wang Chengjie Wang Lizhuang Ma LRM 113 58 0 17 May 2024
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation Jie Xu Karthikeyan P. Saravanan Rogier van Dalen Haaris Mehmood David Tuckey Mete Ozay 173 8 0 10 May 2024
Parameter-Efficient Fine-Tuning with Discrete Fourier Transform Ziqi Gao Qichao Wang Aochuan Chen Zijing Liu Bingzhe Wu Liang Chen Jia Li 103 35 0 05 May 2024
Low-Rank Knowledge Decomposition for Medical Foundation Models Yuhang Zhou Haolin Li Siyuan Du Jiangchao Yao Ya Zhang Yanfeng Wang 64 3 0 26 Apr 2024
PORTULAN ExtraGLUE Datasets and Models: Kick-starting a Benchmark for the Neural Processing of Portuguese T. Osório Bernardo Leite Henrique Lopes Cardoso Luís Gomes João Rodrigues Rodrigo Santos António Branco 78 3 0 08 Apr 2024
DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model Chao Gao Sai Qian Zhang ALM 155 7 0 08 Apr 2024
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models Fanxu Meng Zhaohui Wang Muhan Zhang VLM 155 104 0 03 Apr 2024
A Single Linear Layer Yields Task-Adapted Low-Rank Matrices Hwichan Kim S. Sasaki Sho Hoshino Ukyo Honda 65 3 0 22 Mar 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 300 403 0 21 Mar 2024
BiLoRA: A Bi-level Optimization Framework for Overfitting-Resilient Low-Rank Adaptation of Large Pre-trained Models Rushi Qiang Ruiyi Zhang Pengtao Xie AI4CE 72 10 0 19 Mar 2024
Continual Forgetting for Pre-trained Vision Models Hongbo Zhao Bolin Ni Haochen Wang Junsong Fan Fei Zhu Yuxi Wang Yuntao Chen Gaofeng Meng Zhaoxiang Zhang MU VLM 136 13 0 18 Mar 2024
Introducing Routing Functions to Vision-Language Parameter-Efficient Fine-Tuning with Low-Rank Bottlenecks Tingyu Qu Tinne Tuytelaars Marie-Francine Moens MoE 84 2 0 14 Mar 2024
Block-wise LoRA: Revisiting Fine-grained LoRA for Effective Personalization and Stylization in Text-to-Image Generation Likun Li Haoqi Zeng Changpeng Yang Haozhe Jia Di Xu DiffM 80 5 0 12 Mar 2024
LORS: Low-rank Residual Structure for Parameter-Efficient Network Stacking Jialin Li Qiang Nie Weifu Fu Yuhuan Lin Guangpin Tao Yong-Jin Liu Chengjie Wang 91 5 0 07 Mar 2024
Controllable Generation with Text-to-Image Diffusion Models: A Survey Pu Cao Feng Zhou Qing-Huang Song Lu Yang 132 38 0 07 Mar 2024
RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models Saeed Najafi Alona Fyshe 78 2 0 04 Mar 2024
ResLoRA: Identity Residual Mapping in Low-Rank Adaption Shuhua Shi Shaohan Huang Minghui Song Zhoujun Li Zihan Zhang Haizhen Huang Furu Wei Weiwei Deng Feng Sun Qi Zhang AI4CE 74 15 0 28 Feb 2024
MELoRA: Mini-Ensemble Low-Rank Adapters for Parameter-Efficient Fine-Tuning Pengjie Ren Chengshun Shi Shiguang Wu Mengqi Zhang Zhaochun Ren Maarten de Rijke Zhumin Chen Jiahuan Pei MoE 209 13 0 27 Feb 2024
LLM Inference Unveiled: Survey and Roofline Model Insights Zhihang Yuan Yuzhang Shang Yang Zhou Zhen Dong Zhe Zhou ... Yong Jae Lee Yan Yan Beidi Chen Guangyu Sun Kurt Keutzer 233 91 0 26 Feb 2024
PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization Xiangdi Meng Damai Dai Weiyao Luo Zhe Yang Shaoxiang Wu Xiaochen Wang Peiyi Wang Qingxiu Dong Liang Chen Zhifang Sui 168 12 0 25 Feb 2024