VeRA: Vector-based Random Matrix Adaptation

17 October 2023

Papers citing "VeRA: Vector-based Random Matrix Adaptation"

50 / 106 papers shown

Title
Parameter-Efficient Fine-Tuning via Selective Discrete Cosine Transform Yixian Shen Qi Bi Jia-Hong Huang Hongyi Zhu Anuj Pathania 39 1 0 09 Oct 2024
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs Ruijia Niu D. Wu Rose Yu Yi Ma 38 1 0 09 Oct 2024
LoRTA: Low Rank Tensor Adaptation of Large Language Models Ignacio Hounie Charilaos I. Kanatsoulis Arnuv Tandon Alejandro Ribeiro 43 0 0 05 Oct 2024
AutoLoRA: AutoGuidance Meets Low-Rank Adaptation for Diffusion Models Artur Kasymov Marcin Sendera Michał Stypułkowski Maciej Ziȩba Przemysław Spurek 40 1 0 04 Oct 2024
MoS: Unleashing Parameter Efficiency of Low-Rank Adaptation with Mixture of Shards Sheng Wang Liheng Chen Pengan Chen Jingwei Dong Boyang Xue Jiyue Jiang Lingpeng Kong Chuan Wu MoE 31 8 0 01 Oct 2024
Mining Your Own Secrets: Diffusion Classifier Scores for Continual Personalization of Text-to-Image Diffusion Models Saurav Jha Shiqi Yang Masato Ishii Mengjie Zhao Christian Simon Muhammad Jehanzeb Mirza Dong Gong Lina Yao Shusuke Takahashi Yuki Mitsufuji DiffM 67 2 0 01 Oct 2024
LoRKD: Low-Rank Knowledge Decomposition for Medical Foundation Models Haolin Li Yuhang Zhou Ziheng Zhao Siyuan Du Jiangchao Yao Weidi Xie Ya Zhang Yanfeng Wang 36 1 0 29 Sep 2024
PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularization Yao Ni Shan Zhang Piotr Koniusz 253 5 0 25 Sep 2024
NanoVoice: Efficient Speaker-Adaptive Text-to-Speech for Multiple Speakers Nohil Park Heeseung Kim Che Hyun Lee Jooyoung Choi Jiheum Yeom Sungroh Yoon 31 2 0 24 Sep 2024
HUT: A More Computation Efficient Fine-Tuning Method With Hadamard Updated Transformation Geyuan Zhang Xiaofei Zhou Chuheng Chen 29 0 0 20 Sep 2024
Beyond LoRA: Exploring Efficient Fine-Tuning Techniques for Time Series Foundational Models Divij Gupta Anubhav Bhatti Surajsinh Parmar AI4TS 36 1 0 17 Sep 2024
Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data Atilla Akkus Mingjie Li Junjie Chu Junjie Chu Michael Backes Sinem Sav Sinem Sav SILM SyDa 53 1 0 12 Sep 2024
Advancing Depth Anything Model for Unsupervised Monocular Depth Estimation in Endoscopy Bojian Li Bo Liu Jinghua Yue F. Zhou Fugen Zhou MedIm MDE 53 2 0 12 Sep 2024
Exploring Intrinsic Language-specific Subspaces in Fine-tuning Multilingual Neural Machine Translation Zhe Cao Zhi Qu Hidetaka Kamigaito Taro Watanabe MoE 40 1 0 08 Sep 2024
SARA: Singular-Value Based Adaptive Low-Rank Adaption Jihao Gu Shuai Chen Zelin Wang Yibo Zhang Ping Gong 41 3 0 06 Aug 2024
Parameter-Efficient Fine-Tuning via Circular Convolution Aochuan Chen Jiashun Cheng Zijing Liu Ziqi Gao Fugee Tsung Yu-Feng Li Jia Li 61 2 0 27 Jul 2024
Enhancing Parameter Efficiency and Generalization in Large-Scale Models: A Regularized and Masked Low-Rank Adaptation Approach Yuzhu Mao Siqi Ping Zihao Zhao Yang Liu Wenbo Ding 37 1 0 16 Jul 2024
LoRA-PT: Low-Rank Adapting UNETR for Hippocampus Segmentation Using Principal Tensor Singular Values and Vectors Guanghua He Wangang Cheng Hancan Zhu Gaohang Yu 47 1 0 16 Jul 2024
RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization Xijie Huang Zechun Liu Shih-yang Liu Kwang-Ting Cheng MQ 43 8 0 10 Jul 2024
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences Nikolaos Dimitriadis Pascal Frossard François Fleuret MoE 72 6 0 10 Jul 2024
See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition Chongjie Si Xiaokang Yang Wei Shen 48 5 0 07 Jul 2024
Octo-planner: On-device Language Model for Planner-Action Agents Wei Chen Zhiyuan Li Zhen Guo Yikang Shen LM&Ro LLMAG 31 4 0 26 Jun 2024
Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs Ashwinee Panda Berivan Isik Xiangyu Qi Sanmi Koyejo Tsachy Weissman Prateek Mittal MoMe 50 15 0 24 Jun 2024
LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptation Seyedarmin Azizi Souvik Kundu Massoud Pedram 32 7 0 18 Jun 2024
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion Chenghao Fan Zhenyi Lu Wei Wei Jie Tian Xiaoye Qu Dangyang Chen Yu Cheng MoMe 55 5 0 17 Jun 2024
Mixture-of-Subspaces in Low-Rank Adaptation Taiqiang Wu Jiahao Wang Zhe Zhao Ngai Wong 54 22 0 16 Jun 2024
ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation Yurun Song Junchen Zhao Ian G. Harris Sangeetha Abdu Jyothi 34 3 0 16 Jun 2024
MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning Hanqing Wang Zeguan Xiao Shuo Wang Guanhua Chen Guanhua Chen 46 19 0 13 Jun 2024
The Impact of Initialization on LoRA Finetuning Dynamics Soufiane Hayou Nikhil Ghosh Bin Yu AI4CE 38 12 0 12 Jun 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 42 14 0 06 Jun 2024
Choice of PEFT Technique in Continual Learning: Prompt Tuning is Not All You Need Martin Wistuba Prabhu Teja Sivaprasad Lukas Balles Giovanni Zappella 42 0 0 05 Jun 2024
Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Vision Minglei Li Peng Ye Yongqi Huang Lin Zhang Tao Chen Tong He Jiayuan Fan Wanli Ouyang MoE 47 4 0 05 Jun 2024
LoFiT: Localized Fine-tuning on LLM Representations Fangcong Yin Xi Ye Greg Durrett 38 13 0 03 Jun 2024
Spectrum-Aware Parameter Efficient Fine-Tuning for Diffusion Models Xinxi Zhang Song Wen Ligong Han Felix Juefei Xu Akash Srivastava Junzhou Huang Hao Wang Molei Tao Dimitris N. Metaxas DiffM 41 6 0 31 May 2024
Low-rank finetuning for LLMs: A fairness perspective Saswat Das Marco Romanelli Cuong Tran Zarreen Reza B. Kailkhura Ferdinando Fioretto 42 1 0 28 May 2024
OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning Pengxiang Li Lu Yin Xiaowei Gao Shiwei Liu 38 7 0 28 May 2024
IAPT: Instruction-Aware Prompt Tuning for Large Language Models Wei-wei Zhu Aaron Xuxiang Tian Congrui Yin Yuan Ni Xiaoling Wang Guotong Xie 53 0 0 28 May 2024
LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters Klaudia Bałazy Mohammadreza Banaei Karl Aberer Jacek Tabor 43 19 0 27 May 2024
Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation Shen Yuan Haotian Liu Hongteng Xu 46 2 0 24 May 2024
Maintaining Structural Integrity in Parameter Spaces for Parameter Efficient Fine-tuning Chongjie Si Xuehui Wang Xue Yang Zhengqin Xu Qingyun Li Jifeng Dai Yu Qiao Xiaokang Yang Wei Shen 33 8 0 23 May 2024
MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning Ting Jiang Shaohan Huang Shengyue Luo Zihan Zhang Haizhen Huang ... Weiwei Deng Feng Sun Qi Zhang Deqing Wang Fuzhen Zhuang 40 35 0 20 May 2024
LoRA Learns Less and Forgets Less D. Biderman Jose Javier Gonzalez Ortiz Jacob P. Portes Mansheej Paul Philip Greengard ... Sam Havens Vitaliy Chiley Jonathan Frankle Cody Blakeney John P. Cunningham CLL 48 115 0 15 May 2024
ReFT: Representation Finetuning for Language Models Zhengxuan Wu Aryaman Arora Zheng Wang Atticus Geiger Daniel Jurafsky Christopher D. Manning Christopher Potts OffRL 43 58 0 04 Apr 2024
ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Models Zequan Liu Jiawen Lyn Wei-wei Zhu Xing Tian Yvette Graham OffRL 45 10 0 24 Mar 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 150 319 0 21 Mar 2024
Introducing Routing Functions to Vision-Language Parameter-Efficient Fine-Tuning with Low-Rank Bottlenecks Tingyu Qu Tinne Tuytelaars Marie-Francine Moens MoE 48 2 0 14 Mar 2024
MELoRA: Mini-Ensemble Low-Rank Adapters for Parameter-Efficient Fine-Tuning Pengjie Ren Chengshun Shi Shiguang Wu Mengqi Zhang Zhaochun Ren Maarten de Rijke Zhumin Chen Jiahuan Pei MoE 41 14 0 27 Feb 2024
PRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA Sheng Wang Boyang Xue Jiacheng Ye Jiyue Jiang Liheng Chen Lingpeng Kong Chuan Wu 35 14 0 24 Feb 2024
Advancing Parameter Efficiency in Fine-tuning via Representation Editing Muling Wu Wenhao Liu Xiaohua Wang Tianlong Li Changze Lv Zixuan Ling Jianhao Zhu Cenyuan Zhang Xiaoqing Zheng Xuanjing Huang 28 19 0 23 Feb 2024
LoRA+: Efficient Low Rank Adaptation of Large Models Soufiane Hayou Nikhil Ghosh Bin Yu AI4CE 48 149 0 19 Feb 2024