DoRA: Weight-Decomposed Low-Rank Adaptation

14 February 2024

Papers citing "DoRA: Weight-Decomposed Low-Rank Adaptation"

50 / 245 papers shown

Title
PaCA: Partial Connection Adaptation for Efficient Fine-Tuning Sunghyeon Woo Sol Namkung Sunwoo Lee Inho Jeong Beomseok Kim Dongsuk Jeon 39 0 0 28 Feb 2025
FedMentalCare: Towards Privacy-Preserving Fine-Tuned LLMs to Analyze Mental Health Status Using Federated Learning Framework S M Sarwar AI4MH 46 0 0 27 Feb 2025
Predicting Through Generation: Why Generation Is Better for Prediction Md. Kowsher Nusrat Jahan Prottasha Prakash Bhat Chun-Nam Yu Mojtaba Soltanalian Ivan Garibay O. Garibay Chen Chen Niloofar Yousefi AI4TS 65 0 0 25 Feb 2025
Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models Cao Yuxuan Wu Jiayang Alistair Cheong Liang Chuen Bryan Shan Guanrong Theodore Lee Chong Jen Sherman Chann Zhi Shen 179 0 0 25 Feb 2025
SECURA: Sigmoid-Enhanced CUR Decomposition with Uninterrupted Retention and Low-Rank Adaptation in Large Language Models Yuxuan Zhang CLL ALM 73 1 0 25 Feb 2025
Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning Raghav Singhal Kaustubh Ponkshe Rohit Vartak Lav R. Varshney Praneeth Vepakomma FedML 79 0 0 24 Feb 2025
R-LoRA: Random Initialization of Multi-Head LoRA for Multi-Task Learning Jinda Liu Yi-Ju Chang Yuan Wu 59 0 0 24 Feb 2025
BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Models Yupeng Chang Yi-Ju Chang Yuan Wu AI4CE ALM 92 0 0 24 Feb 2025
NEAT: Nonlinear Parameter-efficient Adaptation of Pre-trained Models Yibo Zhong Haoxiang Jiang Lincan Li Ryumei Nakada Tianci Liu Linjun Zhang Huaxiu Yao Haoyu Wang 79 2 0 24 Feb 2025
MedForge: Building Medical Foundation Models Like Open Source Software Development Zheling Tan Kexin Ding Jin Gao Mu Zhou Dimitris N. Metaxas Shaoting Zhang Dequan Wang AI4CE 45 1 0 22 Feb 2025
Sparsity May Be All You Need: Sparse Random Parameter Adaptation Jesus Rios Pierre L. Dognin Ronny Luss K. Ramamurthy 32 1 0 21 Feb 2025
LSR-Adapt: Ultra-Efficient Parameter Tuning with Matrix Low Separation Rank Kernel Adaptation Xin Li Anand D. Sarwate 57 0 0 20 Feb 2025
GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning Sifan Zhou Shuo Wang Zhihang Yuan Mingjia Shi Yuzhang Shang Dawei Yang ALM MQ 90 0 0 18 Feb 2025
CoT-Valve: Length-Compressible Chain-of-Thought Tuning Xinyin Ma Guangnian Wan Runpeng Yu Gongfan Fang Xinchao Wang LRM 78 22 0 13 Feb 2025
DiTASK: Multi-Task Fine-Tuning with Diffeomorphic Transformations Krishna Sri Ipsit Mantri Carola-Bibiane Schönlieb Bruno Ribeiro Chaim Baskin Moshe Eliasof 43 0 0 09 Feb 2025
SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation Yixian Shen Qi Bi Jia-Hong Huang Hongyi Zhu Andy D. Pimentel Anuj Pathania 46 0 0 08 Feb 2025
LoCA: Location-Aware Cosine Adaptation for Parameter-Efficient Fine-Tuning Zhekai Du Yinjie Min Jingjing Li Ke Lu Changliang Zou Liuhua Peng Tingjin Chu Mingming Gong 174 1 0 05 Feb 2025
RandLoRA: Full-rank parameter-efficient fine-tuning of large models Paul Albert Frederic Z. Zhang Hemanth Saratchandran Cristian Rodriguez-Opazo Anton van den Hengel Ehsan Abbasnejad 96 1 0 03 Feb 2025
Sparse High Rank Adapters K. Bhardwaj N. Pandey Sweta Priyadarshi Viswanath Ganapathy Rafael Esteves ... P. Whatmough Risheek Garrepalli M. V. Baalen Harris Teague Markus Nagel MQ 43 4 0 28 Jan 2025
SEAL: Scaling to Emphasize Attention for Long-Context Retrieval Changhun Lee Jun-gyu Jin Younghyun Cho Eunhyeok Park LRM 56 0 0 28 Jan 2025
Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation Reza Akbarian Bafghi Carden Bagwell Avinash Ravichandran Ashish Shrivastava M. Raissi 48 0 0 28 Jan 2025
Parameter-Efficient Fine-Tuning for Foundation Models Dan Zhang Tao Feng Lilong Xue Yuandong Wang Yuxiao Dong J. Tang 46 8 0 23 Jan 2025
Language Fusion for Parameter-Efficient Cross-lingual Transfer Philipp Borchert Ivan Vulić Marie-Francine Moens Jochen De Weerdt 41 0 0 12 Jan 2025
RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation Jun Liu Zhenglun Kong Peiyan Dong Changdi Yang Xuan Shen ... Wei Niu Wenbin Zhang Xue Lin Dong Huang Yanzhi Wang ALM 46 2 0 08 Jan 2025
Spectral-Aware Low-Rank Adaptation for Speaker Verification Zhe Li Man-Wai Mak Mert Pilanci Hung-yi Lee Helen Meng 41 0 0 07 Jan 2025
DoTA: Weight-Decomposed Tensor Adaptation for Large Language Models Xiaolin Hu Xiang Cheng Peiyu Liu Wei Liu Jian Luan Bin Wang Y. Liu 42 1 0 31 Dec 2024
GaLore $+$ : Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection Xutao Liao Shaohui Li Yuhui Xu Zhi Li Y. Liu You He VLM 59 3 0 31 Dec 2024
YOLO-UniOW: Efficient Universal Open-World Object Detection Lihao Liu Juexiao Feng Hui Chen Ao Wang Lin Song J. Han Guiguang Ding ObjD VLM 49 2 0 31 Dec 2024
Segment-Based Attention Masking for GPTs Shahar Katz Liran Ringel Yaniv Romano Lior Wolf CLL 54 1 0 24 Dec 2024
Test-Time Alignment via Hypothesis Reweighting Yoonho Lee Jonathan Williams Henrik Marklund Archit Sharma E. Mitchell Anikait Singh Chelsea Finn 91 3 0 11 Dec 2024
PETALface: Parameter Efficient Transfer Learning for Low-resolution Face Recognition Kartik Narayan Nithin Gopalakrishnan Nair Jennifer Xu Rama Chellappa Vishal M. Patel CVBM CLL 70 1 0 10 Dec 2024
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models Fan Wang Juyong Jiang Chansung Park Sunghun Kim Jing Tang 94 1 0 08 Dec 2024
LQ-Adapter: ViT-Adapter with Learnable Queries for Gallbladder Cancer Detection from Ultrasound Image Chetan Madan Mayuna Gupta Soumen Basu Pankaj Gupta Chetan Arora 88 0 0 30 Nov 2024
DLaVA: Document Language and Vision Assistant for Answer Localization with Enhanced Interpretability and Trustworthiness Ahmad Mohammadshirazi Pinaki Prasad Guha Neogi Ser-Nam Lim R. Ramnath 70 1 0 29 Nov 2024
PEFTGuard: Detecting Backdoor Attacks Against Parameter-Efficient Fine-Tuning Zhen Sun Tianshuo Cong Yule Liu Chenhao Lin Xinlei He Rongmao Chen Xingshuo Han Xinyi Huang AAML 85 3 0 26 Nov 2024
COAP: Memory-Efficient Training with Correlation-Aware Gradient Projection Jinqi Xiao S. Sang Tiancheng Zhi Jing Liu Qing Yan Linjie Luo Bo Yuan Bo Yuan VLM 86 1 0 26 Nov 2024
RECAST: Reparameterized, Compact weight Adaptation for Sequential Tasks Nazia Tasnim Bryan A. Plummer CLL OffRL 74 0 0 25 Nov 2024
Multi LoRA Meets Vision: Merging multiple adapters to create a multi task model Ege Kesim Selahattin Serdar Helli MoMe 80 0 0 21 Nov 2024
Parameter Efficient Mamba Tuning via Projector-targeted Diagonal-centric Linear Transformation Seokil Ham H. Kim Sangmin Woo Changick Kim Mamba 192 0 0 21 Nov 2024
Hymba: A Hybrid-head Architecture for Small Language Models Xin Dong Y. Fu Shizhe Diao Wonmin Byeon Zijia Chen ... Min-Hung Chen Yoshi Suhara Y. Lin Jan Kautz Pavlo Molchanov Mamba 100 21 0 20 Nov 2024
StreamAdapter: Efficient Test Time Adaptation from Contextual Streams Dilxat Muhtar Yelong Shen Yuqing Yang Xiaodong Liu Yadong Lu ... Feng Sun Xueliang Zhang Jianfeng Gao Weizhu Chen Qi Zhang TTA 67 0 0 14 Nov 2024
FRUGAL: Memory-Efficient Optimization by Reducing State Overhead for Scalable Training Philip Zmushko Aleksandr Beznosikov Martin Takáč Samuel Horváth 44 0 0 12 Nov 2024
MapSAM: Adapting Segment Anything Model for Automated Feature Detection in Historical Maps Xue Xia Daiwei Zhang Wenxuan Song Wei Huang L. Hurni AI4TS VLM 28 0 0 11 Nov 2024
Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass Tong Chen Hao Fang Patrick Xia Xiaodong Liu Benjamin Van Durme Luke Zettlemoyer Jianfeng Gao Hao Cheng KELM 53 2 0 08 Nov 2024
Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation Ayan Sengupta Vaibhav Seth Arinjay Pathak Natraj Raman Sriram Gopalakrishnan Tanmoy Chakraborty BDL 28 2 0 07 Nov 2024
MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba Masakazu Yoshimura Teruaki Hayashi Yota Maeda Mamba 100 2 0 06 Nov 2024
Expanding Sparse Tuning for Low Memory Usage Shufan Shen Junshu Sun Xiangyang Ji Qingming Huang Shuhui Wang 45 0 0 04 Nov 2024
MALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi-Task Learning Xujia Wang Haiyan Zhao Shuo Wang Hanqing Wang Zhiyuan Liu MoMe MoE 37 0 0 30 Oct 2024
Meta-Learning Adaptable Foundation Models Jacob L. Block Sundararajan Srinivasan Liam Collins Aryan Mokhtari Sanjay Shakkottai 28 0 0 29 Oct 2024
LoRA vs Full Fine-tuning: An Illusion of Equivalence Reece Shuttleworth Jacob Andreas Antonio Torralba Pratyusha Sharma 35 10 0 28 Oct 2024