LoRA+: Efficient Low Rank Adaptation of Large Models

19 February 2024

Papers citing "LoRA+: Efficient Low Rank Adaptation of Large Models"

50 / 115 papers shown

Title
AltLoRA: Towards Better Gradient Approximation in Low-Rank Adaptation with Alternating Projections Xin Yu Yujia Wang Jinghui Chen Lingzhou Xue 7 0 0 18 May 2025
ZeroTuning: Unlocking the Initial Token's Power to Enhance Large Language Models Without Training Feijiang Han Xiaodong Yu Jianheng Tang Lyle Ungar 4 0 0 16 May 2025
Bridging Human Oversight and Black-box Driver Assistance: Vision-Language Models for Predictive Alerting in Lane Keeping Assist Systems Yuhang Wang Hao Zhou VLM 2 0 0 14 May 2025
Behind Maya: Building a Multilingual Vision Language Model Nahid Alam Karthik Reddy Kanjula Surya Guthikonda Timothy Chung Bala Krishna S Vegesna ... Isha Chaturvedi Genta Indra Winata Ashvanth.S Snehanshu Mukherjee Alham Fikri Aji MLLM VLM 33 0 0 13 May 2025
ABS-Mamba: SAM2-Driven Bidirectional Spiral Mamba Network for Medical Image Translation Feng Yuan Yifan Gao Wenbin Wu Keqing Wu Xiaotong Guo Jie Jiang Xin Gao Mamba 51 0 0 12 May 2025
SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning Jinpeng Chen Runmin Cong Yuzhi Zhao Hongzheng Yang Guangneng Hu H. Ip Sam Kwong CLL KELM 83 0 0 05 May 2025
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models Zheng Lin Yuxin Zhang Zhe Chen Zihan Fang Xianhao Chen Praneeth Vepakomma Wei Ni Jun Luo Yue Gao MoE 49 2 0 05 May 2025
Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients Yezhen Wang Zhouhao Yang Brian K Chen Fanyi Pu Bo-wen Li Tianyu Gao Kenji Kawaguchi 46 0 0 03 May 2025
LLMSR@XLLM25: Less is More: Enhancing Structured Multi-Agent Reasoning via Quality-Guided Distillation Jiahao Yuan Xingzhe Sun Xing Yu Jingwen Wang Dehui Du Zhiqing Cui Zixiang Di LRM 38 0 0 23 Apr 2025
CSMF: Cascaded Selective Mask Fine-Tuning for Multi-Objective Embedding-Based Retrieval Hao Deng Haibo Xing Kanefumi Matsuyama Moyu Zhang Jinxin Hu Hong Wen Yu Zhang Xiaoyi Zeng Jing-Xuan Zhang 36 0 0 17 Apr 2025
DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models Wenjin Ke Zhe Li D. Li Lu Tian E. Barsoum MQ 37 1 0 12 Apr 2025
Towards Symmetric Low-Rank Adapters Tales Panoutsos Rodrygo L. T. Santos Flavio Figueiredo 33 0 0 29 Mar 2025
Concept-Aware LoRA for Domain-Aligned Segmentation Dataset Generation Minho Park S. Park Jungsoo Lee Hyojin Park Kyuwoong Hwang Fatih Porikli Jaegul Choo Sungha Choi 39 0 0 28 Mar 2025
An Overview of Low-Rank Structures in the Training and Adaptation of Large Models Laura Balzano Tianjiao Ding B. Haeffele Soo Min Kwon Qing Qu Peng Wang Zihan Wang Can Yaras OffRL AI4CE 62 0 0 25 Mar 2025
Hiding Images in Diffusion Models by Editing Learned Score Functions Haoyu Chen Yunqiao Yang Nan Zhong Kede Ma DiffM 68 0 0 24 Mar 2025
Serial Low-rank Adaptation of Vision Transformer Houqiang Zhong Shaocheng Shen Ke Cai Zhenglong Wu Jiangchao Yao Yuan Cheng Xuefei Li Xiaoyun Zhang Li-Na Song Qiang Hu 47 0 0 22 Mar 2025
HyperLoRA: Parameter-Efficient Adaptive Generation for Portrait Synthesis Mengtian Li Jinshu Chen Wanquan Feng Bingchuan Li Fei Dai Mingcong Liu Qian He 3DH 52 0 0 21 Mar 2025
Video-VoT-R1: An efficient video inference model integrating image packing and AoE architecture Cheng Li Jiexiong Liu Yixuan Chen Yanqin Jia MLLM VLM 76 0 0 20 Mar 2025
A Foundational individual Mobility Prediction Model based on Open-Source Large Language Models Zhenlin Qin Leizhen Wang Francisco Camara Pereira Zhenlinag Ma 39 0 0 19 Mar 2025
GKG-LLM: A Unified Framework for Generalized Knowledge Graph Construction Jian Zhang Bifan Wei Shihao Qi Haiping Zhu Jun Liu Qika Lin 47 0 0 14 Mar 2025
Parameter-Efficient Adaptation of Geospatial Foundation Models through Embedding Deflection Romain Thoreau Valerio Marsocci Dawa Derksen AI4CE 77 2 0 12 Mar 2025
Keeping Representation Similarity in Finetuning for Medical Image Analysis Wenqiang Zu Shenghao Xie Hao Chen Yiming Liang Lei Ma MedIm OOD 48 0 0 10 Mar 2025
Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization Ziqing Xu Hancheng Min Lachlan Ewen MacDonald Jinqi Luo Salma Tarmoun Enrique Mallada Rene Vidal AI4CE 56 0 0 10 Mar 2025
Adaptive Audio-Visual Speech Recognition via Matryoshka-Based Multimodal LLMs Umberto Cappellazzo Minsu Kim Stavros Petridis 57 0 0 09 Mar 2025
Quantum-PEFT: Ultra parameter-efficient fine-tuning Toshiaki Koike-Akino F. Tonin Yongtao Wu Frank Zhengqing Wu Leyla Naz Candogan V. Cevher MQ 54 3 0 07 Mar 2025
Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model Wenke Huang Jian Liang Xianda Guo Yiyang Fang Guancheng Wan ... Bin Yang He Li Jiawei Shao Mang Ye Bo Du OffRL LRM MLLM KELM VLM 65 1 0 06 Mar 2025
HaLoRA: Hardware-aware Low-Rank Adaptation for Large Language Models Based on Hybrid Compute-in-Memory Architecture Taiqiang Wu Chenchen Ding Wenyong Zhou Yuxin Cheng Xincheng Feng Shuqi Wang Chufan Shi Z. Liu Ngai Wong 58 0 0 27 Feb 2025
C-LoRA: Continual Low-Rank Adaptation for Pre-trained Models Xin Zhang Liang Bai Xian Yang Jiye Liang CLL 65 1 0 25 Feb 2025
K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs Ziheng Ouyang Zhen Li Qibin Hou MoMe OffRL 139 2 0 25 Feb 2025
Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models Cao Yuxuan Wu Jiayang Alistair Cheong Liang Chuen Bryan Shan Guanrong Theodore Lee Chong Jen Sherman Chann Zhi Shen 181 0 0 25 Feb 2025
Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment Chenghao Fan Zhenyi Lu Sichen Liu Xiaoye Qu Wei Wei Chengfeng Gu Yu-Xi Cheng MoE 168 0 0 24 Feb 2025
R-LoRA: Random Initialization of Multi-Head LoRA for Multi-Task Learning Jinda Liu Yi-Ju Chang Yuan Wu 59 0 0 24 Feb 2025
Swallowing the Poison Pills: Insights from Vulnerability Disparity Among LLMs Peng Yifeng Wu Zhizheng Chen Chen AAML 39 0 0 23 Feb 2025
Enhancing Adversarial Robustness of Vision-Language Models through Low-Rank Adaptation Yuheng Ji Yue Liu Zhicheng Zhang Zhao Zhang Yuting Zhao Gang Zhou Xingwei Zhang Xinwang Liu Xiaolong Zheng VLM 118 4 0 21 Feb 2025
A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models Mengyang Sun Yihao Wang Tao Feng Dan Zhang Yifan Zhu J. Tang MoE 43 0 0 20 Feb 2025
GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning Sifan Zhou Shuo Wang Zhihang Yuan Mingjia Shi Yuzhang Shang Dawei Yang ALM MQ 90 0 0 18 Feb 2025
Low Tensor-Rank Adaptation of Kolmogorov--Arnold Networks Yihang Gao Michael K. Ng Vincent Y. F. Tan 77 0 0 17 Feb 2025
GoRA: Gradient-driven Adaptive Low Rank Adaptation Haonan He Peng Ye Yuchen Ren Yuan Yuan Lei Chen AI4TS AI4CE 193 0 0 13 Feb 2025
Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning Peizhuang Cong Wenpu Liu Wenhan Yu Haochen Zhao Tong Yang ALM MoE 81 0 0 06 Feb 2025
QExplorer: Large Language Model Based Query Extraction for Toxic Content Exploration Shaola Ren Li Ke Longtao Huang Dehong Gao Hui Xue 41 0 0 06 Feb 2025
Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA Shuangyi Chen Yuanxin Guo Yue Ju Harik Dalal Ashish Khisti 48 2 0 03 Feb 2025
Sparse High Rank Adapters K. Bhardwaj N. Pandey Sweta Priyadarshi Viswanath Ganapathy Rafael Esteves ... P. Whatmough Risheek Garrepalli M. V. Baalen Harris Teague Markus Nagel MQ 43 4 0 28 Jan 2025
A Hessian-informed hyperparameter optimization for differential learning rate Shiyun Xu Zhiqi Bu Yiliang Zhang Ian Barnett 39 1 0 12 Jan 2025
Aggregating Low Rank Adapters in Federated Fine-tuning Evelyn Trautmann Ian Hales Martin F. Volk AI4CE FedML 39 0 0 10 Jan 2025
DoTA: Weight-Decomposed Tensor Adaptation for Large Language Models Xiaolin Hu Xiang Cheng Peiyu Liu Wei Liu Jian Luan Bin Wang Y. Liu 42 1 0 31 Dec 2024
DLaVA: Document Language and Vision Assistant for Answer Localization with Enhanced Interpretability and Trustworthiness Ahmad Mohammadshirazi Pinaki Prasad Guha Neogi Ser-Nam Lim R. Ramnath 70 1 0 29 Nov 2024
PEFTGuard: Detecting Backdoor Attacks Against Parameter-Efficient Fine-Tuning Zhen Sun Tianshuo Cong Yule Liu Chenhao Lin Xinlei He Rongmao Chen Xingshuo Han Xinyi Huang AAML 88 3 0 26 Nov 2024
Neural Precision Polarization: Simplifying Neural Network Inference with Dual-Level Precision Dinithi Jayasuriya Nastaran Darabi Maeesha Binte Hashem A. R. Trivedi MQ 34 1 0 06 Nov 2024
MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba Masakazu Yoshimura Teruaki Hayashi Yota Maeda Mamba 100 2 0 06 Nov 2024
Transfer Learning for Finetuning Large Language Models Tobias Strangmann Lennart Purucker Jörg Franke Ivo Rapant Fabio Ferreira Frank Hutter 49 0 0 02 Nov 2024