The Expressive Power of Low-Rank Adaptation

26 October 2023

Papers citing "The Expressive Power of Low-Rank Adaptation"

49 / 49 papers shown

Title
Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Attacks? Zi Liang Haibo Hu Qingqing Ye Yaxin Xiao Ronghua Li AAML 0 0 0 19 May 2025
Fast RoPE Attention: Combining the Polynomial Method and Fast Fourier Transform Josh Alman Zhao-quan Song 0 0 0 17 May 2025
Diffusion Model Quantization: A Review Qian Zeng Chenggong Hu Mingli Song Jie Song MQ 45 0 0 08 May 2025
Fast and Low-Cost Genomic Foundation Models via Outlier Removal Haozheng Luo Chenghao Qiu Maojiang Su Zhihan Zhou Zoe Mehta Guo Ye Jerry Yao-Chieh Hu Han Liu AAML 55 1 0 01 May 2025
Theoretical Foundation of Flow-Based Time Series Generation: Provable Approximation, Generalization, and Efficiency Jiangxuan Long Zhao-quan Song Chiwun Yang AI4TS 162 0 0 18 Mar 2025
RaSA: Rank-Sharing Low-Rank Adaptation Zhiwei He Zhaopeng Tu Xing Wang Xingyu Chen Zhaobo Wang Jiahao Xu Tian Liang Wenxiang Jiao Zhiwei Zhang Rui Wang ALM 90 1 0 16 Mar 2025
Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization Ziqing Xu Hancheng Min Lachlan Ewen MacDonald Jinqi Luo Salma Tarmoun Enrique Mallada Rene Vidal AI4CE 56 0 0 10 Mar 2025
AdaptSR: Low-Rank Adaptation for Efficient and Scalable Real-World Super-Resolution Cansu Korkmaz Nancy Mehta Radu Timofte 67 0 0 10 Mar 2025
GBT-SAM: Adapting a Foundational Deep Learning Model for Generalizable Brain Tumor Segmentation via Efficient Integration of Multi-Parametric MRI Data Cecilia Diana-Albelda Roberto Alcover-Couso Álvaro García-Martín Jesús Bescós Marcos Escudero-Viñolo 42 1 0 06 Mar 2025
The impact of allocation strategies in subset learning on the expressive power of neural networks Ofir Schlisselberg Ran Darshan 93 0 0 10 Feb 2025
Tensor Product Attention Is All You Need Yifan Zhang Yifeng Liu Huizhuo Yuan Zhen Qin Yang Yuan Q. Gu Andrew Chi-Chih Yao 77 9 0 11 Jan 2025
GraphLoRA: Structure-Aware Contrastive Low-Rank Adaptation for Cross-Graph Transfer Learning Zhe-Rui Yang Jindong Han Chang-Dong Wang Hao Liu OffRL 35 1 0 08 Jan 2025
YOLO-UniOW: Efficient Universal Open-World Object Detection Lihao Liu Juexiao Feng Hui Chen Ao Wang Lin Song J. Han Guiguang Ding ObjD VLM 49 2 0 31 Dec 2024
LoRA-Mini : Adaptation Matrices Decomposition and Selective Training Ayush Singh Rajdeep Aher Shivank Garg 74 1 0 24 Nov 2024
The effect of fine-tuning on language model toxicity Will Hawkins Brent Mittelstadt Chris Russell 33 4 0 21 Oct 2024
Fine-grained Attention I/O Complexity: Comprehensive Analysis for Backward Passes Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song Yufa Zhou 54 15 0 12 Oct 2024
Parameter-Efficient Fine-Tuning of State Space Models Kevin Galim Wonjun Kang Yuchen Zeng H. Koo Kangwook Lee 31 4 0 11 Oct 2024
How Much Can RAG Help the Reasoning of LLM? Jingyu Liu Jiaen Lin Yong Liu LRM 33 9 0 03 Oct 2024
Differentially Private Kernel Density Estimation Erzhi Liu Jerry Yao-Chieh Hu Alex Reneau Zhao Song Han Liu 66 3 0 03 Sep 2024
MoRe Fine-Tuning with 10x Fewer Parameters Wenxuan Tan Nicholas Roberts Tzu-Heng Huang Jitian Zhao John Cooper Samuel Guo Chengyu Duan Frederic Sala 34 0 0 30 Aug 2024
LBC: Language-Based-Classifier for Out-Of-Variable Generalization Kangjun Noh Baekryun Seong Hoyoon Byun Youngjun Choi Sungjin Song Kyungwoo Song 31 0 0 20 Aug 2024
Memorization Capacity for Additive Fine-Tuning with Small ReLU Networks Jy-yong Sohn Dohyun Kwon Seoyeon An Kangwook Lee 46 0 0 01 Aug 2024
Parameter-Efficient Fine-Tuning via Circular Convolution Aochuan Chen Jiashun Cheng Zijing Liu Ziqi Gao Fugee Tsung Yu Li Jia Li 56 2 0 27 Jul 2024
Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability Zhuoyan Xu Zhenmei Shi Yingyu Liang CoGe LRM 37 27 0 22 Jul 2024
Enhancing Parameter Efficiency and Generalization in Large-Scale Models: A Regularized and Masked Low-Rank Adaptation Approach Yuzhu Mao Siqi Ping Zihao Zhao Yang Liu Wenbo Ding 37 1 0 16 Jul 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 54 24 0 08 Jul 2024
Compress then Serve: Serving Thousands of LoRA Adapters with Little Overhead Rickard Brüel-Gabrielsson Jiacheng Zhu Onkar Bhardwaj Leshem Choshen Kristjan Greenewald Mikhail Yurochkin Justin Solomon 48 5 0 17 Jun 2024
Exploring Training on Heterogeneous Data with Mixture of Low-rank Adapters Yuhang Zhou Zihua Zhao Haolin Li Siyuan Du Jiangchao Yao Ya Zhang Yanfeng Wang MoMe MoE 40 3 0 14 Jun 2024
$$\textit{Trans-LoRA}$: towards data-free Transferable Parameter Efficient Finetuning$ $\textit{Trans-LoRA}$ : towards data-free Transferable Parameter Efficient Finetuning Runqian Wang Soumya Ghosh David D. Cox Diego Antognini Aude Oliva Rogerio Feris Leonid Karlinsky 37 1 0 27 May 2024
Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective Akiyoshi Tomihari Issei Sato 30 4 0 27 May 2024
LoRA Learns Less and Forgets Less D. Biderman Jose Javier Gonzalez Ortiz Jacob P. Portes Mansheej Paul Philip Greengard ... Sam Havens Vitaliy Chiley Jonathan Frankle Cody Blakeney John P. Cunningham CLL 37 110 0 15 May 2024
Efficiency in Focus: LayerNorm as a Catalyst for Fine-tuning Medical Visual Language Pre-trained Models Jiawei Chen Dingkang Yang Yue Jiang Mingcheng Li Jinjie Wei Xiaolu Hou Lihua Zhang 56 6 0 25 Apr 2024
FL-TAC: Enhanced Fine-Tuning in Federated Learning via Low-Rank, Task-Specific Adapter Clustering Siqi Ping Yuzhu Mao Yang Liu Xiao-Ping Zhang Wenbo Ding FedML 28 3 0 23 Apr 2024
ResLoRA: Identity Residual Mapping in Low-Rank Adaption Shuhua Shi Shaohan Huang Minghui Song Zhoujun Li Zihan Zhang Haizhen Huang Furu Wei Weiwei Deng Feng Sun Qi Zhang AI4CE 25 14 0 28 Feb 2024
Asymmetry in Low-Rank Adapters of Foundation Models Jiacheng Zhu Kristjan Greenewald Kimia Nadjahi Haitz Sáez de Ocáriz Borde Rickard Brüel-Gabrielsson Leshem Choshen Marzyeh Ghassemi Mikhail Yurochkin Justin Solomon 44 27 0 26 Feb 2024
LoRA+: Efficient Low Rank Adaptation of Large Models Soufiane Hayou Nikhil Ghosh Bin Yu AI4CE 37 141 0 19 Feb 2024
LoRA Training in the NTK Regime has No Spurious Local Minima Uijeong Jang Jason D. Lee Ernest K. Ryu 44 14 0 19 Feb 2024
Efficient Fine-tuning of Audio Spectrogram Transformers via Soft Mixture of Adapters Umberto Cappellazzo Daniele Falavigna A. Brutti MoE 43 2 0 01 Feb 2024
From RAG to QA-RAG: Integrating Generative AI for Pharmaceutical Regulatory Compliance Process Jaewoong Kim Moohong Min 49 0 0 26 Jan 2024
SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model Yangfan Zhan Zhitong Xiong Yuan. Yuan MLLM 80 40 0 18 Jan 2024
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment Lingling Xu Haoran Xie S. J. Qin Xiaohui Tao F. Wang 49 135 0 19 Dec 2023
RefinedFields: Radiance Fields Refinement for Unconstrained Scenes Karim Kassab Antoine Schnepf Jean-Yves Franceschi Laurent Caraffa Jeremie Mary Valérie Gouet-Brunet VGen 30 7 0 01 Dec 2023
The Learnability of In-Context Learning Noam Wies Yoav Levine Amnon Shashua 122 91 0 14 Mar 2023
A Kernel-Based View of Language Model Fine-Tuning Sadhika Malladi Alexander Wettig Dingli Yu Danqi Chen Sanjeev Arora VLM 70 60 0 11 Oct 2022
Model Reprogramming: Resource-Efficient Cross-Domain Machine Learning Pin-Yu Chen VLM 109 58 0 22 Feb 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 389 8,495 0 28 Jan 2022
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018
Benefits of depth in neural networks Matus Telgarsky 148 602 0 14 Feb 2016