Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey

21 March 2024

Papers citing "Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey"

50 / 225 papers shown

Title
Analytic Subspace Routing: How Recursive Least Squares Works in Continual Learning of Large Language Model Kai Tong Kang Pan Xiao Zhang Erli Meng Run He Yawen Cui Nuoyan Guo Huiping Zhuang KELM CLL 59 0 0 17 Mar 2025
A Survey on Federated Fine-tuning of Large Language Models Yebo Wu Chunlin Tian Jingguang Li He Sun Kahou Tam Li Li Chengzhong Xu FedML 81 0 0 15 Mar 2025
FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-the-World LoRA Jieming Bian Lei Wang Letian Zhang Jie Xu 52 1 0 14 Mar 2025
Empirical Computation Eric Tang Marcel Böhme 49 0 0 13 Mar 2025
Compositional Subspace Representation Fine-tuning for Adaptive Large Language Models Andy Zhou MoMe 92 0 0 13 Mar 2025
Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More Arvid Frydenlund LRM 48 0 0 13 Mar 2025
Parameter-Efficient Adaptation of Geospatial Foundation Models through Embedding Deflection Romain Thoreau Valerio Marsocci Dawa Derksen AI4CE 71 2 0 12 Mar 2025
AI-native Memory 2.0: Second Me Jiale Wei Xiang Ying Tao Gao Fangyi Bao Felix Tao Jingbo Shang 56 1 0 11 Mar 2025
Fairness-Aware Low-Rank Adaptation Under Demographic Privacy Constraints Parameswaran Kamalaruban Mark Anderson Stuart Burrell Maeve Madigan Piotr Skalski David Sutton 54 0 0 07 Mar 2025
Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model Wenke Huang Jian Liang Xianda Guo Yiyang Fang Guancheng Wan ... Bin Yang He Li Jiawei Shao Mang Ye Bo Du OffRL LRM MLLM KELM VLM 65 1 0 06 Mar 2025
GBT-SAM: Adapting a Foundational Deep Learning Model for Generalizable Brain Tumor Segmentation via Efficient Integration of Multi-Parametric MRI Data Cecilia Diana-Albelda Roberto Alcover-Couso Álvaro García-Martín Jesús Bescós Marcos Escudero-Viñolo 42 1 0 06 Mar 2025
DM-Adapter: Domain-Aware Mixture-of-Adapters for Text-Based Person Retrieval Yating Liu Zimo Liu Xiangyuan Lan Wenming Yang Yaowei Li Q. Liao MoE 72 1 0 06 Mar 2025
Uncovering inequalities in new knowledge learning by large language models across different languages Chenglong Wang Haoyu Tang Xiyuan Yang Yueqi Xie Jina Suh ... Junming Huang Yu Xie Zhaoya Gong Xing Xie Fangzhao Wu 53 0 0 06 Mar 2025
Q&C: When Quantization Meets Cache in Efficient Image Generation Xin Ding X. Li Haotong Qin Zhibo Chen DiffM MQ 75 0 0 04 Mar 2025
Large AI Model for Delay-Doppler Domain Channel Prediction in 6G OTFS-Based Vehicular Networks Jianzhe Xue Dongcheng Yuan Zhanxi Ma Tiankai Jiang Yu Sun Haibo Zhou Xuemin Shen 65 0 0 03 Mar 2025
An Efficient Plugin Method for Metric Optimization of Black-Box Models Siddartha Devic Nurendra Choudhary Anirudh Srinivasan Sahika Genc B. Kveton G. Hiranandani 39 0 0 03 Mar 2025
Re-Imagining Multimodal Instruction Tuning: A Representation View Yiyang Liu James Liang Ruixiang Tang Yugyung Lee Majid Rabbani ... Raghuveer M. Rao Lifu Huang Dongfang Liu Qifan Wang Cheng Han 129 0 0 02 Mar 2025
LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM Yehonathan Refael Iftach Arbel Ofir Lindenbaum Tom Tirer 66 0 0 26 Feb 2025
CLLoRA: An Approach to Measure the Effects of the Context Length for LLM Fine-Tuning Ping Zhang Zhaorui Zhang Sheng Di Yao Xin Benben Liu 48 1 0 26 Feb 2025
BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Models Yupeng Chang Yi-Ju Chang Yuan Wu AI4CE ALM 89 0 0 24 Feb 2025
Parameter Efficient Merging for Multimodal Large Language Models with Complementary Parameter Adaptation Fanhu Zeng Haiyang Guo Fei Zhu Li Shen Hao Tang MoMe 54 1 0 24 Feb 2025
NEAT: Nonlinear Parameter-efficient Adaptation of Pre-trained Models Yibo Zhong Haoxiang Jiang Lincan Li Ryumei Nakada Tianci Liu Linjun Zhang Huaxiu Yao Haoyu Wang 75 2 0 24 Feb 2025
Testing the limits of fine-tuning to improve reasoning in vision language models Luca M. Schulze Buschoff Konstantinos Voudouris Elif Akata Matthias Bethge Joshua B. Tenenbaum Eric Schulz LRM VLM Presented at ResearchTrend Connect \| VLM on 14 Mar 2025 122 0 1 24 Feb 2025
R-LoRA: Random Initialization of Multi-Head LoRA for Multi-Task Learning Jinda Liu Yi-Ju Chang Yuan Wu 55 0 0 24 Feb 2025
Orthogonal Calibration for Asynchronous Federated Learning Jiayun Zhang Shuheng Li Haiyu Huang Xiaofan Yu Rajesh K. Gupta Jingbo Shang FedML 60 0 0 21 Feb 2025
GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning Sifan Zhou Shuo Wang Zhihang Yuan Mingjia Shi Yuzhang Shang Dawei Yang ALM MQ 87 0 0 18 Feb 2025
Mitigating Visual Knowledge Forgetting in MLLM Instruction-tuning via Modality-decoupled Gradient Descent Junda Wu Yuxin Xiong Xintong Li Yu Xia Ruoyu Wang ... Sungchul Kim Ryan Rossi Lina Yao Jingbo Shang Julian McAuley CLL VLM 51 0 0 17 Feb 2025
Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning Peizhuang Cong Wenpu Liu Wenhan Yu Haochen Zhao Tong Yang ALM MoE 81 0 0 06 Feb 2025
TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems Si-Yang Liu Han-Jia Ye 68 5 0 04 Feb 2025
RandLoRA: Full-rank parameter-efficient fine-tuning of large models Paul Albert Frederic Z. Zhang Hemanth Saratchandran Cristian Rodriguez-Opazo Anton van den Hengel Ehsan Abbasnejad 94 0 0 03 Feb 2025
CE-LoRA: Computation-Efficient LoRA Fine-Tuning for Language Models Guanduo Chen Yutong He Yipeng Hu Kun Yuan Binhang Yuan 46 0 0 03 Feb 2025
Memory-Efficient Fine-Tuning of Transformers via Token Selection Antoine Simoulin Namyong Park Xiaoyi Liu Grey Yang 110 0 0 31 Jan 2025
Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learning Ziyu Zhao Yixiao Zhou Didi Zhu Tao Shen X. Wang Jing Su Kun Kuang Zhongyu Wei Fei Wu Yu Cheng MoE 38 1 0 28 Jan 2025
Parameter-Efficient Fine-Tuning for Foundation Models Dan Zhang Tao Feng Lilong Xue Yuandong Wang Yuxiao Dong J. Tang 46 7 0 23 Jan 2025
Toyteller: AI-powered Visual Storytelling Through Toy-Playing with Character Symbols John Joon Young Chung Melissa Roemmele Max Kreminski VGen 72 0 0 23 Jan 2025
O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning Haotian Luo Li Shen Haiying He Y. Wang Shiwei Liu Wei Li Naiqiang Tan Xiaochun Cao Dacheng Tao VLM LRM 89 37 0 22 Jan 2025
Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Review Rock Yuren Pang Hope Schroeder Kynnedy Simone Smith Solon Barocas Ziang Xiao Emily Tseng Danielle Bragg 77 3 0 22 Jan 2025
A Hessian-informed hyperparameter optimization for differential learning rate Shiyun Xu Zhiqi Bu Yiliang Zhang Ian J. Barnett 39 1 0 12 Jan 2025
AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning Yehonathan Refael Jonathan Svirsky Boris Shustin Wasim Huleihel Ofir Lindenbaum 34 3 0 31 Dec 2024
Adapting Image-to-Video Diffusion Models for Large-Motion Frame Interpolation Luoxu Jin Hiroshi Watanabe DiffM VGen 101 0 0 22 Dec 2024
Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions Hao Du Shang Liu Lele Zheng Yang Cao Atsuyoshi Nakamura Lei Chen AAML 114 3 0 21 Dec 2024
Transducer Tuning: Efficient Model Adaptation for Software Tasks Using Code Property Graphs Imam Nur Bani Yusuf Lingxiao Jiang 79 0 0 18 Dec 2024
Efficient Fine-Tuning of Single-Cell Foundation Models Enables Zero-Shot Molecular Perturbation Prediction Sepideh Maleki Jan-Christian Huetter Kangway V Chuang Gabriele Scalia Tommaso Biancalani Tommaso Biancalani AI4CE 87 2 0 18 Dec 2024
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models Fan Wang Juyong Jiang Chansung Park Sunghun Kim Jing Tang 94 1 0 08 Dec 2024
Customize Segment Anything Model for Multi-Modal Semantic Segmentation with Mixture of LoRA Experts Chenyang Zhu Bin Xiao Lin Shi Shoukun Xu Xu Zheng MoE 91 9 0 05 Dec 2024
MIND: Effective Incorrect Assignment Detection through a Multi-Modal Structure-Enhanced Language Model Yunhe Pang Bo Chen Fanjin Zhang Yanghui Rao Jie Tang 82 0 0 05 Dec 2024
Mixture of Physical Priors Adapter for Parameter-Efficient Fine-Tuning Z. Wang C. J. Li QiXiang Ye Tong Zhang MoE 76 1 0 03 Dec 2024
Forensics Adapter: Adapting CLIP for Generalizable Face Forgery Detection Xinjie Cui Y. Li Ao Luo Jiaran Zhou Junyu Dong 122 4 0 29 Nov 2024
Task Arithmetic Through The Lens Of One-Shot Federated Learning Zhixu Tao I. Mason Sanjeev R. Kulkarni Xavier Boix MoMe FedML 84 3 0 27 Nov 2024
Challenges in Adapting Multilingual LLMs to Low-Resource Languages using LoRA PEFT Tuning Omkar Khade Shruti Jagdale Abhishek Phaltankar Gauri Takalikar Raviraj Joshi 73 3 0 27 Nov 2024