Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment

19 December 2023

Haoran Xie

Papers citing "Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment"

50 / 80 papers shown

Title
Camera Control at the Edge with Language Models for Scene Understanding Alexiy Buynitsky Sina Ehsani Bhanu Pallakonda Pragyana Mishra VLM 37 0 0 09 May 2025
Fine-Tuning Large Language Models and Evaluating Retrieval Methods for Improved Question Answering on Building Codes Mohammad Aqib Mohd Hamza Qipei Mei Ying Hei Chui RALM ELM 52 0 0 07 May 2025
Block Circulant Adapter for Large Language Models Xinyu Ding Meiqi Wang Siyu Liao Zhongfeng Wang 38 0 0 01 May 2025
COMPACT: COMPositional Atomic-to-Complex Visual Capability Tuning Xindi Wu Hee Seung Hwang Polina Kirichenko Olga Russakovsky VLM CoGe 68 0 0 30 Apr 2025
Can Differentially Private Fine-tuning LLMs Protect Against Privacy Attacks? Hao Du Shang Liu Yang Cao AAML 50 0 0 28 Apr 2025
Prompt-Tuning SAM: From Generalist to Specialist with only 2048 Parameters and 16 Training Images Tristan Piater Björn Barz Alexander Freytag VLM MedIm 64 0 0 23 Apr 2025
Efficient Parameter Adaptation for Multi-Modal Medical Image Segmentation and Prognosis Numan Saeed Shahad Hardan Muhammad Ridzuan Nada Saadi Karthik Nandakumar Mohammad Yaqub MedIm 139 0 0 18 Apr 2025
SynWorld: Virtual Scenario Synthesis for Agentic Action Knowledge Refinement Runnan Fang Xiaobin Wang Yuan Liang Shuofei Qiao Jialong Wu ... N. Zhang Yong-feng Jiang Pengjun Xie Fei Huang H. Chen LLMAG 69 0 0 04 Apr 2025
Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation Baban Gain Dibyanayan Bandyopadhyay Asif Ekbal LM&MA 56 1 0 02 Apr 2025
Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots Erfan Shayegani G M Shahariar Sara Abdali Lei Yu Nael B. Abu-Ghazaleh Yue Dong AAML 78 0 0 01 Apr 2025
Focus Directions Make Your Language Models Pay More Attention to Relevant Contexts Youxiang Zhu Ruochen Li Danqing Wang Daniel Haehn Xiaohui Liang LRM 57 1 0 30 Mar 2025
MoLEx: Mixture of Layer Experts for Finetuning with Sparse Upcycling R. Teo T. Nguyen MoE 58 2 0 14 Mar 2025
Supervised Fine-Tuning LLMs to Behave as Pedagogical Agents in Programming Education Emily Ross Yuval Kansal Jake Renzella Alexandra Vassar Andrew Taylor 36 1 0 27 Feb 2025
SubZero: Composing Subject, Style, and Action via Zero-Shot Personalization Shubhankar Borse K. Bhardwaj Mohammad Reza Karimi Dastjerdi Hyojin Park Shreya Kadambi ... Prathamesh Mandke Ankita Nayak Harris Teague Munawar Hayat Fatih Porikli DiffM 84 1 0 27 Feb 2025
BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Models Yupeng Chang Yi-Ju Chang Yuan Wu AI4CE ALM 89 0 0 24 Feb 2025
SAM-DA: Decoder Adapter for Efficient Medical Domain Adaptation Javier Gamazo Tejero Moritz Schmid Pablo Márquez-Neila M. Zinkernagel Sebastian Wolf Raphael Sznitman MedIm 41 0 0 12 Jan 2025
Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Tasks Jiacheng Hu Xiaoxuan Liao Jia Gao Zhen Qi Hongye Zheng Chihang Wang 33 1 0 25 Dec 2024
Efficient Adaptation of Multilingual Models for Japanese ASR Mark Bajo Haruka Fukukawa Ryuji Morita Yuma Ogasawara 72 0 0 14 Dec 2024
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models Fan Wang Juyong Jiang Chansung Park Sunghun Kim Jing Tang 94 1 0 08 Dec 2024
PEFT-as-an-Attack! Jailbreaking Language Models during Federated Parameter-Efficient Fine-Tuning Shenghui Li Edith C. H. Ngai Fanghua Ye Thiemo Voigt SILM 90 6 0 28 Nov 2024
Adaptive Blind All-in-One Image Restoration David Serrano-Lozano Luis Herranz Shaolin Su Javier Vázquez-Corral VLM 95 0 0 27 Nov 2024
Leveraging Multimodal Models for Enhanced Neuroimaging Diagnostics in Alzheimer's Disease Francesco Chiumento Mingming Liu LM&MA 48 0 0 12 Nov 2024
HAFLQ: Heterogeneous Adaptive Federated LoRA Fine-tuned LLM with Quantization Yang Su Na Yan Yansha Deng Mischa Dohler Robert Schober 37 0 0 10 Nov 2024
AI Ethics by Design: Implementing Customizable Guardrails for Responsible AI Development Kristina Šekrst Jeremy McHugh Jonathan Rodriguez Cefalu 67 0 0 05 Nov 2024
Lina-Speech: Gated Linear Attention is a Fast and Parameter-Efficient Learner for text-to-speech synthesis Théodor Lemerle Harrison Vanderbyl Vaibhav Srivastav Nicolas Obin Axel Roebel 37 1 0 30 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 49 3 0 24 Oct 2024
Stacking Small Language Models for Generalizability Laurence Liang LRM 21 0 0 21 Oct 2024
RoCoFT: Efficient Finetuning of Large Language Models with Row-Column Updates Md. Kowsher Tara Esmaeilbeig Chun-Nam Yu Mojtaba Soltanalian Niloofar Yousefi 27 0 0 14 Oct 2024
Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models Fei Wang Li Shen Liang Ding Chao Xue Ye Liu Changxing Ding 32 0 0 13 Oct 2024
Hybrid Training Approaches for LLMs: Leveraging Real and Synthetic Data to Enhance Model Performance in Domain-Specific Applications Alexey Zhezherau Alexei Yanockin SyDa 30 3 0 11 Oct 2024
Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation Grigory Malinovsky Umberto Michieli Hasan Hammoud Taha Ceritli Hayder Elesedy Mete Ozay Peter Richtárik AI4CE 32 2 0 10 Oct 2024
DLP-LoRA: Efficient Task-Specific LoRA Fusion with a Dynamic, Lightweight Plugin for Large Language Models Yuxuan Zhang Ruizhe Li MoMe 55 0 0 02 Oct 2024
LLM-Agent-UMF: LLM-based Agent Unified Modeling Framework for Seamless Integration of Multi Active/Passive Core-Agents Amine B. Hassouna Hana Chaari Ines Belhaj LLMAG 37 1 0 17 Sep 2024
LLMs4OL 2024 Overview: The 1st Large Language Models for Ontology Learning Challenge Hamed Babaei Giglou Jennifer D'Souza Sören Auer 41 8 0 16 Sep 2024
Towards Generalizable Scene Change Detection Jaewoo Kim Uehwan Kim 48 0 0 10 Sep 2024
CoRA: Optimizing Low-Rank Adaptation with Common Subspace of Large Language Models Xiaojun Xiao Sen Shen Qiming Bao Hongfei Rong Kairui Liu Zhongsheng Wang J. Liu 36 2 0 31 Aug 2024
Tuning a SAM-Based Model with Multi-Cognitive Visual Adapter to Remote Sensing Instance Segmentation Linghao Zheng Xinyang Pu Feng Xu 34 2 0 16 Aug 2024
Unleashing Artificial Cognition: Integrating Multiple AI Systems Muntasir Adnan Buddhi Gamage Zhiwei Xu Damith Herath Carlos Noschang Kuhn 25 0 0 09 Aug 2024
Polynomial Regression as a Task for Understanding In-context Learning Through Finetuning and Alignment Max Wilcoxson Morten Svendgård Ria Doshi Dylan Davis Reya Vir Anant Sahai 36 0 0 27 Jul 2024
CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning Emanuele Frascaroli Aniello Panariello Pietro Buzzega Lorenzo Bonicelli Angelo Porrello Simone Calderara VLM CLL 35 3 0 22 Jul 2024
MedSAGa: Few-shot Memory Efficient Medical Image Segmentation using Gradient Low-Rank Projection in SAM Navyansh Mahla Annie D'souza Shubh Gupta B. Kanekar Kshitij S. Jadhav VLM MedIm 27 2 0 21 Jul 2024
SwitchCIT: Switching for Continual Instruction Tuning of Large Language Models Xinbo Wu Max Hartman Vidhata Arjun Jayaraman L. Varshney CLL LRM 28 1 0 16 Jul 2024
Automated Text Scoring in the Age of Generative AI for the GPU-poor C. Ormerod Alexander Kwako 46 2 0 02 Jul 2024
Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning Arijit Sehanobish Avinava Dubey Krzysztof Choromanski Somnath Basu Roy Chowdhury Deepali Jain Vikas Sindhwani Snigdha Chaturvedi ALM 43 1 0 25 Jun 2024
From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Models Eleni Nisioti Claire Glanois Elias Najarro Andrew Dai Elliot Meyerson J. Pedersen Laetitia Teodorescu Conor F. Hayes Shyam Sudhakaran Sebastian Risi AI4CE LM&Ro 51 3 0 14 Jun 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang Xiaojie Li Zhongzhu Zhou Shuaiwen Leon Song Jianlong Wu Liqiang Nie Guohao Li 45 6 0 07 Jun 2024
LipGER: Visually-Conditioned Generative Error Correction for Robust Automatic Speech Recognition Sreyan Ghosh Sonal Kumar Ashish Seth Purva Chiniya Utkarsh Tyagi R. Duraiswami Dinesh Manocha 41 0 0 06 Jun 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 37 12 0 06 Jun 2024
Implicit Neural Image Field for Biological Microscopy Image Compression Gaole Dai Cheng-Ching Tseng Qingpo Wuwu Rongyu Zhang Shaokang Wang ... Yu Zhou A. A. Tuz Matthias Gunzer Jianxu Chen Shanghang Zhang 25 1 0 29 May 2024
Cross-Modal Safety Alignment: Is textual unlearning all you need? Trishna Chakraborty Erfan Shayegani Zikui Cai Nael B. Abu-Ghazaleh M. Salman Asif Yue Dong A. Roy-Chowdhury Chengyu Song 41 16 0 27 May 2024