Parameter-Efficient Fine-Tuning Design Spaces

4 January 2023

Jiaao Chen

Diyi Yang

Papers citing "Parameter-Efficient Fine-Tuning Design Spaces"

50 / 51 papers shown

Title
On the Robustness Tradeoff in Fine-Tuning Kunyang Li Jean-Charles Noirot Ferrand Ryan Sheatsley Blaine Hoak Yohan Beugin Eric Pauley Patrick McDaniel 61 0 0 19 Mar 2025
NEAT: Nonlinear Parameter-efficient Adaptation of Pre-trained Models Yibo Zhong Haoxiang Jiang Lincan Li Ryumei Nakada Tianci Liu Linjun Zhang Huaxiu Yao Haoyu Wang 77 2 0 24 Feb 2025
LoCA: Location-Aware Cosine Adaptation for Parameter-Efficient Fine-Tuning Zhekai Du Yinjie Min Jingjing Li Ke Lu Changliang Zou Liuhua Peng Tingjin Chu Mingming Gong 166 1 0 05 Feb 2025
TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems Si-Yang Liu Han-Jia Ye 68 6 0 04 Feb 2025
Parameter-Efficient Fine-Tuning for Foundation Models Dan Zhang Tao Feng Lilong Xue Yuandong Wang Yuxiao Dong J. Tang 46 8 0 23 Jan 2025
A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames Pinelopi Papalampidi Skanda Koppula Shreya Pathak Justin T Chiu Joseph Heyward Viorica Patraucean Jiajun Shen Antoine Miech Andrew Zisserman Aida Nematzdeh VLM 63 24 0 31 Dec 2024
MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba Masakazu Yoshimura Teruaki Hayashi Yota Maeda Mamba 100 2 0 06 Nov 2024
CleaR: Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label Learning Yeachan Kim Junho Kim SangKeun Lee NoLa AAML 35 2 0 31 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 49 3 0 24 Oct 2024
Towards Optimal Adapter Placement for Efficient Transfer Learning Aleksandra I. Nowak Otniel-Bogdan Mercea Anurag Arnab Jonas Pfeiffer Yann N. Dauphin Utku Evci 25 0 0 21 Oct 2024
Mitigating Forgetting in LLM Supervised Fine-Tuning and Preference Learning H. Fernando Han Shen Parikshit Ram Yi Zhou Horst Samulowitz Nathalie Baracaldo Tianyi Chen CLL 56 2 0 20 Oct 2024
Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models Kai Yao P. Gao Lichun Li Yuan Zhao Xiaofeng Wang Wei Wang Jianke Zhu 26 1 0 15 Oct 2024
AI Act for the Working Programmer Holger Hermanns Anne Lauber-Rönsberg Philip Meinel Sarah Sterz Hanwei Zhang 39 1 0 23 Jul 2024
Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software Dezhi Ran Mengzhou Wu Wei Yang Tao Xie AI4CE 36 1 0 11 Jul 2024
Towards Minimal Targeted Updates of Language Models with Targeted Negative Training Lily H. Zhang Rajesh Ranganath Arya Tafvizi 33 1 0 19 Jun 2024
SuperPos-Prompt: Enhancing Soft Prompt Tuning of Language Models with Superposition of Multi Token Embeddings MohammadAli SadraeiJavaeri Ehsaneddin Asgari A. Mchardy Hamid R. Rabiee VLM AAML 24 0 0 07 Jun 2024
Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Vision Minglei Li Peng Ye Yongqi Huang Lin Zhang Tao Chen Tong He Jiayuan Fan Wanli Ouyang MoE 34 4 0 05 Jun 2024
ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections Massimo Bini Karsten Roth Zeynep Akata Anna Khoreva 34 4 0 30 May 2024
Efficient Prompt Tuning by Multi-Space Projection and Prompt Fusion Pengxiang Lan Enneng Yang Yuting Liu Guibing Guo Linying Jiang Jianzhe Zhao Xingwei Wang VLM AAML 35 1 0 19 May 2024
Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning Jing Xu Jingzhao Zhang 39 7 0 04 May 2024
RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarization Dongqi Pu Vera Demberg 38 5 0 01 May 2024
Towards Incremental Learning in Large Language Models: A Critical Review M. Jovanovic Peter Voss ELM CLL KELM 37 5 0 28 Apr 2024
Facial Affective Behavior Analysis with Instruction Tuning Yifan Li Anh Dao Wentao Bao Zhen Tan Tianlong Chen Huan Liu Yu Kong CVBM 60 15 0 07 Apr 2024
Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation Xinyu Ma Xu Chu Zhibang Yang Yang Lin Xin Gao Junfeng Zhao 43 7 0 05 Apr 2024
Query-driven Relevant Paragraph Extraction from Legal Judgments Santosh T.Y.S.S Elvin Quero Hernandez Matthias Grabmair AILaw 42 2 0 31 Mar 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 150 310 0 21 Mar 2024
HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy Yongkang Liu Yiqun Zhang Qian Li Tong Liu Shi Feng Daling Wang Yifei Zhang Hinrich Schütze 40 6 0 26 Jan 2024
Design Principles for Generative AI Applications Justin D. Weisz Jessica He Michael J. Muller Gabriela Hoefer Rachel Miles Werner Geyer AI4CE 44 126 0 25 Jan 2024
PRILoRA: Pruned and Rank-Increasing Low-Rank Adaptation Nadav Benedek Lior Wolf 26 5 0 20 Jan 2024
MAPLE: Multilingual Evaluation of Parameter Efficient Finetuning of Large Language Models Divyanshu Aggarwal Ashutosh Sathe Ishaan Watts Sunayana Sitaram 29 1 0 15 Jan 2024
PEFT for Speech: Unveiling Optimal Placement, Merging Strategies, and Ensemble Techniques Tzu-Han Lin How-Shing Wang Hao-Yung Weng Kuang-Chen Peng Zih-Ching Chen Hung-yi Lee 37 5 0 04 Jan 2024
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment Lingling Xu Haoran Xie S. J. Qin Xiaohui Tao F. Wang 46 135 0 19 Dec 2023
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization Weiyang Liu Zeju Qiu Yao Feng Yuliang Xiu Yuxuan Xue ... Songyou Peng Yandong Wen Michael J. Black Adrian Weller Bernhard Schölkopf 50 57 0 10 Nov 2023
Unlearn What You Want to Forget: Efficient Unlearning for LLMs Jiaao Chen Diyi Yang MU 30 136 0 31 Oct 2023
Non-Intrusive Adaptation: Input-Centric Parameter-efficient Fine-Tuning for Versatile Multimodal Modeling Yaqing Wang Jialin Wu T. Dabral Jiageng Zhang Geoff Brown ... Frederick Liu Yi Liang Bo Pang Michael Bendersky Radu Soricut VLM 23 14 0 18 Oct 2023
Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuning Ted Zadouri A. Ustun Arash Ahmadian Beyza Ermics Acyr F. Locatelli Sara Hooker MoE 35 88 0 11 Sep 2023
Efficient Finetuning Large Language Models For Vietnamese Chatbot Vu-Thuan Doan Quoc-Truong Truong Duc-Vu Nguyen Vinh-Tiep Nguyen Thuy-Ngan Nguyen Luu ALM 35 5 0 09 Sep 2023
IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning Feiyu F. Zhang Liangzhi Li Jun-Cheng Chen Zhouqian Jiang Bowen Wang Yiming Qian 51 32 0 23 Aug 2023
PRISMA-DFLLM: An Extension of PRISMA for Systematic Literature Reviews using Domain-specific Finetuned Large Language Models Teo Susnjak 17 11 0 15 Jun 2023
Artificial General Intelligence for Medical Imaging Xiang Li Lu Zhang Zihao Wu Zheng Liu Lin Zhao ... Pingkuan Yan Quanzheng Li Wei Liu Tianming Liu Dinggang Shen LM&MA AI4CE 19 40 0 08 Jun 2023
Exploring the Impact of Model Scaling on Parameter-Efficient Tuning Yusheng Su Chi-Min Chan Jiali Cheng Yujia Qin Yankai Lin ... Ning Ding Xingzhi Sun Guotong Xie Zhiyuan Liu Maosong Sun 21 6 0 04 Jun 2023
BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks Kai Zhang Jun Yu Eashan Adhikarla Rong-Er Zhou Zhilin Yan ... Xun Chen Yong Chen Quanzheng Li Hongfang Liu Lichao Sun LM&MA MedIm 34 157 0 26 May 2023
Automated Few-shot Classification with Instruction-Finetuned Language Models Rami Aly Xingjian Shi Kaixiang Lin Aston Zhang A. Wilson 38 9 0 21 May 2023
G-Adapter: Towards Structure-Aware Parameter-Efficient Transfer Learning for Graph Transformer Networks Anchun Gui Jinqiang Ye Han Xiao 27 19 0 17 May 2023
LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models Zhiqiang Hu Lei Wang Yihuai Lan Wanyu Xu Ee-Peng Lim Lidong Bing Xing Xu Soujanya Poria Roy Ka-Wei Lee ALM 39 233 0 04 Apr 2023
Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning Vladislav Lialin Vijeta Deshpande Anna Rumshisky 33 167 0 28 Mar 2023
AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning Han Zhou Xingchen Wan Ivan Vulić Anna Korhonen 21 45 0 28 Jan 2023
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with $1/n$ Parameters Aston Zhang Yi Tay Shuai Zhang Alvin Chan A. Luu S. Hui Jie Fu MQ 179 83 0 17 Feb 2021
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,589 0 21 Jan 2020