Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment

19 December 2023

Haoran Xie

Papers citing "Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment"

33 / 83 papers shown

Title
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 37 13 0 06 Jun 2024
Implicit Neural Image Field for Biological Microscopy Image Compression Gaole Dai Cheng-Ching Tseng Qingpo Wuwu Rongyu Zhang Shaokang Wang ... Yu Zhou A. A. Tuz Matthias Gunzer Jianxu Chen Shanghang Zhang 25 1 0 29 May 2024
Cross-Modal Safety Alignment: Is textual unlearning all you need? Trishna Chakraborty Erfan Shayegani Zikui Cai Nael B. Abu-Ghazaleh M. Salman Asif Yue Dong A. Roy-Chowdhury Chengyu Song 41 16 0 27 May 2024
Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation Shen Yuan Haotian Liu Hongteng Xu 44 2 0 24 May 2024
Worldwide Federated Training of Language Models Alexandru Iacob Lorenzo Sani Bill Marino Preslav Aleksandrov William F. Shen Nicholas D. Lane FedML 35 2 0 23 May 2024
Spectral Adapter: Fine-Tuning in Spectral Space Fangzhao Zhang Mert Pilanci 42 10 0 22 May 2024
FeTT: Continual Class Incremental Learning via Feature Transformation Tuning Sunyuan Qiang Xuxin Lin Yanyan Liang Jun Wan Du Zhang CLL 48 1 0 20 May 2024
BugBlitz-AI: An Intelligent QA Assistant Yi Yao Jun Wang Yabai Hu Lifeng Wang Yi Zhou Jack Chen Xuming Gai Zhenming Wang Wenjun Liu 19 1 0 17 May 2024
Refining Joint Text and Source Code Embeddings for Retrieval Task with Parameter-Efficient Fine-Tuning Karim Galliamov Leila Khaertdinova Karina Denisova 38 1 0 07 May 2024
A Survey on the Memory Mechanism of Large Language Model based Agents Zeyu Zhang Xiaohe Bo Chen Ma Rui Li Xu Chen Quanyu Dai Jieming Zhu Zhenhua Dong Ji-Rong Wen LLMAG KELM 42 107 0 21 Apr 2024
LoRA Dropout as a Sparsity Regularizer for Overfitting Control Yang Lin Xinyu Ma Xu Chu Yujie Jin Zhibang Yang Yasha Wang Hong-yan Mei 52 19 0 15 Apr 2024
How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model Han Gu Haoyu Dong Jichen Yang Maciej Mazurowski MedIm VLM 80 14 0 15 Apr 2024
Certified PEFTSmoothing: Parameter-Efficient Fine-Tuning with Randomized Smoothing Chengyan Fu Wenjie Wang AAML 32 0 0 08 Apr 2024
Facial Affective Behavior Analysis with Instruction Tuning Yifan Li Anh Dao Wentao Bao Zhen Tan Tianlong Chen Huan Liu Yu Kong CVBM 65 15 0 07 Apr 2024
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models Fanxu Meng Zhaohui Wang Muhan Zhang VLM 64 73 0 03 Apr 2024
A Review of Multi-Modal Large Language and Vision Models Kilian Carolan Laura Fennelly Alan F. Smeaton VLM 22 23 0 28 Mar 2024
Dual-Personalizing Adapter for Federated Foundation Models Yiyuan Yang Guodong Long Taoshu Shen Jing Jiang Michael Blumenstein 94 19 0 28 Mar 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 150 318 0 21 Mar 2024
PERL: Parameter Efficient Reinforcement Learning from Human Feedback Hakim Sidahmed Samrat Phatale Alex Hutcheson Zhuonan Lin Zhan Chen ... Jessica Hoffmann Hassan Mansoor Wei Li Abhinav Rastogi Lucas Dixon 38 2 0 15 Mar 2024
PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation Yizhe Xiong Hui Chen Tianxiang Hao Zijia Lin Jungong Han Yuesong Zhang Guoxin Wang Yongjun Bao Guiguang Ding 51 17 0 14 Mar 2024
In-Context Data Distillation with TabPFN Junwei Ma Valentin Thomas Guangwei Yu Anthony L. Caterini 36 7 0 10 Feb 2024
Let Your Graph Do the Talking: Encoding Structured Data for LLMs Bryan Perozzi Bahare Fatemi Dustin Zelle Anton Tsitsulin Mehran Kazemi Rami Al-Rfou Jonathan J. Halcrow GNN 42 57 0 08 Feb 2024
Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problem Maciej Wolczyk Bartłomiej Cupiał M. Ostaszewski Michal Bortkiewicz Michal Zajkac Razvan Pascanu Lukasz Kuciñski Piotr Milo's CLL 48 13 0 05 Feb 2024
Health Text Simplification: An Annotated Corpus for Digestive Cancer Education and Novel Strategies for Reinforcement Learning Md Mushfiqur Rahman Mohammad Sabik Irbaz Kai North Michelle S. Williams Marcos Zampieri Kevin Lybarger 35 0 0 26 Jan 2024
Parameter-Efficient Tuning with Special Token Adaptation Xiaoocong Yang James Y. Huang Wenxuan Zhou Muhao Chen 31 12 0 10 Oct 2022
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 130 100 0 24 May 2022
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer Tu Vu Brian Lester Noah Constant Rami Al-Rfou Daniel Cer VLM LRM 137 277 0 15 Oct 2021
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 151 178 0 13 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,858 0 18 Apr 2021
Memorisation versus Generalisation in Pre-trained Language Models Michael Tänzer Sebastian Ruder Marek Rei 94 50 0 16 Apr 2021
Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with $1/n$ Parameters Aston Zhang Yi Tay Shuai Zhang Alvin Chan A. Luu S. Hui Jie Fu MQ 182 83 0 17 Feb 2021
WARP: Word-level Adversarial ReProgramming Karen Hambardzumyan Hrant Khachatrian Jonathan May AAML 254 342 0 01 Jan 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 299 6,984 0 20 Apr 2018