AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning

28 January 2023

Xingchen Wan

Papers citing "AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning"

42 / 42 papers shown

Title
Block Circulant Adapter for Large Language Models Xinyu Ding Meiqi Wang Siyu Liao Zhongfeng Wang 38 0 0 01 May 2025
NEAT: Nonlinear Parameter-efficient Adaptation of Pre-trained Models Yibo Zhong Haoxiang Jiang Lincan Li Ryumei Nakada Tianci Liu Linjun Zhang Huaxiu Yao Haoyu Wang 77 2 0 24 Feb 2025
A Survey of Personalized Large Language Models: Progress and Future Directions Jiahong Liu Zexuan Qiu Zhongyang Li Quanyu Dai Jieming Zhu Minda Hu Menglin Yang Irwin King LM&MA 54 2 0 17 Feb 2025
Memory-Efficient Fine-Tuning of Transformers via Token Selection Antoine Simoulin Namyong Park Xiaoyi Liu Grey Yang 112 0 0 31 Jan 2025
Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators Yinhong Liu Han Zhou Zhijiang Guo Ehsan Shareghi Ivan Vulić Anna Korhonen Nigel Collier ALM 132 69 0 20 Jan 2025
MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba Masakazu Yoshimura Teruaki Hayashi Yota Maeda Mamba 100 2 0 06 Nov 2024
Transfer Learning for Finetuning Large Language Models Tobias Strangmann Lennart Purucker Jörg Franke Ivo Rapant Fabio Ferreira Frank Hutter 44 0 0 02 Nov 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 49 3 0 24 Oct 2024
Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models Yuncheng Yang Yulei Qin Tong Wu Zihan Xu Gang Li ... Yuchen Shi Ke Li Xing Sun Jie Yang Yun Gu ALM OffRL MoE 52 0 0 28 Aug 2024
Better Python Programming for all: With the focus on Maintainability Karthik Shivashankar Antonio Martini KELM 41 0 0 17 Aug 2024
Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning Arijit Sehanobish Avinava Dubey Krzysztof Choromanski Somnath Basu Roy Chowdhury Deepali Jain Vikas Sindhwani Snigdha Chaturvedi ALM 40 1 0 25 Jun 2024
Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments Han Zhou Xingchen Wan Yinhong Liu Nigel Collier Ivan Vulić Anna Korhonen ALM 36 9 0 17 Jun 2024
Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning Naibin Gu Peng Fu Xiyu Liu Bowen Shen Zheng-Shen Lin Weiping Wang 38 6 0 06 Jun 2024
Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Vision Minglei Li Peng Ye Yongqi Huang Lin Zhang Tao Chen Tong He Jiayuan Fan Wanli Ouyang MoE 34 4 0 05 Jun 2024
Instruction Tuning With Loss Over Instructions Zhengyan Shi Adam X. Yang Bin Wu Laurence Aitchison Emine Yilmaz Aldo Lipani ALM 24 20 0 23 May 2024
Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning Jing Xu Jingzhao Zhang 39 7 0 04 May 2024
Towards Incremental Learning in Large Language Models: A Critical Review M. Jovanovic Peter Voss ELM CLL KELM 37 5 0 28 Apr 2024
Shears: Unstructured Sparsity with Neural Low-rank Adapter Search J. P. Muñoz Jinjie Yuan Nilesh Jain 27 7 0 16 Apr 2024
Personalized LLM Response Generation with Parameterized Memory Injection Kai Zhang Lizhi Qing Yangyang Kang 36 11 0 04 Apr 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 150 310 0 21 Mar 2024
Introducing Routing Functions to Vision-Language Parameter-Efficient Fine-Tuning with Low-Rank Bottlenecks Tingyu Qu Tinne Tuytelaars Marie-Francine Moens MoE 43 2 0 14 Mar 2024
FlexLLM: A System for Co-Serving Large Language Model Inference and Parameter-Efficient Finetuning Xupeng Miao Gabriele Oliaro Xinhao Cheng Vineeth Kada Ruohan Gao ... April Yang Yingcheng Wang Mengdi Wu Colin Unger Zhihao Jia MoE 94 9 0 29 Feb 2024
Bayesian Reward Models for LLM Alignment Adam X. Yang Maxime Robeyns Thomas Coste Zhengyan Shi Jun Wang Haitham Bou-Ammar Laurence Aitchison 32 17 0 20 Feb 2024
Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning? Cheng Han Qifan Wang Yiming Cui Wenguan Wang Lifu Huang Siyuan Qi Dongfang Liu VLM 49 19 0 23 Jan 2024
Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models Zhengxin Zhang Dan Zhao Xupeng Miao Gabriele Oliaro Qing Li Yong-jia Jiang Zhihao Jia MQ 36 7 0 13 Jan 2024
PEFT for Speech: Unveiling Optimal Placement, Merging Strategies, and Ensemble Techniques Tzu-Han Lin How-Shing Wang Hao-Yung Weng Kuang-Chen Peng Zih-Ching Chen Hung-yi Lee 34 5 0 04 Jan 2024
On Task Performance and Model Calibration with Supervised and Self-Ensembled In-Context Learning Chengzu Li Han Zhou Goran Glavavs Anna Korhonen Ivan Vulić 21 11 0 21 Dec 2023
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment Lingling Xu Haoran Xie S. J. Qin Xiaohui Tao F. Wang 46 132 0 19 Dec 2023
Mini but Mighty: Finetuning ViTs with Mini Adapters Imad Eddine Marouf Enzo Tartaglione Stéphane Lathuilière 28 5 0 07 Nov 2023
Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruning Han Zhou Xingchen Wan Ivan Vulić Anna Korhonen LLMAG 23 15 0 19 Oct 2023
Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering Han Zhou Xingchen Wan Lev Proleev Diana Mincu Jilin Chen Katherine A. Heller Subhrajit Roy UQLM 31 53 0 29 Sep 2023
SLoRA: Federated Parameter Efficient Fine-Tuning of Language Models Sara Babakniya A. Elkordy Yahya H. Ezzeldin Qingfeng Liu Kee-Bong Song Mostafa El-Khamy Salman Avestimehr 19 56 0 12 Aug 2023
Exploring the Impact of Model Scaling on Parameter-Efficient Tuning Yusheng Su Chi-Min Chan Jiali Cheng Yujia Qin Yankai Lin ... Ning Ding Xingzhi Sun Guotong Xie Zhiyuan Liu Maosong Sun 16 6 0 04 Jun 2023
Make Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning Baohao Liao Shaomu Tan Christof Monz KELM 23 29 0 01 Jun 2023
G-Adapter: Towards Structure-Aware Parameter-Efficient Transfer Learning for Graph Transformer Networks Anchun Gui Jinqiang Ye Han Xiao 22 19 0 17 May 2023
Bayesian Quadrature for Neural Ensemble Search Saad Hamid Xingchen Wan Martin Jørgensen Binxin Ru Michael A. Osborne BDL UQCV 28 1 0 15 Mar 2023
Modular Deep Learning Jonas Pfeiffer Sebastian Ruder Ivan Vulić E. Ponti MoMe OOD 32 73 0 22 Feb 2023
AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition Shoufa Chen Chongjian Ge Zhan Tong Jiangliu Wang Yibing Song Jue Wang Ping Luo 146 638 0 26 May 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,657 0 15 Oct 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 271 5,329 0 05 Nov 2016