Sparse Structure Search for Parameter-Efficient Tuning

Sparse Structure Search for Parameter-Efficient Tuning

15 June 2022

Zhiyuan Liu

Maosong Sun

Papers citing "Sparse Structure Search for Parameter-Efficient Tuning"

15 / 15 papers shown

Title
PARA: Parameter-Efficient Fine-tuning with Prompt Aware Representation Adjustment Zequan Liu Yi Zhao Ming Tan Wei Zhu Aaron Xuxiang Tian 39 0 0 03 Feb 2025
Scalable Efficient Training of Large Language Models with Low-dimensional Projected Attention Xingtai Lv Ning Ding Kaiyan Zhang Ermo Hua Ganqu Cui Bowen Zhou 37 1 0 04 Nov 2024
MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning Jingfan Zhang Yi Zhao Dan Chen Xing Tian Huanran Zheng Wei Zhu MoE 37 12 0 23 Oct 2024
PEDRO: Parameter-Efficient Fine-tuning with Prompt DEpenDent Representation MOdification Tianfang Xie Tianjing Li Wei Zhu Wei Han Yi Zhao 34 5 0 26 Sep 2024
IAPT: Instruction-Aware Prompt Tuning for Large Language Models Wei-wei Zhu Aaron Xuxiang Tian Congrui Yin Yuan Ni Xiaoling Wang Guotong Xie 48 0 0 28 May 2024
ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Models Zequan Liu Jiawen Lyn Wei-wei Zhu Xing Tian Yvette Graham OffRL 32 10 0 24 Mar 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 150 310 0 21 Mar 2024
Gradient-based Parameter Selection for Efficient Fine-Tuning Zhi Zhang Qizhe Zhang Zijun Gao Renrui Zhang Ekaterina Shutova Shiji Zhou Shanghang Zhang 28 15 0 15 Dec 2023
Sparse Low-rank Adaptation of Pre-trained Language Models Ning Ding Xingtai Lv Qiaosen Wang Yulin Chen Bowen Zhou Zhiyuan Liu Maosong Sun 22 55 0 20 Nov 2023
Mini but Mighty: Finetuning ViTs with Mini Adapters Imad Eddine Marouf Enzo Tartaglione Stéphane Lathuilière 33 5 0 07 Nov 2023
Improving Prompt Tuning with Learned Prompting Layers Wei Zhu Ming Tan VLM 31 1 0 31 Oct 2023
CLIP-Adapter: Better Vision-Language Models with Feature Adapters Peng Gao Shijie Geng Renrui Zhang Teli Ma Rongyao Fang Yongfeng Zhang Hongsheng Li Yu Qiao VLM CLIP 92 982 0 09 Oct 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 271 5,329 0 05 Nov 2016