v1v2 (latest)

When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations

30 October 2023

Papers citing "When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations"

25 / 25 papers shown

Title
Parameter-Efficient Fine-Tuning of State Space Models Kevin Galim Wonjun Kang Yuchen Zeng H. Koo Kangwook Lee 116 4 0 11 Oct 2024
Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts Minh Le Chau Nguyen Huy Nguyen Quyen Tran Trung Le Nhat Ho 104 8 0 03 Oct 2024
Fine-tuning can cripple your foundation model; preserving features may be the solution Jishnu Mukhoti Y. Gal Philip Torr P. Dokania CLL 116 47 0 25 Aug 2023
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning Yun Luo Zhen Yang Fandong Meng Yafu Li Jie Zhou Yue Zhang CLL KELM 184 318 0 17 Aug 2023
Universal and Transferable Adversarial Attacks on Aligned Language Models Andy Zou Zifan Wang Nicholas Carlini Milad Nasr J. Zico Kolter Matt Fredrikson 295 1,518 0 27 Jul 2023
Black-box Prompt Tuning with Subspace Learning Yuanhang Zheng Zhixing Tan Peng Li Yang Liu VLM 112 11 0 04 May 2023
Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning Vladislav Lialin Vijeta Deshpande Anna Rumshisky 100 177 0 28 Mar 2023
Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning Zhen Wang Yikang Shen Leonid Karlinsky Rogerio Feris Huan Sun Yoon Kim VLM VPVLM 88 115 0 06 Mar 2023
Transformers learn in-context by gradient descent J. Oswald Eyvind Niklasson E. Randazzo João Sacramento A. Mordvintsev A. Zhmoginov Max Vladymyrov MLT 116 496 0 15 Dec 2022
PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao VLM CLL 81 43 0 22 Aug 2022
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey Bonan Min Hayley L Ross Elior Sulem Amir Pouran Ben Veyseh Thien Huu Nguyen Oscar Sainz Eneko Agirre Ilana Heinz Dan Roth LM&MA VLM AI4CE 174 1,085 0 01 Nov 2021
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer Tu Vu Brian Lester Noah Constant Rami Al-Rfou Daniel Cer VLM LRM 202 290 0 15 Oct 2021
Towards a Unified View of Parameter-Efficient Transfer Learning Junxian He Chunting Zhou Xuezhe Ma Taylor Berg-Kirkpatrick Graham Neubig AAML 137 953 0 08 Oct 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 246 3,789 0 03 Sep 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 502 10,526 0 17 Jun 2021
On the Expressive Power of Self-Attention Matrices Valerii Likhosherstov K. Choromanski Adrian Weller 89 36 0 07 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 589 4,093 0 18 Apr 2021
Learning How to Ask: Querying LMs with Mixtures of Soft Prompts Guanghui Qin J. Eisner 63 549 0 14 Apr 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 252 4,305 0 01 Jan 2021
WARP: Word-level Adversarial ReProgramming Karen Hambardzumyan Hrant Khachatrian Jonathan May AAML 334 353 0 01 Jan 2021
DART: Open-Domain Structured Data Record to Text Generation Linyong Nan Dragomir R. Radev Rui Zhang Amrit Rau Abhinand Sivaprasad ... Y. Tan Xi Lin Caiming Xiong R. Socher Nazneen Rajani 60 202 0 06 Jul 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 638 4,921 0 23 Jan 2020
Attention is not not Explanation Sarah Wiegreffe Yuval Pinter XAI AAML FAtt 124 914 0 13 Aug 2019
Attention is not Explanation Sarthak Jain Byron C. Wallace FAtt 148 1,329 0 26 Feb 2019
Learning multiple visual domains with residual adapters Sylvestre-Alvise Rebuffi Hakan Bilen Andrea Vedaldi OOD 176 939 0 22 May 2017