Towards a Unified View of Parameter-Efficient Transfer Learning

8 October 2021

Junxian He

Chunting Zhou

Xuezhe Ma

Taylor Berg-Kirkpatrick

Graham Neubig

AAML

ArXiv PDF HTML

Papers citing "Towards a Unified View of Parameter-Efficient Transfer Learning"

50 / 633 papers shown

Title
Visual Tuning Bruce X. B. Yu Jianlong Chang Haixin Wang Lin Liu Shijie Wang ... Lingxi Xie Haojie Li Zhouchen Lin Qi Tian Chang Wen Chen VLM 51 38 0 10 May 2023
HiFi: High-Information Attention Heads Hold for Parameter-Efficient Model Adaptation Anchun Gui Han Xiao 21 4 0 08 May 2023
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models Lei Wang Wanyu Xu Yihuai Lan Zhiqiang Hu Yunshi Lan Roy Ka-Wei Lee Ee-Peng Lim ReLM LRM 36 314 0 06 May 2023
Now It Sounds Like You: Learning Personalized Vocabulary On Device Sida Wang Ashish Shenoy P. Chuang John Nguyen VLM 38 2 0 05 May 2023
Personalize Segment Anything Model with One Shot Renrui Zhang Zhengkai Jiang Ziyu Guo Shilin Yan Junting Pan Xianzheng Ma Hao Dong Peng Gao Hongsheng Li MLLM VLM 36 207 0 04 May 2023
Learning Language-Specific Layers for Multilingual Machine Translation Telmo Pires Robin M. Schmidt Yi-Hsiu Liao Stephan Peitz 42 17 0 04 May 2023
Prompt-ICM: A Unified Framework towards Image Coding for Machines with Task-driven Prompts Ruoyu Feng Jinming Liu Xin Jin Xiaohan Pan Heming Sun Zhibo Chen VLM 60 11 0 04 May 2023
Using Language Models on Low-end Hardware Silin Gao Beatriz Borges Saya Kanno Antoine Bosselut 21 0 0 03 May 2023
Parameter-Efficient Cross-lingual Transfer of Vision and Language Models via Translation-based Alignment Zhen Zhang Jialu Wang Qing Guo VLM 31 2 0 02 May 2023
PEFT-Ref: A Modular Reference Architecture and Typology for Parameter-Efficient Finetuning Techniques Mohammed Sabry Anya Belz 38 8 0 24 Apr 2023
UBC-DLNLP at SemEval-2023 Task 12: Impact of Transfer Learning on African Sentiment Analysis Gagan Bhatia Ife Adebara AbdelRahim Elmadany Muhammad Abdul-Mageed 29 1 0 21 Apr 2023
Scaling Transformer to 1M tokens and beyond with RMT Aydar Bulatov Yuri Kuratov Yermek Kapushev Andrey Kravchenko LRM 25 87 0 19 Apr 2023
AdapterGNN: Parameter-Efficient Fine-Tuning Improves Generalization in GNNs Shengrui Li Xueting Han Jing Bai AI4CE 18 10 0 19 Apr 2023
TransHP: Image Classification with Hierarchical Prompting Wenhao Wang Yifan Sun W. Li Yi Yang 24 16 0 13 Apr 2023
Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference Tao Lei Junwen Bai Siddhartha Brahma Joshua Ainslie Kenton Lee ... Vincent Zhao Yuexin Wu Bo-wen Li Yu Zhang Ming-Wei Chang BDL AI4CE 30 55 0 11 Apr 2023
LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models Zhiqiang Hu Lei Wang Yihuai Lan Wanyu Xu Ee-Peng Lim Lidong Bing Xing Xu Soujanya Poria Roy Ka-Wei Lee ALM 48 233 0 04 Apr 2023
A Closer Look at Parameter-Efficient Tuning in Diffusion Models Chendong Xiang Fan Bao Chongxuan Li Hang Su Jun Zhu DiffM 14 16 0 31 Mar 2023
Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentation Julio Silva-Rodríguez Jose Dolz Ismail Ben Ayed 66 13 0 29 Mar 2023
Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning Vladislav Lialin Vijeta Deshpande Anna Rumshisky 45 167 0 28 Mar 2023
Troika: Multi-Path Cross-Modal Traction for Compositional Zero-Shot Learning Siteng Huang Biao Gong Yutong Feng Min Zhang Yiliang Lv Donglin Wang CoGe 35 10 0 27 Mar 2023
Task-oriented Memory-efficient Pruning-Adapter Guorun Wang Jun Yang Yaoru Sun 42 4 0 26 Mar 2023
Contrastive Alignment of Vision to Language Through Parameter-Efficient Transfer Learning Zaid Khan Yun Fu VLM 38 12 0 21 Mar 2023
AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning Qingru Zhang Minshuo Chen Alexander Bukharin Nikos Karampatziakis Pengcheng He Yu Cheng Weizhu Chen Tuo Zhao 23 103 0 18 Mar 2023
A Unified Continual Learning Framework with General Parameter-Efficient Tuning Qiankun Gao Chen Zhao Yifan Sun Teng Xi Gang Zhang Guohao Li Jing Zhang CLL 42 89 0 17 Mar 2023
Dual-path Adaptation from Image to Video Transformers Jungin Park Jiyoung Lee Kwanghoon Sohn ViT 21 37 0 17 Mar 2023
Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning Haoyu He Jianfei Cai Jing Zhang Dacheng Tao Bohan Zhuang VPVLM 22 50 0 15 Mar 2023
Diversity-Aware Meta Visual Prompting Qidong Huang Xiaoyi Dong Dongdong Chen Weiming Zhang Feifei Wang Gang Hua Neng H. Yu VLM VPVLM 41 52 0 14 Mar 2023
Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need Da-Wei Zhou Han-Jia Ye De-Chuan Zhan Ziwei Liu CLL 33 100 0 13 Mar 2023
Multimodal Parameter-Efficient Few-Shot Class Incremental Learning Marco D’Alessandro Alberto Alonso Enrique Calabrés M. Galar CLL 43 13 0 08 Mar 2023
Your representations are in the network: composable and parallel adaptation for large scale models Yonatan Dukler Alessandro Achille Hao Yang Varsha Vivek L. Zancato Benjamin Bowman Avinash Ravichandran Charless C. Fowlkes A. Swaminathan Stefano Soatto 28 3 0 07 Mar 2023
Dynamic Prompting: A Unified Framework for Prompt Tuning Xianjun Yang Wei Cheng Xujiang Zhao Wenchao Yu Linda R. Petzold Haifeng Chen VLM 25 15 0 06 Mar 2023
Effectiveness of Data Augmentation for Parameter Efficient Tuning with Limited Data Stephen Obadinma Hongyu Guo Xiao-Dan Zhu 32 2 0 05 Mar 2023
FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks Xiaoping Han Xiatian Zhu Licheng Yu Li Zhang Yi-Zhe Song Tao Xiang VLM 24 38 0 04 Mar 2023
MixPHM: Redundancy-Aware Parameter-Efficient Tuning for Low-Resource Visual Question Answering Jingjing Jiang Nanning Zheng MoE 40 6 0 02 Mar 2023
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages Yu Zhang Wei Han James Qin Yongqiang Wang Ankur Bapna ... Pedro J. Moreno Chung-Cheng Chiu J. Schalkwyk Franccoise Beaufays Yonghui Wu VLM 79 253 0 02 Mar 2023
Rethinking Efficient Tuning Methods from a Unified Perspective Zeyinzi Jiang Chaojie Mao Ziyuan Huang Yiliang Lv Deli Zhao Jingren Zhou 32 10 0 01 Mar 2023
SMoA: Sparse Mixture of Adapters to Mitigate Multiple Dataset Biases Yanchen Liu Jing Yang Yan Chen Jing Liu Huaqin Wu MoE 47 2 0 28 Feb 2023
Modular Deep Learning Jonas Pfeiffer Sebastian Ruder Ivan Vulić E. Ponti MoMe OOD 32 73 0 22 Feb 2023
Role of Bias Terms in Dot-Product Attention Mahdi Namazifar Devamanyu Hazarika Dilek Z. Hakkani-Tür 26 3 0 16 Feb 2023
Prompt Tuning of Deep Neural Networks for Speaker-adaptive Visual Speech Recognition Minsu Kim Hyungil Kim Y. Ro VLM 13 18 0 16 Feb 2023
Learning Performance-Improving Code Edits Alex Shypula Aman Madaan Yiming Yang Uri Alon Jacob R. Gardner Milad Hashemi Graham Neubig Parthasarathy Ranganathan Osbert Bastani Amir Yazdanbakhsh SyDa 30 81 0 15 Feb 2023
UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling Haoyu Lu Yuqi Huo Guoxing Yang Zhiwu Lu Wei Zhan Masayoshi Tomizuka Mingyu Ding 33 31 0 13 Feb 2023
Less is More: Selective Layer Finetuning with SubTuning Gal Kaplun Andrey Gurevich Tal Swisa Mazor David Shai Shalev-Shwartz Eran Malach 21 7 0 13 Feb 2023
Cross-Modal Fine-Tuning: Align then Refine Junhong Shen Liam Li Lucio Dery Corey Staten M. Khodak Graham Neubig Ameet Talwalkar 33 36 0 11 Feb 2023
UDApter -- Efficient Domain Adaptation Using Adapters Bhavitvya Malik Abhinav Ramesh Kashyap MingSung Kan Soujanya Poria 24 16 0 07 Feb 2023
AIM: Adapting Image Models for Efficient Video Action Recognition Taojiannan Yang Yi Zhu Yusheng Xie Aston Zhang Chong Chen Mu Li ViT 58 144 0 06 Feb 2023
Efficient Domain Adaptation for Speech Foundation Models Bo-wen Li DongSeon Hwang Zhouyuan Huo Junwen Bai Guru Prakash ... K. Sim Yu Zhang Wei Han Trevor Strohman F. Beaufays AI4CE 44 23 0 03 Feb 2023
The Flan Collection: Designing Data and Methods for Effective Instruction Tuning Shayne Longpre Le Hou Tu Vu Albert Webson Hyung Won Chung ... Denny Zhou Quoc V. Le Barret Zoph Jason W. Wei Adam Roberts ALM 41 633 0 31 Jan 2023
AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning Han Zhou Xingchen Wan Ivan Vulić Anna Korhonen 24 45 0 28 Jan 2023
Domain-Agnostic Molecular Generation with Chemical Feedback Yin Fang Ningyu Zhang Zhuo Chen Lingbing Guo Xiaohui Fan Huajun Chen 36 11 0 26 Jan 2023