Towards Optimal Adapter Placement for Efficient Transfer Learning

21 October 2024

Papers citing "Towards Optimal Adapter Placement for Efficient Transfer Learning"

34 / 34 papers shown

Title
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 257 398 0 21 Mar 2024
Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters Jiazuo Yu Yunzhi Zhuge Lu Zhang Ping Hu Dong Wang Huchuan Lu You He VLM KELM CLL OODD 180 87 0 18 Mar 2024
Higher Layers Need More LoRA Experts Chongyang Gao Kezhen Chen Jinmeng Rao Baochen Sun Ruibo Liu Daiyi Peng Yawen Zhang Xiaoyuan Guo Jie Yang V. Subrahmanian MoE 50 52 0 13 Feb 2024
Time-, Memory- and Parameter-Efficient Visual Adaptation Otniel-Bogdan Mercea Alexey Gritsenko Cordelia Schmid Anurag Arnab VLM 70 15 0 05 Feb 2024
The Tunnel Effect: Building Data Representations in Deep Neural Networks Wojciech Masarczyk M. Ostaszewski Ehsan Imani Razvan Pascanu Piotr Milo's Tomasz Trzciñski 86 25 0 31 May 2023
READ: Recurrent Adaptation of Large Transformers Sida I. Wang John Nguyen Ke Li Carole-Jean Wu 50 11 0 24 May 2023
Towards Efficient Visual Adaption via Structural Re-parameterization Gen Luo Minglang Huang Yiyi Zhou Xiaoshuai Sun Guannan Jiang Zhiyu Wang Rongrong Ji VLM VPVLM 57 80 0 16 Feb 2023
Convolutional Bypasses Are Better Vision Transformer Adapters Shibo Jie Zhi-Hong Deng VPVLM 82 135 0 14 Jul 2022
ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning Junting Pan Ziyi Lin Xiatian Zhu Jing Shao Hongsheng Li 90 206 0 27 Jun 2022
Rank Diminishing in Deep Neural Networks Ruili Feng Kecheng Zheng Yukun Huang Deli Zhao Michael I. Jordan Zhengjun Zha 69 33 0 13 Jun 2022
Neural Prompt Search Yuanhan Zhang Kaiyang Zhou Ziwei Liu VPVLM VLM 106 151 0 09 Jun 2022
Understanding and Preventing Capacity Loss in Reinforcement Learning Clare Lyle Mark Rowland Will Dabney CLL 97 114 0 20 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 535 6,301 0 05 Apr 2022
Visual Prompt Tuning Menglin Jia Luming Tang Bor-Chun Chen Claire Cardie Serge Belongie Bharath Hariharan Ser-Nam Lim VLM VPVLM 159 1,645 0 23 Mar 2022
UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning Yuning Mao Lambert Mathias Rui Hou Amjad Almahairi Hao Ma Jiawei Han Wen-tau Yih Madian Khabsa 66 193 0 14 Oct 2021
Towards a Unified View of Parameter-Efficient Transfer Learning Junxian He Chunting Zhou Xuezhe Ma Taylor Berg-Kirkpatrick Graham Neubig AAML 137 953 0 08 Oct 2021
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers Rabeeh Karimi Mahabadi James Henderson Sebastian Ruder MoE 119 493 0 08 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 594 4,093 0 18 Apr 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 252 4,305 0 01 Jan 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 684 41,483 0 22 Oct 2020
Deep Ensembles for Low-Data Transfer Learning Basil Mustafa C. Riquelme J. Puigcerver andAndré Susano Pinto Daniel Keysers N. Houlsby FedML OOD 53 24 0 14 Oct 2020
AdapterFusion: Non-Destructive Task Composition for Transfer Learning Jonas Pfeiffer Aishwarya Kamath Andreas Rucklé Kyunghyun Cho Iryna Gurevych CLL MoMe 158 859 0 01 May 2020
MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer Jonas Pfeiffer Ivan Vulić Iryna Gurevych Sebastian Ruder 106 630 0 30 Apr 2020
Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning Zhaojiang Lin Andrea Madotto Pascale Fung 101 162 0 08 Apr 2020
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters Ruize Wang Duyu Tang Nan Duan Zhongyu Wei Xuanjing Huang Jianshu Ji Guihong Cao Daxin Jiang Ming Zhou KELM 129 556 0 05 Feb 2020
The intriguing role of module criticality in the generalization of deep networks Niladri S. Chatterji Behnam Neyshabur Hanie Sedghi 87 52 0 02 Dec 2019
Rigging the Lottery: Making All Tickets Winners Utku Evci Trevor Gale Jacob Menick Pablo Samuel Castro Erich Elsen 199 607 0 25 Nov 2019
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark Xiaohua Zhai J. Puigcerver Alexander Kolesnikov P. Ruyssen C. Riquelme ... Michael Tschannen Marcin Michalski Olivier Bousquet Sylvain Gelly N. Houlsby SSL 97 447 0 01 Oct 2019
Simple, Scalable Adaptation for Neural Machine Translation Ankur Bapna N. Arivazhagan Orhan Firat AI4CE 116 417 0 18 Sep 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 697 24,557 0 26 Jul 2019
SNIP: Single-shot Network Pruning based on Connection Sensitivity Namhoon Lee Thalaiyasingam Ajanthan Philip Torr VLM 271 1,211 0 04 Oct 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,201 0 20 Apr 2018
Learning multiple visual domains with residual adapters Sylvestre-Alvise Rebuffi Hakan Bilen Andrea Vedaldi OOD 179 939 0 22 May 2017
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 524 4,497 0 18 Apr 2017