Compacter: Efficient Low-Rank Hypercomplex Adapter Layers

8 June 2021

Rabeeh Karimi Mahabadi

Papers citing "Compacter: Efficient Low-Rank Hypercomplex Adapter Layers"

50 / 343 papers shown

Title
Efficient Computation Sharing for Multi-Task Visual Scene Understanding Sara Shoouri Mingyu Yang Zichen Fan Hun-Seok Kim MoE 26 3 0 16 Mar 2023
Dynamic Prompting: A Unified Framework for Prompt Tuning Xianjun Yang Wei Cheng Xujiang Zhao Wenchao Yu Linda R. Petzold Haifeng Chen VLM 25 14 0 06 Mar 2023
Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning Zhen Wang Rameswar Panda Leonid Karlinsky Rogerio Feris Huan Sun Yoon Kim VLM VPVLM 33 107 0 06 Mar 2023
Choosing Public Datasets for Private Machine Learning via Gradient Subspace Distance Xin Gu Gautam Kamath Zhiwei Steven Wu 28 12 0 02 Mar 2023
MixPHM: Redundancy-Aware Parameter-Efficient Tuning for Low-Resource Visual Question Answering Jingjing Jiang Nanning Zheng MoE 40 6 0 02 Mar 2023
Rethinking Efficient Tuning Methods from a Unified Perspective Zeyinzi Jiang Chaojie Mao Ziyuan Huang Yiliang Lv Deli Zhao Jingren Zhou 30 10 0 01 Mar 2023
Modular Deep Learning Jonas Pfeiffer Sebastian Ruder Ivan Vulić E. Ponti MoMe OOD 32 73 0 22 Feb 2023
Parameter-efficient Modularised Bias Mitigation via AdapterFusion Deepak Kumar Oleg Lesota George Zerveas Daniel Cohen Carsten Eickhoff Markus Schedl Navid Rekabsaz MoMe KELM 18 24 0 13 Feb 2023
On the Efficacy of Differentially Private Few-shot Image Classification Marlon Tobaben Aliaksandra Shysheya J. Bronskill Andrew J. Paverd Shruti Tople Santiago Zanella Béguelin Richard Turner Antti Honkela 38 11 0 02 Feb 2023
An Empirical Study on the Transferability of Transformer Modules in Parameter-Efficient Fine-Tuning Mohammad AkbarTajari S. Rajaee Mohammad Taher Pilehvar 11 2 0 01 Feb 2023
Progressive Prompts: Continual Learning for Language Models Anastasia Razdaibiedina Yuning Mao Rui Hou Madian Khabsa M. Lewis Amjad Almahairi VLM KELM CLL 27 127 0 29 Jan 2023
AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning Han Zhou Xingchen Wan Ivan Vulić Anna Korhonen 21 45 0 28 Jan 2023
Differentially Private Natural Language Models: Recent Advances and Future Directions Lijie Hu Ivan Habernal Lei Shen Di Wang AAML 30 18 0 22 Jan 2023
Parameter-Efficient Fine-Tuning Design Spaces Jiaao Chen Aston Zhang Xingjian Shi Mu Li Alexander J. Smola Diyi Yang 42 59 0 04 Jan 2023
SPT: Semi-Parametric Prompt Tuning for Multitask Prompted Learning M Saiful Bari Aston Zhang Shuai Zheng Xingjian Shi Yi Zhu Shafiq R. Joty Mu Li RALM VLM VPVLM LRM 45 5 0 21 Dec 2022
KronA: Parameter Efficient Tuning with Kronecker Adapter Ali Edalati Marzieh S. Tahaei I. Kobyzev V. Nia J. Clark Mehdi Rezagholizadeh 45 92 0 20 Dec 2022
Parameter-efficient Zero-shot Transfer for Cross-Language Dense Retrieval with Adapters Eugene Yang Suraj Nair Dawn J Lawrie J. Mayfield Douglas W. Oard 27 4 0 20 Dec 2022
Vision Transformers are Parameter-Efficient Audio-Visual Learners Yan-Bo Lin Yi-Lin Sung Jie Lei Joey Tianyi Zhou Gedas Bertasius 28 73 0 15 Dec 2022
Searching for Effective Multilingual Fine-Tuning Methods: A Case Study in Summarization Yiwei Qin Graham Neubig Pengfei Liu 21 3 0 12 Dec 2022
FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer Shibo Jie Zhi-Hong Deng 23 126 0 06 Dec 2022
Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning Shih-Cheng Huang Shi Wang Min-Han Shih Saurav Sahay Hung-yi Lee 36 0 0 02 Dec 2022
Learning Label Modular Prompts for Text Classification in the Wild Hailin Chen Amrita Saha Shafiq R. Joty Steven C. H. Hoi OOD VLM 18 5 0 30 Nov 2022
On the Effectiveness of Parameter-Efficient Fine-Tuning Z. Fu Haoran Yang Anthony Man-Cho So Wai Lam Lidong Bing Nigel Collier 24 156 0 28 Nov 2022
Towards continually learning new languages Ngoc-Quan Pham J. Niehues A. Waibel CLL 11 1 0 21 Nov 2022
Cross-Modal Adapter for Text-Video Retrieval Haojun Jiang Jianke Zhang Rui Huang Chunjiang Ge Zanlin Ni Jiwen Lu Jie Zhou S. Song Gao Huang 48 36 0 17 Nov 2022
Structured Pruning Adapters Lukas Hedegaard Aman Alok Juby Jose Alexandros Iosifidis 38 10 0 17 Nov 2022
QAmeleon: Multilingual QA with Only 5 Examples Priyanka Agrawal Chris Alberti Fantine Huot Joshua Maynez Ji Ma Sebastian Ruder Kuzman Ganchev Dipanjan Das Mirella Lapata 13 28 0 15 Nov 2022
One-Time Model Adaptation to Heterogeneous Clients: An Intra-Client and Inter-Image Attention Design Yikai Yan Chaoyue Niu Fan Wu Qinya Li Shaojie Tang Chengfei Lyu Guihai Chen 29 0 0 11 Nov 2022
Motion Style Transfer: Modular Low-Rank Adaptation for Deep Motion Forecasting Parth Kothari Danyang Li Yuejiang Liu Alexandre Alahi TTA AI4TS 42 15 0 06 Nov 2022
On the Domain Adaptation and Generalization of Pretrained Language Models: A Survey Xu Guo Han Yu LM&MA VLM 28 29 0 06 Nov 2022
Resource-Efficient Transfer Learning From Speech Foundation Model Using Hierarchical Feature Fusion Zhouyuan Huo K. Sim Bo-wen Li DongSeon Hwang Tara N. Sainath Trevor Strohman 22 5 0 04 Nov 2022
Universal Deep Image Compression via Content-Adaptive Optimization with Adapters Koki Tsubota Hiroaki Akutsu Kiyoharu Aizawa 29 17 0 02 Nov 2022
Inducer-tuning: Connecting Prefix-tuning and Adapter-tuning Yifan Chen Devamanyu Hazarika Mahdi Namazifar Yang Liu Di Jin Dilek Z. Hakkani-Tür 21 3 0 26 Oct 2022
Evaluating Parameter Efficient Learning for Generation Peng-Tao Xu M. Patwary Shrimai Prabhumoye Virginia Adams R. Prenger Ming-Yu Liu Nayeon Lee M. Shoeybi Bryan Catanzaro MoE 33 3 0 25 Oct 2022
Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts Xiangyang Liu Tianxiang Sun Xuanjing Huang Xipeng Qiu VLM 36 27 0 20 Oct 2022
Tiny-Attention Adapter: Contexts Are More Important Than the Number of Parameters Hongyu Zhao Hao Tan Hongyuan Mei MoE 33 16 0 18 Oct 2022
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning Dongze Lian Daquan Zhou Jiashi Feng Xinchao Wang 36 247 0 17 Oct 2022
DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation Mojtaba Valipour Mehdi Rezagholizadeh I. Kobyzev A. Ghodsi 29 159 0 14 Oct 2022
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models S. Kwon Jeonghoon Kim Jeongin Bae Kang Min Yoo Jin-Hwa Kim Baeseong Park Byeongwook Kim Jung-Woo Ha Nako Sung Dongsoo Lee MQ 29 30 0 08 Oct 2022
Polyhistor: Parameter-Efficient Multi-Task Adaptation for Dense Vision Tasks Yen-Cheng Liu Chih-Yao Ma Junjiao Tian Zijian He Z. Kira 126 47 0 07 Oct 2022
Unsupervised Neural Stylistic Text Generation using Transfer learning and Adapters Vinayshekhar Bannihatti Kumar Rashmi Gangadharaiah Dan Roth 30 1 0 07 Oct 2022
Improving the Sample Efficiency of Prompt Tuning with Domain Adaptation Xu Guo Boyang Albert Li Han Yu VLM 39 22 0 06 Oct 2022
Towards a Unified View on Visual Parameter-Efficient Transfer Learning Bruce X. B. Yu Jianlong Chang Lin Liu Qi Tian Changan Chen VPVLM VLM 68 34 0 03 Oct 2022
Differentially Private Optimization on Large Model at Small Cost Zhiqi Bu Yu-Xiang Wang Sheng Zha George Karypis 38 52 0 30 Sep 2022
Differentially Private Bias-Term Fine-tuning of Foundation Models Zhiqi Bu Yu-Xiang Wang Sheng Zha George Karypis 28 46 0 30 Sep 2022
Towards Parameter-Efficient Integration of Pre-Trained Language Models In Temporal Video Grounding Erica K. Shimomoto Edison Marrese-Taylor Hiroya Takamura Ichiro Kobayashi Hideki Nakayama Yusuke Miyao 27 7 0 26 Sep 2022
EffEval: A Comprehensive Evaluation of Efficiency for MT Evaluation Metrics Daniil Larionov Jens Grunwald Christoph Leiter Steffen Eger 22 5 0 20 Sep 2022
StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuation A. Maharana Darryl Hannan Joey Tianyi Zhou DiffM 31 77 0 13 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 30 109 0 31 Aug 2022
Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation Zejiang Hou Julian Salazar George Polovets 30 14 0 07 Jul 2022