Compacter: Efficient Low-Rank Hypercomplex Adapter Layers

8 June 2021

Rabeeh Karimi Mahabadi

Papers citing "Compacter: Efficient Low-Rank Hypercomplex Adapter Layers"

50 / 343 papers shown

Title
Parameter-efficient Dysarthric Speech Recognition Using Adapter Fusion and Householder Transformation Jinzi Qi Hugo Van hamme 38 3 0 12 Jun 2023
ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation Jiaming Liu Senqiao Yang Peidong Jia Renrui Zhang Ming Lu Yandong Guo Wei Xue Shanghang Zhang TTA OOD VLM 30 36 0 07 Jun 2023
Cross-Lingual Transfer with Target Language-Ready Task Adapters Marinela Parović Alan Ansell Ivan Vulić Anna Korhonen 30 8 0 05 Jun 2023
Prompt to be Consistent is Better than Self-Consistent? Few-Shot and Zero-Shot Fact Verification with Pre-trained Language Models Fengzhu Zeng Wei Gao 17 5 0 05 Jun 2023
Exploring the Impact of Model Scaling on Parameter-Efficient Tuning Yusheng Su Chi-Min Chan Jiali Cheng Yujia Qin Yankai Lin ... Ning Ding Xingzhi Sun Guotong Xie Zhiyuan Liu Maosong Sun 21 6 0 04 Jun 2023
Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models Shuo Chen Jindong Gu Zhen Han Yunpu Ma Philip H. S. Torr Volker Tresp VPVLM VLM 32 17 0 03 Jun 2023
Jointly Reparametrized Multi-Layer Adaptation for Efficient and Private Tuning Umang Gupta Aram Galstyan Greg Ver Steeg 6 2 0 30 May 2023
Universality and Limitations of Prompt Tuning Yihan Wang Jatin Chauhan Wei Wang Cho-Jui Hsieh 37 17 0 30 May 2023
Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey Chen Ling Xujiang Zhao Jiaying Lu Chengyuan Deng Can Zheng ... Chris White Quanquan Gu Jian Pei Carl Yang Liang Zhao ALM 27 126 0 30 May 2023
Efficient Storage of Fine-Tuned Models via Low-Rank Approximation of Weight Residuals Simo Ryu S. Seo Jaejun Yoo 29 5 0 28 May 2023
CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers Dachuan Shi Chaofan Tao Anyi Rao Zhendong Yang Chun Yuan Jiaqi Wang VLM 30 22 0 27 May 2023
Fine-tuning Happens in Tiny Subspaces: Exploring Intrinsic Task-specific Subspaces of Pre-trained Language Models Zhong Zhang Bang Liu Junming Shao 28 6 0 27 May 2023
Parameter-Efficient Fine-Tuning without Introducing New Latency Baohao Liao Yan Meng Christof Monz 13 48 0 26 May 2023
CAILA: Concept-Aware Intra-Layer Adapters for Compositional Zero-Shot Learning Zhao-Heng Zheng Haidong Zhu Ramkant Nevatia CoGe 31 7 0 26 May 2023
Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language Models Neal Lawton Anoop Kumar Govind Thattai Aram Galstyan Greg Ver Steeg 17 16 0 26 May 2023
READ: Recurrent Adaptation of Large Transformers Sida I. Wang John Nguyen Ke Li Carole-Jean Wu 20 11 0 24 May 2023
Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model Zirui Liu Guanchu Wang Shaochen Zhong Zhaozhuo Xu Daochen Zha ... Zhimeng Jiang Kaixiong Zhou V. Chaudhary Shuai Xu Xia Hu 41 12 0 24 May 2023
Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning Ximing Lu Faeze Brahman Peter West Jaehun Jang Khyathi Raghavi Chandu ... Bill Yuchen Lin Skyler Hallinan Xiang Ren Sean Welleck Yejin Choi 22 26 0 24 May 2023
Few-shot Unified Question Answering: Tuning Models or Prompts? Srijan Bansal Semih Yavuz Bo Pang Meghana Moorthy Bhat Yingbo Zhou 26 2 0 23 May 2023
QLoRA: Efficient Finetuning of Quantized LLMs Tim Dettmers Artidoro Pagnoni Ari Holtzman Luke Zettlemoyer ALM 41 2,342 0 23 May 2023
VIP5: Towards Multimodal Foundation Models for Recommendation Shijie Geng Juntao Tan Shuchang Liu Zuohui Fu Yongfeng Zhang 29 69 0 23 May 2023
mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations Jonas Pfeiffer Francesco Piccinno Massimo Nicosia Xinyi Wang Machel Reid Sebastian Ruder VLM LRM 34 27 0 23 May 2023
In-Context Probing: Toward Building Robust Classifiers via Probing Large Language Models Afra Amini Massimiliano Ciaramita ReLM 17 1 0 23 May 2023
Condensing Multilingual Knowledge with Lightweight Language-Specific Modules Haoran Xu Weiting Tan Shuyue Stella Li Yunmo Chen Benjamin Van Durme Philipp Koehn Kenton W. Murray 18 6 0 23 May 2023
Small Language Models Improve Giants by Rewriting Their Outputs Giorgos Vernikos Arthur Bravzinskas Jakub Adamek Jonathan Mallinson Aliaksei Severyn Eric Malmi BDL LRM 25 14 0 22 May 2023
Automated Few-shot Classification with Instruction-Finetuned Language Models Rami Aly Xingjian Shi Kaixiang Lin Aston Zhang A. Wilson 38 9 0 21 May 2023
Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adapter Yi Liu Xiaohan Bi Lei Li Sishuo Chen Wenkai Yang Xu Sun FedML 32 12 0 21 May 2023
ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings Shibo Hao Tianyang Liu Zhen Wang Zhiting Hu RALM LLMAG 54 173 0 19 May 2023
Annotation-free Audio-Visual Segmentation Jinxian Liu Yu Wang Chen Ju Chaofan Ma Ya-Qin Zhang Weidi Xie VOS VLM 36 28 0 18 May 2023
G-Adapter: Towards Structure-Aware Parameter-Efficient Transfer Learning for Graph Transformer Networks Anchun Gui Jinqiang Ye Han Xiao 27 19 0 17 May 2023
Parameter-efficient Tuning of Large-scale Multimodal Foundation Model Haixin Wang Xinlong Yang Jianlong Chang Di Jin Jinan Sun Shikun Zhang Xiao Luo Qi Tian 25 23 0 15 May 2023
Parameter-Efficient Fine-Tuning with Layer Pruning on Free-Text Sequence-to-Sequence Modeling Y. Zhu Xuebing Yang Yuanyuan Wu Wensheng Zhang MedIm 20 2 0 15 May 2023
A Comprehensive Analysis of Adapter Efficiency Nandini Mundra Sumanth Doddapaneni Raj Dabre Anoop Kunchukuttan Ratish Puduppully Mitesh M. Khapra 20 10 0 12 May 2023
Visual Tuning Bruce X. B. Yu Jianlong Chang Haixin Wang Lin Liu Shijie Wang ... Lingxi Xie Haojie Li Zhouchen Lin Qi Tian Chang Wen Chen VLM 46 38 0 10 May 2023
HiFi: High-Information Attention Heads Hold for Parameter-Efficient Model Adaptation Anchun Gui Han Xiao 19 4 0 08 May 2023
Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterization Anastasia Razdaibiedina Yuning Mao Rui Hou Madian Khabsa M. Lewis Jimmy Ba Amjad Almahairi VLM 24 42 0 06 May 2023
Now It Sounds Like You: Learning Personalized Vocabulary On Device Sida Wang Ashish Shenoy P. Chuang John Nguyen VLM 35 2 0 05 May 2023
Learngene: Inheriting Condensed Knowledge from the Ancestry Model to Descendant Models Qiufeng Wang Xu Yang Shuxia Lin Jing Wang Xin Geng 23 10 0 03 May 2023
Consolidator: Mergeable Adapter with Grouped Connections for Visual Adaptation Tianxiang Hao Hui Chen Yuchen Guo Guiguang Ding 42 16 0 30 Apr 2023
LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model Peng Gao Jiaming Han Renrui Zhang Ziyi Lin Shijie Geng ... Pan Lu Conghui He Xiangyu Yue Hongsheng Li Yu Qiao MLLM 33 550 0 28 Apr 2023
PEFT-Ref: A Modular Reference Architecture and Typology for Parameter-Efficient Finetuning Techniques Mohammed Sabry Anya Belz 38 8 0 24 Apr 2023
LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models Zhiqiang Hu Lei Wang Yihuai Lan Wanyu Xu Ee-Peng Lim Lidong Bing Xing Xu Soujanya Poria Roy Ka-Wei Lee ALM 39 233 0 04 Apr 2023
EPVT: Environment-aware Prompt Vision Transformer for Domain Generalization in Skin Lesion Recognition Siyuan Yan Chih-Chen Liu Zhen Yu Lie Ju Dwarikanath Mahapatrainst Victoria Mar Monika Janda Peter Soyer Z. Ge ViT MedIm VLM 41 9 0 04 Apr 2023
Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentation Julio Silva-Rodríguez Jose Dolz Ismail Ben Ayed 66 13 0 29 Mar 2023
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention Renrui Zhang Jiaming Han Chris Liu Peng Gao Aojun Zhou Xiangfei Hu Shilin Yan Pan Lu Hongsheng Li Yu Qiao MLLM 41 743 0 28 Mar 2023
Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning Vladislav Lialin Vijeta Deshpande Anna Rumshisky 33 167 0 28 Mar 2023
Task-oriented Memory-efficient Pruning-Adapter Guorun Wang Jun Yang Yaoru Sun 34 4 0 26 Mar 2023
Contrastive Alignment of Vision to Language Through Parameter-Efficient Transfer Learning Zaid Khan Yun Fu VLM 35 12 0 21 Mar 2023
eP-ALM: Efficient Perceptual Augmentation of Language Models Mustafa Shukor Corentin Dancette Matthieu Cord MLLM VLM 32 29 0 20 Mar 2023
A Unified Continual Learning Framework with General Parameter-Efficient Tuning Qiankun Gao Chen Zhao Yifan Sun Teng Xi Gang Zhang Guohao Li Jing Zhang CLL 34 87 0 17 Mar 2023