Compacter: Efficient Low-Rank Hypercomplex Adapter Layers

8 June 2021

Rabeeh Karimi Mahabadi

Papers citing "Compacter: Efficient Low-Rank Hypercomplex Adapter Layers"

50 / 343 papers shown

Title
Decorrelating Structure via Adapters Makes Ensemble Learning Practical for Semi-supervised Learning Jiaqi Wu Junbiao Pang Qingming Huang 31 0 0 08 Aug 2024
UNLEARN Efficient Removal of Knowledge in Large Language Models Tyler Lizzo Larry Heck KELM MoMe MU 43 1 0 08 Aug 2024
A Survey of Mamba Shuwei Shi Shibing Chu Rui An Wenqi Fan Yuee Xie Hui Liu Yuanping Chen Qing Li AI4CE 42 26 0 02 Aug 2024
Enhancing Parameter Efficiency and Generalization in Large-Scale Models: A Regularized and Masked Low-Rank Adaptation Approach Yuzhu Mao Siqi Ping Zihao Zhao Yang Liu Wenbo Ding 37 1 0 16 Jul 2024
Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software Dezhi Ran Mengzhou Wu Wei Yang Tao Xie AI4CE 39 1 0 11 Jul 2024
SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning Haiwen Diao Bo Wan Xu Jia Yunzhi Zhuge Ying Zhang Huchuan Lu Long Chen VLM 50 4 0 10 Jul 2024
Controllable Navigation Instruction Generation with Chain of Thought Prompting Xianghao Kong Jinyu Chen Wenguan Wang Hang Su Xiaolin Hu Yi Yang Si Liu LRM 45 4 0 10 Jul 2024
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences Nikolaos Dimitriadis Pascal Frossard F. Fleuret MoE 67 6 0 10 Jul 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 54 24 0 08 Jul 2024
See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition Chongjie Si Xiaokang Yang Wei Shen 42 5 0 07 Jul 2024
Revealing the Utilized Rank of Subspaces of Learning in Neural Networks Isha Garg Christian Koguchi Eshan Verma Daniel Ulbricht 32 0 0 05 Jul 2024
Lateralization LoRA: Interleaved Instruction Tuning with Modality-Specialized Adaptations Zhiyang Xu Minqian Liu Ying Shen Joy Rimchala Jiaxin Zhang Qifan Wang Yu Cheng Lifu Huang VLM 39 2 0 04 Jul 2024
SAVE: Segment Audio-Visual Easy way using Segment Anything Model Khanh-Binh Nguyen Chae Jung Park VLM VOS 42 1 0 02 Jul 2024
HyperLoader: Integrating Hypernetwork-Based LoRA and Adapter Layers into Multi-Task Transformers for Sequence Labelling Jesús-Germán Ortiz-Barajas Helena Gómez-Adorno Thamar Solorio 32 1 0 01 Jul 2024
Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning Arijit Sehanobish Avinava Dubey Krzysztof Choromanski Somnath Basu Roy Chowdhury Deepali Jain Vikas Sindhwani Snigdha Chaturvedi ALM 43 1 0 25 Jun 2024
MoESD: Mixture of Experts Stable Diffusion to Mitigate Gender Bias Guorun Wang Lucia Specia DiffM MoE 48 0 0 25 Jun 2024
Group and Shuffle: Efficient Structured Orthogonal Parametrization Mikhail Gorbunov Nikolay Yudin Vera Soboleva Aibek Alanov Alexey Naumov Maxim Rakhuba 42 1 0 14 Jun 2024
Talking Heads: Understanding Inter-layer Communication in Transformer Language Models Jack Merullo Carsten Eickhoff Ellie Pavlick 58 13 0 13 Jun 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang Xiaojie Li Zhongzhu Zhou Shuaiwen Leon Song Jianlong Wu Liqiang Nie Guohao Li 45 6 0 07 Jun 2024
Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Vision Minglei Li Peng Ye Yongqi Huang Lin Zhang Tao Chen Tong He Jiayuan Fan Wanli Ouyang MoE 34 4 0 05 Jun 2024
RAP: Efficient Text-Video Retrieval with Sparse-and-Correlated Adapter Meng Cao Haoran Tang Jinfa Huang Peng Jin Can Zhang Ruyang Liu Long Chen Xiaodan Liang Li-ming Yuan Ge Li 101 11 0 29 May 2024
Low-Rank Few-Shot Adaptation of Vision-Language Models Maxime Zanella Ismail Ben Ayed OffRL VLM 56 26 0 28 May 2024
IAPT: Instruction-Aware Prompt Tuning for Large Language Models Wei-wei Zhu Aaron Xuxiang Tian Congrui Yin Yuan Ni Xiaoling Wang Guotong Xie 48 0 0 28 May 2024
DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distribution Yulong Mao Kaiyu Huang Changhao Guan Ganglin Bao Fengran Mo Jinan Xu 29 10 0 27 May 2024
PLUG: Revisiting Amodal Segmentation with Foundation Model and Hierarchical Focus Zhaochen Liu Limeng Qiao Xiangxiang Chu Tingting Jiang 37 2 0 25 May 2024
Prompt Tuning Strikes Back: Customizing Foundation Models with Low-Rank Prompt Adaptation Abhinav C. P. Jain Swarat Chaudhuri Thomas W. Reps Christopher M. Jermaine 20 1 0 24 May 2024
VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks Yang Li Shaobo Han Shihao Ji MoE 27 11 0 24 May 2024
Maintaining Structural Integrity in Parameter Spaces for Parameter Efficient Fine-tuning Chongjie Si Xuehui Wang Xue Yang Zhengqin Xu Qingyun Li Jifeng Dai Yu Qiao Xiaokang Yang Wei Shen 31 8 0 23 May 2024
A Survey on Transformers in NLP with Focus on Efficiency Wazib Ansar Saptarsi Goswami Amlan Chakrabarti MedIm 40 2 0 15 May 2024
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation Jie Xu Karthikeyan P. Saravanan Rogier van Dalen Haaris Mehmood David Tuckey Mete Ozay 56 5 0 10 May 2024
Memory-Space Visual Prompting for Efficient Vision-Language Fine-Tuning Shibo Jie Yehui Tang Ning Ding Zhi-Hong Deng Kai Han Yunhe Wang VLM 33 6 0 09 May 2024
FlexEControl: Flexible and Efficient Multimodal Control for Text-to-Image Generation Xuehai He Jian Zheng Jacob Zhiyuan Fang Robinson Piramuthu Mohit Bansal Vicente Ordonez Gunnar A. Sigurdsson Nanyun Peng Xin Eric Wang DiffM 53 1 0 08 May 2024
Group-aware Parameter-efficient Updating for Content-Adaptive Neural Video Compression Zhenghao Chen Luping Zhou Zhihao Hu Dong Xu 28 9 0 07 May 2024
Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning Jing Xu Jingzhao Zhang 39 7 0 04 May 2024
FL-TAC: Enhanced Fine-Tuning in Federated Learning via Low-Rank, Task-Specific Adapter Clustering Siqi Ping Yuzhu Mao Yang Liu Xiao-Ping Zhang Wenbo Ding FedML 26 3 0 23 Apr 2024
AdaIR: Exploiting Underlying Similarities of Image Restoration Tasks with Adapters Hao-Wei Chen Yu-Syuan Xu Kelvin C. K. Chan Hsien-Kai Kuo Chun-Yi Lee Ming-Hsuan Yang 31 1 0 17 Apr 2024
Low-rank Adaptation for Spatio-Temporal Forecasting Weilin Ruan Wei Chen Xilin Dang Jianxiang Zhou Weichuang Li Xu Liu Yuxuan Liang AI4TS 25 2 0 11 Apr 2024
Automated Federated Pipeline for Parameter-Efficient Fine-Tuning of Large Language Models Zihan Fang Zheng Lin Zhe Chen Xianhao Chen Yue Gao Yuguang Fang 54 35 0 09 Apr 2024
PORTULAN ExtraGLUE Datasets and Models: Kick-starting a Benchmark for the Neural Processing of Portuguese T. Osório Bernardo Leite Henrique Lopes Cardoso Luís Gomes João Rodrigues Rodrigo Santos António Branco 27 3 0 08 Apr 2024
DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model Chao Gao Sai Qian Zhang ALM 120 7 0 08 Apr 2024
Facial Affective Behavior Analysis with Instruction Tuning Yifan Li Anh Dao Wentao Bao Zhen Tan Tianlong Chen Huan Liu Yu Kong CVBM 60 15 0 07 Apr 2024
Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translation Tong Su Xin Peng Sarubi Thillainathan David Guzmán Surangika Ranathunga En-Shiun Annie Lee 35 2 0 05 Apr 2024
IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled PEFT Junchen Fu Xuri Ge Xin Xin Alexandros Karatzoglou Ioannis Arapakis Jie Wang Joemon M. Jose 42 17 0 02 Apr 2024
InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning Yan-Shuo Liang Wu-Jun Li CLL 40 39 0 30 Mar 2024
MTLoRA: A Low-Rank Adaptation Approach for Efficient Multi-Task Learning Ahmed A. Agiza Marina Neseem Sherief Reda MoE 48 12 0 29 Mar 2024
ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Models Zequan Liu Jiawen Lyn Wei-wei Zhu Xing Tian Yvette Graham OffRL 34 10 0 24 Mar 2024
Not All Attention is Needed: Parameter and Computation Efficient Transfer Learning for Multi-modal Large Language Models Qiong Wu Weihao Ye Yiyi Zhou Xiaoshuai Sun Rongrong Ji MoE 49 1 0 22 Mar 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 150 310 0 21 Mar 2024
BiLoRA: A Bi-level Optimization Framework for Overfitting-Resilient Low-Rank Adaptation of Large Pre-trained Models Rushi Qiang Ruiyi Zhang Pengtao Xie AI4CE 28 8 0 19 Mar 2024
SuperLoRA: Parameter-Efficient Unified Adaptation of Multi-Layer Attention Modules Xiangyu Chen Jing Liu Ye Wang Pu Wang Matthew Brand Guanghui Wang T. Koike-Akino 48 7 0 18 Mar 2024