Compacter: Efficient Low-Rank Hypercomplex Adapter Layers

8 June 2021

Rabeeh Karimi Mahabadi

Papers citing "Compacter: Efficient Low-Rank Hypercomplex Adapter Layers"

50 / 343 papers shown

Title
Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters Jiazuo Yu Yunzhi Zhuge Lu Zhang Ping Hu Dong Wang Huchuan Lu You He VLM KELM CLL OODD 113 70 0 18 Mar 2024
Introducing Routing Functions to Vision-Language Parameter-Efficient Fine-Tuning with Low-Rank Bottlenecks Tingyu Qu Tinne Tuytelaars Marie-Francine Moens MoE 43 2 0 14 Mar 2024
AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning Ruiyi Zhang Rushi Qiang Sai Ashish Somayajula Pengtao Xie 39 13 0 14 Mar 2024
Block-wise LoRA: Revisiting Fine-grained LoRA for Effective Personalization and Stylization in Text-to-Image Generation Likun Li Haoqi Zeng Changpeng Yang Haozhe Jia Di Xu DiffM 34 4 0 12 Mar 2024
Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis Xin Zhou Dingkang Liang Wei Xu Xingkui Zhu Yihan Xu Zhikang Zou Xiang Bai 34 26 0 03 Mar 2024
Large Convolutional Model Tuning via Filter Subspace Wei Chen Zichen Miao Qiang Qiu 51 3 0 01 Mar 2024
DiffuseKronA: A Parameter Efficient Fine-tuning Method for Personalized Diffusion Models Shyam Marjit Harshit Singh Nityanand Mathur Sayak Paul Chia-Mu Yu Pin-Yu Chen DiffM 41 6 0 27 Feb 2024
When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method Biao Zhang Zhongtao Liu Colin Cherry Orhan Firat LRM 57 126 0 27 Feb 2024
Multimodal Instruction Tuning with Conditional Mixture of LoRA Ying Shen Zhiyang Xu Qifan Wang Yu Cheng Wenpeng Yin Lifu Huang 42 13 0 24 Feb 2024
Advancing Parameter Efficiency in Fine-tuning via Representation Editing Muling Wu Wenhao Liu Xiaohua Wang Tianlong Li Changze Lv Zixuan Ling Jianhao Zhu Cenyuan Zhang Xiaoqing Zheng Xuanjing Huang 20 19 0 23 Feb 2024
PEMT: Multi-Task Correlation Guided Mixture-of-Experts Enables Parameter-Efficient Transfer Learning Zhisheng Lin Han Fu Chenghao Liu Zhuo Li Jianling Sun MoE MoMe 30 5 0 23 Feb 2024
SIBO: A Simple Booster for Parameter-Efficient Fine-Tuning Zhihao Wen Jie Zhang Yuan Fang MoE 34 3 0 19 Feb 2024
Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark Yihua Zhang Pingzhi Li Junyuan Hong Jiaxiang Li Yimeng Zhang ... Wotao Yin Mingyi Hong Zhangyang Wang Sijia Liu Tianlong Chen 28 45 0 18 Feb 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 41 48 0 15 Feb 2024
DoRA: Weight-Decomposed Low-Rank Adaptation Shih-yang Liu Chien-Yi Wang Hongxu Yin Pavlo Molchanov Yu-Chiang Frank Wang Kwang-Ting Cheng Min-Hung Chen 29 342 0 14 Feb 2024
Bayesian Multi-Task Transfer Learning for Soft Prompt Tuning Haeju Lee Minchan Jeong SeYoung Yun Kee-Eung Kim AAML VPVLM 53 2 0 13 Feb 2024
Read to Play (R2-Play): Decision Transformer with Multimodal Game Instruction Yonggang Jin Ge Zhang Hao Zhao Tianyu Zheng Jiawei Guo Liuyu Xiang Shawn Yue Stephen W. Huang Zhaofeng He Jie Fu OffRL 31 4 0 06 Feb 2024
Low-rank Attention Side-Tuning for Parameter-Efficient Fine-Tuning Ningyuan Tang Minghao Fu Ke Zhu Jianxin Wu 50 7 0 06 Feb 2024
Time-, Memory- and Parameter-Efficient Visual Adaptation Otniel-Bogdan Mercea Alexey Gritsenko Cordelia Schmid Anurag Arnab VLM 35 13 0 05 Feb 2024
Effective Controllable Bias Mitigation for Classification and Retrieval using Gate Adapters Shahed Masoudian Cornelia Volaucnik Markus Schedl Navid Rekabsaz 16 5 0 29 Jan 2024
HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy Yongkang Liu Yiqun Zhang Qian Li Tong Liu Shi Feng Daling Wang Yifei Zhang Hinrich Schütze 40 6 0 26 Jan 2024
SliceGPT: Compress Large Language Models by Deleting Rows and Columns Saleh Ashkboos Maximilian L. Croci Marcelo Gennari do Nascimento Torsten Hoefler James Hensman VLM 132 145 0 26 Jan 2024
Assessing the Portability of Parameter Matrices Trained by Parameter-Efficient Finetuning Methods Mohammed Sabry Anya Belz 36 0 0 25 Jan 2024
MM-LLMs: Recent Advances in MultiModal Large Language Models Duzhen Zhang Yahan Yu Jiahua Dong Chenxing Li Dan Su Chenhui Chu Dong Yu OffRL LRM 52 179 0 24 Jan 2024
What the Weight?! A Unified Framework for Zero-Shot Knowledge Composition Carolin Holtermann Markus Frohmann Navid Rekabsaz Anne Lauscher MoMe 24 5 0 23 Jan 2024
APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference Bowen Zhao Hannaneh Hajishirzi Qingqing Cao 26 17 0 22 Jan 2024
Harnessing Orthogonality to Train Low-Rank Neural Networks D. Coquelin Katharina Flügel Marie Weiel Nicholas Kiefer Charlotte Debus Achim Streit Markus Goetz 18 1 0 16 Jan 2024
Prompt-driven Latent Domain Generalization for Medical Image Classification Siyuan Yan Chi Liu Zhen Yu Lie Ju Dwarikanath Mahapatra B. Betz‐Stablein Victoria Mar Monika Janda Peter Soyer Zongyuan Ge OOD VLM MedIm 35 7 0 05 Jan 2024
FullLoRA-AT: Efficiently Boosting the Robustness of Pretrained Vision Transformers Zheng Yuan Jie Zhang Shiguang Shan 44 2 0 03 Jan 2024
Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models Terry Yue Zhuo A. Zebaze Nitchakarn Suppattarachai Leandro von Werra H. D. Vries Qian Liu Niklas Muennighoff ALM 38 15 0 01 Jan 2024
Overview of the PromptCBLUE Shared Task in CHIP2023 Wei-wei Zhu Xiaoling Wang Mosha Chen Buzhou Tang LM&MA ELM 36 7 0 29 Dec 2023
Differentially Private Low-Rank Adaptation of Large Language Model Using Federated Learning Xiao-Yang Liu Rongyi Zhu Daochen Zha Jiechao Gao Shan Zhong Matt White Meikang Qiu 23 15 0 29 Dec 2023
PPEA-Depth: Progressive Parameter-Efficient Adaptation for Self-Supervised Monocular Depth Estimation Yue-Jiang Dong Yuanchen Guo Ying-Tian Liu Fang-Lue Zhang Song-Hai Zhang MDE 40 4 0 20 Dec 2023
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment Lingling Xu Haoran Xie S. J. Qin Xiaohui Tao F. Wang 49 135 0 19 Dec 2023
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 26 1 0 18 Dec 2023
When Parameter-efficient Tuning Meets General-purpose Vision-language Models Yihang Zhai Haixin Wang Jianlong Chang Xinlong Yang Jinan Sun Shikun Zhang Qi Tian VLM MLLM 17 1 0 16 Dec 2023
SA $^2$ VP: Spatially Aligned-and-Adapted Visual Prompt Wenjie Pei Tongqi Xia Fanglin Chen Jinsong Li Jiandong Tian Guangming Lu VLM VPVLM 15 8 0 16 Dec 2023
Gradient-based Parameter Selection for Efficient Fine-Tuning Zhi Zhang Qizhe Zhang Zijun Gao Renrui Zhang Ekaterina Shutova Shiji Zhou Shanghang Zhang 28 15 0 15 Dec 2023
VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding Yi Xin Junlong Du Qiang Wang Zhiwen Lin Ke Yan VPVLM 89 49 0 14 Dec 2023
Traffic Signal Control Using Lightweight Transformers: An Offline-to-Online RL Approach Xingshuai Huang Di Wu Benoit Boulet OffRL 19 2 0 12 Dec 2023
GIST: Improving Parameter Efficient Fine Tuning via Knowledge Interaction Jiacheng Ruan Jingsheng Gao Mingye Xie Suncheng Xiang Zefang Yu Ting Liu Yuzhuo Fu MoE 53 4 0 12 Dec 2023
READ-PVLA: Recurrent Adapter with Partial Video-Language Alignment for Parameter-Efficient Transfer Learning in Low-Resource Video-Language Modeling Thong Nguyen Xiaobao Wu Xinshuai Dong Khoi M. Le Zhiyuan Hu Cong-Duy Nguyen See-Kiong Ng Anh Tuan Luu 36 2 0 12 Dec 2023
Batched Low-Rank Adaptation of Foundation Models Yeming Wen Swarat Chaudhuri OffRL 21 19 0 09 Dec 2023
MoSA: Mixture of Sparse Adapters for Visual Efficient Tuning Qizhe Zhang Bocheng Zou Ruichuan An Jiaming Liu Shanghang Zhang MoE 27 2 0 05 Dec 2023
CAST: Cross-Attention in Space and Time for Video Action Recognition Dongho Lee Jongseo Lee Jinwoo Choi EgoV 35 12 0 30 Nov 2023
LLaFS: When Large Language Models Meet Few-Shot Segmentation Lanyun Zhu Tianrun Chen Deyi Ji Jieping Ye Jun Liu VLM 39 37 0 28 Nov 2023
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization Prateek Yadav Leshem Choshen Colin Raffel Mohit Bansal 32 13 0 22 Nov 2023
Sparse Low-rank Adaptation of Pre-trained Language Models Ning Ding Xingtai Lv Qiaosen Wang Yulin Chen Bowen Zhou Zhiyuan Liu Maosong Sun 22 55 0 20 Nov 2023
ADAPTER-RL: Adaptation of Any Agent using Reinforcement Learning Yi-Fan Jin Greg Slabaugh Simon Lucas OnRL AI4CE 10 0 0 20 Nov 2023
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning Clifton A. Poth Hannah Sterz Indraneil Paul Sukannya Purkayastha Leon Arne Engländer Timo Imhof Ivan Vulić Sebastian Ruder Iryna Gurevych Jonas Pfeiffer 32 45 0 18 Nov 2023