Compacter: Efficient Low-Rank Hypercomplex Adapter Layers

8 June 2021

Rabeeh Karimi Mahabadi

Papers citing "Compacter: Efficient Low-Rank Hypercomplex Adapter Layers"

50 / 343 papers shown

Title
Schema Graph-Guided Prompt for Multi-Domain Dialogue State Tracking Ruolin Su Ting-Wei Wu B. Juang 21 1 0 10 Nov 2023
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization Weiyang Liu Zeju Qiu Yao Feng Yuliang Xiu Yuxuan Xue ... Songyou Peng Yandong Wen Michael J. Black Adrian Weller Bernhard Schölkopf 50 57 0 10 Nov 2023
Mini but Mighty: Finetuning ViTs with Mini Adapters Imad Eddine Marouf Enzo Tartaglione Stéphane Lathuilière 33 5 0 07 Nov 2023
Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuning Sarkar Snigdha Sarathi Das Ranran Haoran Zhang Peng Shi Wenpeng Yin Rui Zhang 115 14 0 07 Nov 2023
Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE Zeren Chen Ziqin Wang Zhen Wang Huayang Liu Zhen-fei Yin Si Liu Lu Sheng Wanli Ouyang Yu Qiao Jing Shao MoE 36 7 0 05 Nov 2023
Improving Prompt Tuning with Learned Prompting Layers Wei Zhu Ming Tan VLM 31 1 0 31 Oct 2023
Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone Zeyinzi Jiang Chaojie Mao Ziyuan Huang Ao Ma Yiliang Lv Yujun Shen Deli Zhao Jingren Zhou 30 15 0 30 Oct 2023
Improving Language Models Meaning Understanding and Consistency by Learning Conceptual Roles from Dictionary Myeongjun Jang Thomas Lukasiewicz 27 4 0 24 Oct 2023
Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization Tianshi Che Ji Liu Yang Zhou Jiaxiang Ren Jiwen Zhou Victor S. Sheng H. Dai Dejing Dou 25 50 0 23 Oct 2023
Prototype-based HyperAdapter for Sample-Efficient Multi-task Tuning Hao Zhao Jie Fu Zhaofeng He 105 6 0 18 Oct 2023
Domain Generalization Using Large Pretrained Models with Mixture-of-Adapters Gyuseong Lee Wooseok Jang Jin Hyeon Kim Jaewoo Jung Seungryong Kim MoE OOD 30 2 0 17 Oct 2023
Matrix Compression via Randomized Low Rank and Low Precision Factorization R. Saha Varun Srivastava Mert Pilanci 26 19 0 17 Oct 2023
Decomposed Prompt Tuning via Low-Rank Reparameterization Yao Xiao Lu Xu Jiaxi Li Wei Lu Xiaoli Li VLM 22 6 0 16 Oct 2023
NOLA: Compressing LoRA using Linear Combination of Random Basis Soroush Abbasi Koohpayegani K. Navaneet Parsa Nooralinejad Soheil Kolouri Hamed Pirsiavash 40 12 0 04 Oct 2023
Empirical Study of PEFT techniques for Winter Wheat Segmentation Mohamad Hasan Zahweh Hasan Nasrallah Mustafa Shukor G. Faour A. Ghandour 19 3 0 03 Oct 2023
ScaLearn: Simple and Highly Parameter-Efficient Task Transfer by Learning to Scale Markus Frohmann Carolin Holtermann Shahed Masoudian Anne Lauscher Navid Rekabsaz 34 2 0 02 Oct 2023
BLIP-Adapter: Parameter-Efficient Transfer Learning for Mobile Screenshot Captioning Ching-Yu Chiang I-Hua Chang Shih-Wei Liao 44 1 0 26 Sep 2023
Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference Parsa Kavehzadeh Mojtaba Valipour Marzieh S. Tahaei Ali Ghodsi Boxing Chen Mehdi Rezagholizadeh 35 6 0 16 Sep 2023
Hydra: Multi-head Low-rank Adaptation for Parameter Efficient Fine-tuning Sanghyeon Kim Hyunmo Yang Younghyun Kim Youngjoon Hong Eunbyung Park AI4CE 24 16 0 13 Sep 2023
Scaled Prompt-Tuning for Few-Shot Natural Language Generation Ting Hu Christoph Meinel Haojin Yang 11 0 0 13 Sep 2023
DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Pass Minxin Du Xiang Yue Sherman S. M. Chow Tianhao Wang Chenyu Huang Huan Sun SILM 32 58 0 13 Sep 2023
Dynamic Visual Prompt Tuning for Parameter Efficient Transfer Learning Chunqing Ruan Hongjian Wang VLM VPVLM 32 1 0 12 Sep 2023
DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning Zhengxiang Shi Aldo Lipani VLM 28 30 0 11 Sep 2023
ResFields: Residual Neural Fields for Spatiotemporal Signals Marko Mihajlovic Sergey Prokudin Marc Pollefeys Siyu Tang 32 14 0 06 Sep 2023
Parameter and Computation Efficient Transfer Learning for Vision-Language Pre-trained Models Qiong Wu Wei Yu Yiyi Zhou Shubin Huang Xiaoshuai Sun Rongrong Ji VLM 26 7 0 04 Sep 2023
FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning Weirui Kuang Bingchen Qian Zitao Li Daoyuan Chen Dawei Gao Xuchen Pan Yuexiang Xie Yaliang Li Bolin Ding Jingren Zhou FedML 21 112 0 01 Sep 2023
Fine-Tuning Llama 2 Large Language Models for Detecting Online Sexual Predatory Chats and Abusive Texts Thanh Thi Nguyen Campbell Wilson Janis Dalins 19 23 0 28 Aug 2023
UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memory Haiwen Diao Bo Wan Yuhang Zhang Xuecong Jia Huchuan Lu Long Chen VLM 33 18 0 28 Aug 2023
Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrieval Yuan. Yuan Yangfan Zhan Zhitong Xiong VLM 30 39 0 24 Aug 2023
IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning Feiyu F. Zhang Liangzhi Li Jun-Cheng Chen Zhouqian Jiang Bowen Wang Yiming Qian 51 32 0 23 Aug 2023
VLN-PETL: Parameter-Efficient Transfer Learning for Vision-and-Language Navigation Yanyuan Qiao Zheng Yu Qi Wu VLM 22 16 0 20 Aug 2023
VL-PET: Vision-and-Language Parameter-Efficient Tuning via Granularity Control Zi-Yuan Hu Yanyang Li M. Lyu Liwei Wang VLM 27 15 0 18 Aug 2023
Towards Instance-adaptive Inference for Federated Learning Chunhui Feng Kai Yu Nian Liu Xinxing Xu Salman Khan W. Zuo FedML 36 11 0 11 Aug 2023
Revisiting the Parameter Efficiency of Adapters from the Perspective of Precision Redundancy Shibo Jie Haoqing Wang Zhiwei Deng 19 31 0 31 Jul 2023
Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentation Zunnan Xu Zhihong Chen Yong Zhang Yibing Song Xiang Wan Guanbin Li VLM 35 47 0 21 Jul 2023
Tuning Pre-trained Model via Moment Probing Mingze Gao Qilong Wang Zhenyi Lin Pengfei Zhu Qinghua Hu Jingbo Zhou 19 6 0 21 Jul 2023
FinGPT: Democratizing Internet-scale Data for Financial Large Language Models Xiao-Yang Liu Guoxuan Wang Hongyang Yang Daochen Zha AIFin 38 43 0 19 Jul 2023
ReLoRA: High-Rank Training Through Low-Rank Updates Vladislav Lialin Namrata Shivagunde Sherin Muckatira Anna Rumshisky BDL 29 94 0 11 Jul 2023
Scaling In-Context Demonstrations with Structured Attention Tianle Cai Kaixuan Huang Jason D. Lee Mengdi Wang LRM 31 8 0 05 Jul 2023
OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models Shengding Hu Ning Ding Weilin Zhao Xingtai Lv Zhen Zhang Zhiyuan Liu Maosong Sun 34 7 0 05 Jul 2023
A ChatGPT Aided Explainable Framework for Zero-Shot Medical Image Diagnosis Jiaxiang Liu Tianxiang Hu Yan Zhang Xiaotang Gai Yang Feng Zuozhu Liu LM&MA MedIm 34 32 0 05 Jul 2023
Stitched ViTs are Flexible Vision Backbones Zizheng Pan Jing Liu Haoyu He Jianfei Cai Bohan Zhuang 20 2 0 30 Jun 2023
Approximated Prompt Tuning for Vision-Language Pre-trained Models Qiong Wu Shubin Huang Yiyi Zhou Pingyang Dai Annan Shu Guannan Jiang Rongrong Ji VLM VPVLM 25 2 0 27 Jun 2023
Learning to Modulate pre-trained Models in RL Thomas Schmied M. Hofmarcher Fabian Paischer Razvan Pascanu Sepp Hochreiter CLL OffRL 24 14 0 26 Jun 2023
Composing Parameter-Efficient Modules with Arithmetic Operations Jinghan Zhang Shiqi Chen Junteng Liu Junxian He KELM MoMe 26 109 0 26 Jun 2023
Efficient Online Processing with Deep Neural Networks Lukas Hedegaard 23 0 0 23 Jun 2023
Harnessing the Power of Adversarial Prompting and Large Language Models for Robust Hypothesis Generation in Astronomy I. Ciucă Y. Ting 丁 Sandor Kruk K. Iyer 19 7 0 20 Jun 2023
RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing Zilun Zhang Tiancheng Zhao Yulong Guo Jianwei Yin DiffM VLM 32 52 0 20 Jun 2023
Group Orthogonalization Regularization For Vision Models Adaptation and Robustness Yoav Kurtz Noga Bar Raja Giryes 24 0 0 16 Jun 2023
Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictions Dongshuo Yin Xueting Han Bin Li Hao Feng Jinghua Bai VPVLM 28 17 0 16 Jun 2023