Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 1,010 papers shown

Title
Read Between the Layers: Leveraging Multi-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models Kyra Ahrens Hans Hergen Lehmann Jae Hee Lee Stefan Wermter CLL 40 7 0 13 Dec 2023
Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models Junhao Zheng Shengjie Qiu Qianli Ma 32 9 0 13 Dec 2023
Traffic Signal Control Using Lightweight Transformers: An Offline-to-Online RL Approach Xingshuai Huang Di Wu Benoit Boulet OffRL 32 2 0 12 Dec 2023
Dynamic Corrective Self-Distillation for Better Fine-Tuning of Pretrained Models Ibtihel Amara Vinija Jain Aman Chadha 32 0 0 12 Dec 2023
Mutual Enhancement of Large and Small Language Models with Cross-Silo Knowledge Transfer Yongheng Deng Ziqing Qiao Ju Ren Yang Liu Yaoxue Zhang 51 11 0 10 Dec 2023
Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts Jialin Wu Xia Hu Yaqing Wang Bo Pang Radu Soricut MoE 32 14 0 01 Dec 2023
Hyperparameter Optimization for Large Language Model Instruction-Tuning C. Tribes Sacha Benarroch-Lelong Peng Lu I. Kobyzev 37 12 0 01 Dec 2023
Vision-Language Models Learn Super Images for Efficient Partially Relevant Video Retrieval Taichi Nishimura Shota Nakada Masayoshi Kondo VLM 26 0 0 01 Dec 2023
The Philosopher's Stone: Trojaning Plugins of Large Language Models Tian Dong Minhui Xue Guoxing Chen Rayne Holland Shaofeng Li Yan Meng Zhen Liu Haojin Zhu AAML 39 11 0 01 Dec 2023
Efficient Stitchable Task Adaptation Haoyu He Zizheng Pan Jing Liu Jianfei Cai Bohan Zhuang 60 3 0 29 Nov 2023
End-to-End Temporal Action Detection with 1B Parameters Across 1000 Frames Shuming Liu Chen-Da Liu-Zhang Chen Zhao Guohao Li 55 25 0 28 Nov 2023
A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA Damjan Kalajdzievski ALM 33 80 0 28 Nov 2023
DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer Junyuan Hong Jiachen T. Wang Chenhui Zhang Zhangheng Li Yue Liu Zhangyang Wang 64 32 0 27 Nov 2023
Efficient Rehearsal Free Zero Forgetting Continual Learning using Adaptive Weight Modulation Yonatan Sverdlov Shimon Ullman 58 0 0 26 Nov 2023
PrivateLoRA For Efficient Privacy Preserving LLM Yiming Wang Yu Lin Xiaodong Zeng Guannan Zhang 71 11 0 23 Nov 2023
Exploring Methods for Cross-lingual Text Style Transfer: The Case of Text Detoxification Daryna Dementieva Daniil Moskovskiy David Dale Alexander Panchenko 59 16 0 23 Nov 2023
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning Han Guo P. Greengard Eric P. Xing Yoon Kim MQ 49 45 0 20 Nov 2023
PELMS: Pre-training for Effective Low-Shot Multi-Document Summarization Joseph Peper Wenzhao Qiu Lu Wang 33 0 0 16 Nov 2023
Tied-Lora: Enhancing parameter efficiency of LoRA with weight tying Adithya Renduchintala Tugrul Konuk Oleksii Kuchaiev MoMe 46 43 0 16 Nov 2023
Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization Alexandra Chronopoulou Jonas Pfeiffer Joshua Maynez Xinyi Wang Sebastian Ruder Priyanka Agrawal MoMe 43 16 0 15 Nov 2023
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 38 27 0 15 Nov 2023
When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks Hao Peng Xiaozhi Wang Jianhui Chen Weikai Li Yunjia Qi ... Zhili Wu Kaisheng Zeng Bin Xu Lei Hou Juanzi Li 39 28 0 15 Nov 2023
On the Analysis of Cross-Lingual Prompt Tuning for Decoder-based Multilingual Model Nohil Park Joonsuk Park Kang Min Yoo Sungroh Yoon 53 3 0 14 Nov 2023
Aggregate, Decompose, and Fine-Tune: A Simple Yet Effective Factor-Tuning Method for Vision Transformer Dongping Chen 46 3 0 12 Nov 2023
DONUT-hole: DONUT Sparsification by Harnessing Knowledge and Optimizing Learning Efficiency Azhar Shaikh Michael Cochez Denis Diachkov Michiel de Rijcke Sahar Yousefi 51 0 0 09 Nov 2023
AdaSent: Efficient Domain-Adapted Sentence Embeddings for Few-Shot Classification Yongxin Huang Kexin Wang Sourav Dutta Raj Nath Patel Goran Glavaš Iryna Gurevych VLM 30 4 0 01 Nov 2023
Unlearn What You Want to Forget: Efficient Unlearning for LLMs Jiaao Chen Diyi Yang MU 40 141 0 31 Oct 2023
Unified Representation for Non-compositional and Compositional Expressions Ziheng Zeng Suma Bhat 32 3 0 29 Oct 2023
Punica: Multi-Tenant LoRA Serving Lequn Chen Zihao Ye Yongji Wu Danyang Zhuo Luis Ceze Arvind Krishnamurthy 44 34 0 28 Oct 2023
Parameter-Efficient Methods for Metastases Detection from Clinical Notes Maede Ashofteh Barabadi Xiaodan Zhu Wai-Yip Chan Amber L. Simpson Richard K G Do 52 1 0 27 Oct 2023
FedPEAT: Convergence of Federated Learning, Parameter-Efficient Fine Tuning, and Emulator Assisted Tuning for Artificial Intelligence Foundation Models with Mobile Edge Computing Terence Jie Chua Wen-li Yu Junfeng Zhao Kwok-Yan Lam FedML 37 5 0 26 Oct 2023
Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policies Michael Beukman Devon Jarvis Richard Klein Steven D. James Benjamin Rosman 31 10 0 25 Oct 2023
Performative Prediction: Past and Future Moritz Hardt Celestine Mendler-Dünner 36 21 0 25 Oct 2023
Cascaded Multi-task Adaptive Learning Based on Neural Architecture Search Yingying Gao Shilei Zhang Zihao Cui Chao Deng Junlan Feng 30 0 0 23 Oct 2023
Scalable Neural Network Kernels Arijit Sehanobish Krzysztof Choromanski Yunfan Zhao Kumar Avinava Dubey Valerii Likhosherstov 67 5 0 20 Oct 2023
Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Model Abhijith Chintam Rahel Beloch Willem H. Zuidema Michael Hanna Oskar van der Wal 47 16 0 19 Oct 2023
Audio-AdapterFusion: A Task-ID-free Approach for Efficient and Non-Destructive Multi-task Speech Recognition Hillary Ngai Rohan Agrawal Neeraj Gaur Ronny Huang Parisa Haghani P. M. Mengibar MoMe 57 0 0 17 Oct 2023
Rethinking Class-incremental Learning in the Era of Large Pre-trained Models via Test-Time Adaptation Imad Eddine Marouf Subhankar Roy Enzo Tartaglione Stéphane Lathuilière CLL 42 3 0 17 Oct 2023
Domain Generalization Using Large Pretrained Models with Mixture-of-Adapters Gyuseong Lee Wooseok Jang Jin Hyeon Kim Jaewoo Jung Seungryong Kim MoE OOD 32 3 0 17 Oct 2023
Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning Chong Li Shaonan Wang Yunhao Zhang Jiajun Zhang Chengqing Zong 43 5 0 16 Oct 2023
Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook Ming Jin Qingsong Wen Yuxuan Liang Chaoli Zhang Siqiao Xue ... Shirui Pan Vincent S. Tseng Yu Zheng Lei Chen Hui Xiong AI4TS SyDa 57 118 0 16 Oct 2023
Decomposed Prompt Tuning via Low-Rank Reparameterization Yao Xiao Lu Xu Jiaxi Li Wei Lu Xiaoli Li VLM 33 6 0 16 Oct 2023
HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion Xian Liu Jian Ren Aliaksandr Siarohin Ivan Skorokhodov Yanyu Li Dahua Lin Xihui Liu Ziwei Liu Sergey Tulyakov 37 57 0 12 Oct 2023
Defending Our Privacy With Backdoors Dominik Hintersdorf Lukas Struppek Daniel Neider Kristian Kersting SILM AAML 36 2 0 12 Oct 2023
InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations Nils Feldhus Qianli Wang Tatiana Anikina Sahil Chopra Cennet Oguz Sebastian Möller 66 11 0 09 Oct 2023
Improving Discriminative Multi-Modal Learning with Large-Scale Pre-Trained Models Chenzhuang Du Yue Zhao Chonghua Liao Jiacheng You Jie Fu Hang Zhao 60 2 0 08 Oct 2023
Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Model Cheng Qian Chenyan Xiong Zhenghao Liu Zhiyuan Liu LRM 37 12 0 08 Oct 2023
SoK: Access Control Policy Generation from High-level Natural Language Requirements Sakuna Jayasundara N. Arachchilage Giovanni Russello 18 2 0 05 Oct 2023
Sweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task Adaptation Chen Dun Mirian Hipolito Garcia Guoqing Zheng Ahmed Hassan Awadallah Anastasios Kyrillidis Robert Sim 105 6 0 04 Oct 2023
NOLA: Compressing LoRA using Linear Combination of Random Basis Soroush Abbasi Koohpayegani K. Navaneet Parsa Nooralinejad Soheil Kolouri Hamed Pirsiavash 51 13 0 04 Oct 2023