Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 947 papers shown

Title
Transferable Adversarial Attacks on SAM and Its Downstream Models Song Xia Wenhan Yang Yi Yu Xun Lin Henghui Ding Lingyu Duan Xudong Jiang AAML SILM 66 6 0 26 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies Liwen Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 52 3 0 24 Oct 2024
Understanding Layer Significance in LLM Alignment Guangyuan Shi Zexin Lu Xiaoyu Dong Wenlong Zhang Xuanyu Zhang Yujie Feng Xiao-Ming Wu 58 2 0 23 Oct 2024
Closed-form merging of parameter-efficient modules for Federated Continual Learning Riccardo Salami Pietro Buzzega Matteo Mosconi Jacopo Bonato Luigi Sabetta Simone Calderara FedML MoMe CLL 44 2 0 23 Oct 2024
LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics Thomas Robert M. Safaryan Ionut-Vlad Modoranu Dan Alistarh ODL 36 2 0 21 Oct 2024
Swiss Army Knife: Synergizing Biases in Knowledge from Vision Foundation Models for Multi-Task Learning Yuxiang Lu Shengcao Cao Yu-xiong Wang 57 1 0 18 Oct 2024
A Unified View of Delta Parameter Editing in Post-Trained Large-Scale Models Qiaoyu Tang Le Yu Bowen Yu Hongyu Lin Keming Lu Yaojie Lu Xianpei Han Le Sun MoMe 39 1 0 17 Oct 2024
Router-Tuning: A Simple and Effective Approach for Enabling Dynamic-Depth in Transformers Shwai He Tao Ge Guoheng Sun Bowei Tian Xiaoyang Wang Ang Li MoE 54 1 0 17 Oct 2024
Transfer Learning with Foundational Models for Time Series Forecasting using Low-Rank Adaptations M. Germán-Morales A. J. Rivera-Rivas M. J. del Jesus Díaz C. J. Carmona AI4TS AI4CE 56 0 0 15 Oct 2024
ATLAS: Adapter-Based Multi-Modal Continual Learning with a Two-Stage Learning Strategy Hong Li Zhiquan Tan Xingyu Li Weiran Huang CLL MoMe 43 1 0 14 Oct 2024
Scalable Multi-Domain Adaptation of Language Models using Modular Experts Peter Schafhalter Shun Liao Yanqi Zhou Chih-Kuan Yeh Arun Kandoor James Laudon MoE 34 1 0 14 Oct 2024
Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation Guozhi Liu Weiwei Lin Tiansheng Huang Ruichao Mo Qi Mu Li Shen AAML 69 11 0 13 Oct 2024
ELICIT: LLM Augmentation via External In-Context Capability Futing Wang Jianhao Yan Yue Zhang Tao Lin 47 0 0 12 Oct 2024
Adapters for Altering LLM Vocabularies: What Languages Benefit the Most? HyoJung Han Akiko Eriguchi Haoran Xu Hieu T. Hoang Marine Carpuat Huda Khayrallah VLM 43 2 0 12 Oct 2024
MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning Yaming Yang Dilxat Muhtar Yelong Shen Yuefeng Zhan Jianfeng Liu ... Denvy Deng Feng Sun Qi Zhang Weizhu Chen Yunhai Tong MoE MoMe 46 2 0 12 Oct 2024
DA-Ada: Learning Domain-Aware Adapter for Domain Adaptive Object Detection Yiming Li Rui Zhang Hantao Yao X. Zhang Yifan Hao Xinkai Song Xiaqing Li Yongwei Zhao Ling Li Yunji Chen ObjD VLM 39 4 0 11 Oct 2024
Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both Abhijnan Nath Changsoo Jung Ethan Seefried Nikhil Krishnaswamy 212 1 0 11 Oct 2024
Parameter-Efficient Fine-Tuning of State Space Models Kevin Galim Wonjun Kang Yuchen Zeng H. Koo Kangwook Lee 33 4 0 11 Oct 2024
SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture Jiayi Han Liang Du Hongwei Du Xiangguo Zhou Yiwen Wu Weibo Zheng Donghong Han CLL MoMe MoE 38 2 0 10 Oct 2024
Unstable Unlearning: The Hidden Risk of Concept Resurgence in Diffusion Models Vinith Suriyakumar Rohan Alur Ayush Sekhari Manish Raghavan Ashia Wilson 58 2 0 10 Oct 2024
Detecting Bias and Enhancing Diagnostic Accuracy in Large Language Models for Healthcare Pardis Sadat Zahraei Zahra Shakeri LM&MA 26 0 0 09 Oct 2024
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs Ruijia Niu D. Wu Rose Yu Yi Ma 35 1 0 09 Oct 2024
PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model Patches Rana Muhammad Shahroz Khan Pingzhi Li Sukwon Yun Zhenyu Wang S. Nirjon Chau-Wai Wong Tianlong Chen KELM 43 2 0 08 Oct 2024
Provable Weak-to-Strong Generalization via Benign Overfitting David X. Wu A. Sahai 79 6 0 06 Oct 2024
LoRTA: Low Rank Tensor Adaptation of Large Language Models Ignacio Hounie Charilaos I. Kanatsoulis Arnuv Tandon Alejandro Ribeiro 38 0 0 05 Oct 2024
Reward-RAG: Enhancing RAG with Reward Driven Supervision Thang Nguyen Peter Chin Yu-Wing Tai RALM 45 4 0 03 Oct 2024
Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts Minh Le Chau Nguyen Huy Nguyen Quyen Tran Trung Le Nhat Ho 44 5 0 03 Oct 2024
Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization Xinhao Yao Hongjin Qian Xiaolin Hu Gengze Xu Wei Liu Jian Luan Bin Wang Yong-Jin Liu 48 0 0 03 Oct 2024
Differentially Private Parameter-Efficient Fine-tuning for Large ASR Models Hongbin Liu Lun Wang Om Thakkar Abhradeep Thakurta Arun Narayanan 37 0 0 02 Oct 2024
DLP-LoRA: Efficient Task-Specific LoRA Fusion with a Dynamic, Lightweight Plugin for Large Language Models Yuxuan Zhang Ruizhe Li MoMe 58 0 0 02 Oct 2024
Selective Aggregation for Low-Rank Adaptation in Federated Learning Pengxin Guo Shuang Zeng Y. Wang Huijie Fan Feifei Wang Liangqiong Qu FedML 50 11 0 02 Oct 2024
LoRanPAC: Low-rank Random Features and Pre-trained Models for Bridging Theory and Practice in Continual Learning Liangzu Peng Juan Elenter Joshua Agterberg Alejandro Ribeiro René Vidal VLM CLL 48 1 0 01 Oct 2024
HDMoLE: Mixture of LoRA Experts with Hierarchical Routing and Dynamic Thresholds for Fine-Tuning LLM-based ASR Models Bingshen Mu Kun Wei Qijie Shao Yong Xu Lei Xie MoE 47 2 0 30 Sep 2024
Designing Domain-Specific Large Language Models: The Critical Role of Fine-Tuning in Public Opinion Simulation Haocheng Lin ALM 29 1 0 28 Sep 2024
A Parameter-Efficient Tuning Framework for Language-guided Object Grounding and Robot Grasping Houjian Yu Mingen Li Alireza Rezazadeh Yang Yang Changhyun Choi 52 1 0 28 Sep 2024
On the Implicit Relation Between Low-Rank Adaptation and Differential Privacy Saber Malekmohammadi G. Farnadi 32 2 0 26 Sep 2024
PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularization Yao Ni Shan Zhang Piotr Koniusz 226 5 0 25 Sep 2024
Cross-Lingual Speech Emotion Recognition: Humans vs. Self-Supervised Models Zhichen Han Tianqi Geng Hui Feng Jiahong Yuan Korin Richmond Yuanchao Li 43 1 0 25 Sep 2024
Sequential Learning in the Dense Associative Memory Hayden McAlister Anthony Robins Lech Szymanski CLL 216 1 0 24 Sep 2024
Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely Siyun Zhao Yuqing Yang Zilong Wang Zhiyuan He Luna Qiu Lili Qiu SyDa RALM 3DV 46 36 0 23 Sep 2024
Orthogonal Finetuning for Direct Preference Optimization Chenxu Yang Ruipeng Jia Naibin Gu Zheng Lin Siyuan Chen Chao Pang Weichong Yin Yu Sun Hua Wu Weiping Wang 37 0 0 23 Sep 2024
Dynamic Integration of Task-Specific Adapters for Class Incremental Learning Jiashuo Li Shaokun Wang Bo Qian Yuhang He Xing Wei Qiang Wang Yihong Gong CLL 85 2 0 23 Sep 2024
Interpreting Arithmetic Mechanism in Large Language Models through Comparative Neuron Analysis Zeping Yu Sophia Ananiadou LRM MILM 32 7 0 21 Sep 2024
Drift to Remember Jin Du Xinsong Zhang Hao Shen Xun Xian Ganghua Wang Jiawei Zhang Yuhong Yang Na Li Jia Liu Jie Ding CLL 21 0 0 21 Sep 2024
RingMo-Aerial: An Aerial Remote Sensing Foundation Model With A Affine Transformation Contrastive Learning Wenhui Diao Haichen Yu Kaiyue Kang Tong Ling Di Liu ... Hanbo Bi Libo Ren Xuexue Li Yongqiang Mao Xian Sun 42 1 0 20 Sep 2024
Decoding Style: Efficient Fine-Tuning of LLMs for Image-Guided Outfit Recommendation with Preference Najmeh Forouzandehmehr Nima Farrokhsiar Ramin Giahi Evren Körpeoglu Kannan Achan 20 0 0 18 Sep 2024
Recent advances in deep learning and language models for studying the microbiome Binghao Yan Yunbi Nam Lingyao Li Rebecca A Deek Hongzhe Li Siyuan Ma 23 1 0 15 Sep 2024
Large Language Model Can Transcribe Speech in Multi-Talker Scenarios with Versatile Instructions Lingwei Meng Shujie Hu Jiawen Kang Zhaoqing Li Yuejiao Wang Wenxuan Wu Xixin Wu Xunying Liu Helen Meng AuLLM 75 2 0 13 Sep 2024
iConFormer: Dynamic Parameter-Efficient Tuning with Input-Conditioned Adaptation Hayeon Jo Hyesong Choi Minhee Cho Dongbo Min 44 1 0 04 Sep 2024
Multi-Modal Adapter for Vision-Language Models Dominykas Seputis Serghei Mihailov Soham Chatterjee Zehao Xiao VLM 38 1 0 03 Sep 2024