Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 987 papers shown

Title
Do We Really Need a Large Number of Visual Prompts? Youngeun Kim Yuhang Li Abhishek Moitra Ruokai Yin Priyadarshini Panda VLM VPVLM 50 5 0 26 May 2023
Parameter-Efficient Fine-Tuning without Introducing New Latency Baohao Liao Yan Meng Christof Monz 24 49 0 26 May 2023
Task-aware Distributed Source Coding under Dynamic Bandwidth Po-han Li S. Ankireddy Ruihan Zhao Hossein Nourkhiz Mahjoub Ehsan Moradi-Pari Ufuk Topcu Sandeep Chinchali Hyeji Kim 42 9 0 24 May 2023
MultiFusion: Fusing Pre-Trained Models for Multi-Lingual, Multi-Modal Image Generation Marco Bellagente Manuel Brack H. Teufel Felix Friedrich Bjorn Deiseroth ... Koen Oostermeijer Andres Felipe Cruz Salinas P. Schramowski Kristian Kersting Samuel Weinbach 45 16 0 24 May 2023
Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning Zhen-Ru Zhang Chuanqi Tan Haiyang Xu Chengyu Wang Jun Huang Songfang Huang 35 29 0 24 May 2023
Bi-Drop: Enhancing Fine-tuning Generalization via Synchronous sub-net Estimation and Optimization Shoujie Tong Heming Xia Damai Dai Runxin Xu Tianyu Liu Binghuai Lin Yunbo Cao Zhifang Sui 33 0 0 24 May 2023
VIP5: Towards Multimodal Foundation Models for Recommendation Shijie Geng Juntao Tan Shuchang Liu Zuohui Fu Yongfeng Zhang 37 73 0 23 May 2023
Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding Yu Zhang Hao Cheng Zhihong Shen Xiaodong Liu Yejiang Wang Jianfeng Gao 37 14 0 23 May 2023
Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantization Jeonghoon Kim J. H. Lee Sungdong Kim Joonsuk Park Kang Min Yoo S. Kwon Dongsoo Lee MQ 49 100 0 23 May 2023
Out-of-Distribution Generalization in Text Classification: Past, Present, and Future Linyi Yang Yangqiu Song Xuan Ren Chenyang Lyu Yidong Wang Lingqiao Liu Jindong Wang Jennifer Foster Yue Zhang OOD 44 2 0 23 May 2023
When Does Aggregating Multiple Skills with Multi-Task Learning Work? A Case Study in Financial NLP Jingwei Ni Zhijing Jin Qian Wang Mrinmaya Sachan Markus Leippold AIFin 28 6 0 23 May 2023
MasakhaPOS: Part-of-Speech Tagging for Typologically Diverse African Languages Cheikh M. Bamba Dione David Ifeoluwa Adelani Peter Nabende Jesujoba Oluwadara Alabi Thapelo Sindane ... Seydou T. Traoré C. Uchechukwu Aliyu Yusuf M. Abdullahi Dietrich Klakow 32 13 0 23 May 2023
CombLM: Adapting Black-Box Language Models through Small Fine-Tuned Models Aitor Ormazabal Mikel Artetxe Eneko Agirre 50 19 0 23 May 2023
A Dive into SAM Prior in Image Restoration Zeyu Xiao Jiawang Bai Zhihe Lu Zhiwei Xiong 34 16 0 23 May 2023
Small Language Models Improve Giants by Rewriting Their Outputs Giorgos Vernikos Arthur Bravzinskas Jakub Adamek Jonathan Mallinson Aliaksei Severyn Eric Malmi BDL LRM 43 14 0 22 May 2023
TaskWeb: Selecting Better Source Tasks for Multi-task NLP Joongwon Kim Akari Asai Gabriel Ilharco Hannaneh Hajishirzi 42 11 0 22 May 2023
SPARSEFIT: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanations Jesus Solano Oana-Maria Camburu Pasquale Minervini 25 1 0 22 May 2023
TADA: Efficient Task-Agnostic Domain Adaptation for Transformers Chia-Chien Hung Lukas Lange Jannik Strötgen 43 9 0 22 May 2023
ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings Shibo Hao Tianyang Liu Zhen Wang Zhiting Hu RALM LLMAG 78 174 0 19 May 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 52 83 0 19 May 2023
Differentially Private Adapters for Parameter Efficient Acoustic Modeling Chun-Wei Ho Chao-Han Huck Yang Sabato Marco Siniscalchi 31 1 0 19 May 2023
Parameter-Efficient Learning for Text-to-Speech Accent Adaptation Lijie Yang Chao-Han Huck Yang Jen-Tzung Chien 27 11 0 18 May 2023
PMIndiaSum: Multilingual and Cross-lingual Headline Summarization for Languages in India Ashok Urlana Pinzhen Chen Zheng Zhao Shay B. Cohen Manish Shrivastava Barry Haddow 44 9 0 15 May 2023
Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunity Raman Dutt Linus Ericsson Pedro Sanchez Sotirios A. Tsaftaris Timothy M. Hospedales MedIm 42 50 0 14 May 2023
A Comprehensive Analysis of Adapter Efficiency Nandini Mundra Sumanth Doddapaneni Raj Dabre Anoop Kunchukuttan Ratish Puduppully Mitesh M. Khapra 31 10 0 12 May 2023
Self-Chained Image-Language Model for Video Localization and Question Answering Shoubin Yu Jaemin Cho Prateek Yadav Joey Tianyi Zhou 61 131 0 11 May 2023
LACoS-BLOOM: Low-rank Adaptation with Contrastive objective on 8 bits Siamese-BLOOM Wenhui Hua Brian Williams Davood Shamsi 30 3 0 10 May 2023
Adapt and Align to Improve Zero-Shot Sketch-Based Image Retrieval Shiyin Dong Mingrui Zhu N. Wang Xinbo Gao VLM 40 3 0 09 May 2023
Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterization Anastasia Razdaibiedina Yuning Mao Rui Hou Madian Khabsa M. Lewis Jimmy Ba Amjad Almahairi VLM 29 43 0 06 May 2023
VendorLink: An NLP approach for Identifying & Linking Vendor Migrants & Potential Aliases on Darknet Markets V. Saxena Nils Rethmeier Gijs Van Dijck Gerasimos Spanakis 26 6 0 04 May 2023
Learning Language-Specific Layers for Multilingual Machine Translation Telmo Pires Robin M. Schmidt Yi-Hsiu Liao Stephan Peitz 55 17 0 04 May 2023
Using Language Models on Low-end Hardware Silin Gao Beatriz Borges Saya Kanno Antoine Bosselut 24 0 0 03 May 2023
Making the Most of What You Have: Adapting Pre-trained Visual Language Models in the Low-data Regime Chuhan Zhang Antoine Miech Jiajun Shen Jean-Baptiste Alayrac Pauline Luc VLM VPVLM 47 2 0 03 May 2023
Consolidator: Mergeable Adapter with Grouped Connections for Visual Adaptation Tianxiang Hao Hui Chen Yuchen Guo Guiguang Ding 61 16 0 30 Apr 2023
GMNLP at SemEval-2023 Task 12: Sentiment Analysis with Phylogeny-Based Adapters Md Mahfuz Ibn Alam Ruoyu Xie Fahim Faisal Antonios Anastasopoulos 37 3 0 25 Apr 2023
PEFT-Ref: A Modular Reference Architecture and Typology for Parameter-Efficient Finetuning Techniques Mohammed Sabry Anya Belz 48 8 0 24 Apr 2023
MasakhaNEWS: News Topic Classification for African languages David Ifeoluwa Adelani Marek Masiak Israel Abebe Azime Jesujoba Oluwadara Alabi A. Tonja ... Moges Ahmed Mehamed Evrard Ngabire Jules Jules Ivan Ssenkungu Pontus Stenetorp 33 24 0 19 Apr 2023
Learning to Compress Prompts with Gist Tokens Jesse Mu Xiang Lisa Li Noah D. Goodman VLM 60 208 0 17 Apr 2023
A Comparative Study between Full-Parameter and LoRA-based Fine-Tuning on Chinese Instruction Data for Instruction Following Large Language Model Xianghui Sun Yunjie Ji Baochang Ma Xiangang Li ALM 13 18 0 17 Apr 2023
Instance-aware Dynamic Prompt Tuning for Pre-trained Point Cloud Models Yaohua Zha Jinpeng Wang Tao Dai Bin Chen Zhi Wang Shutao Xia VLM 53 45 0 14 Apr 2023
Uncovering the Inner Workings of STEGO for Safe Unsupervised Semantic Segmentation Alexander Koenig Maximilian Schambach Johannes Otterbach 32 6 0 14 Apr 2023
Lossless Adaptation of Pretrained Vision Models For Robotic Manipulation Mohit Sharma Claudio Fantacci Yuxiang Zhou Skanda Koppula N. Heess Jonathan Scholz Y. Aytar VLM 55 29 0 13 Apr 2023
Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyond Ensheng Shi Yanlin Wang Hongyu Zhang Lun Du Shi Han Dongmei Zhang Hongbin Sun 44 43 0 11 Apr 2023
Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference Tao Lei Junwen Bai Siddhartha Brahma Joshua Ainslie Kenton Lee ... Vincent Zhao Yuexin Wu Yue Liu Yu Zhang Ming-Wei Chang BDL AI4CE 34 55 0 11 Apr 2023
CAVL: Learning Contrastive and Adaptive Representations of Vision and Language Shentong Mo Jingfei Xia Ihor Markevych CLIP VLM 35 1 0 10 Apr 2023
I2I: Initializing Adapters with Improvised Knowledge Tejas Srinivasan Furong Jia Mohammad Rostami Jesse Thomason CLL 37 6 0 04 Apr 2023
Strong Baselines for Parameter Efficient Few-Shot Fine-tuning S. Basu Daniela Massiceti S. Hu S. Feizi VLM 42 29 0 04 Apr 2023
Safety Analysis in the Era of Large Language Models: A Case Study of STPA using ChatGPT Yi Qi Xingyu Zhao Siddartha Khastgir Xiaowei Huang 26 15 0 03 Apr 2023
Probabilistic Prompt Learning for Dense Prediction Hyeongjun Kwon Taeyong Song Somi Jeong Jin-Hwa Kim Jinhyun Jang Kwanghoon Sohn VLM 56 19 0 03 Apr 2023
Procedure-Aware Pretraining for Instructional Video Understanding Honglu Zhou Roberto Martín-Martín Mubbasir Kapadia Silvio Savarese Juan Carlos Niebles 59 39 0 31 Mar 2023