Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 985 papers shown

Title
Efficient Domain Adaptation for Speech Foundation Models Yue Liu DongSeon Hwang Zhouyuan Huo Junwen Bai Guru Prakash ... K. Sim Yu Zhang Wei Han Trevor Strohman F. Beaufays AI4CE 54 24 0 03 Feb 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 36 47 0 02 Feb 2023
idT5: Indonesian Version of Multilingual T5 Transformer Mukhlish Fuadi A. Wibawa S. Sumpeno 19 6 0 02 Feb 2023
An Empirical Study on the Transferability of Transformer Modules in Parameter-Efficient Fine-Tuning Mohammad AkbarTajari S. Rajaee Mohammad Taher Pilehvar 19 2 0 01 Feb 2023
Differentiable Entailment for Parameter Efficient Few Shot Learning Ethan Kim Jerry Yang 29 0 0 31 Jan 2023
AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning Han Zhou Xingchen Wan Ivan Vulić Anna Korhonen 29 45 0 28 Jan 2023
Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling Kolby Nottingham Prithviraj Ammanabrolu Alane Suhr Yejin Choi Hannaneh Hajishirzi Sameer Singh Roy Fox LLMAG LM&Ro 55 78 0 28 Jan 2023
Probing Out-of-Distribution Robustness of Language Models with Parameter-Efficient Transfer Learning Hyunsoo Cho Choonghyun Park Junyeop Kim Sungmin Cho Kang Min Yoo Sang-goo Lee OODD 35 3 0 27 Jan 2023
One Model for All Domains: Collaborative Domain-Prefix Tuning for Cross-Domain NER Xiang Chen Lei Li Q. Fei Ningyu Zhang Chuanqi Tan Yong-jia Jiang Fei Huang Huajun Chen 31 23 0 25 Jan 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 28 3 0 24 Jan 2023
The Semantic Scholar Open Data Platform Rodney Michael Kinney Chloe Anastasiades Russell Authur Iz Beltagy Jonathan Bragg ... Caroline M Wu Jiangjiang Yang Angele Zamarron Madeleine van Zuylen Daniel S. Weld 34 94 0 24 Jan 2023
Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning Malte Ostendorff Georg Rehm CLIP VLM CLL 46 24 0 23 Jan 2023
Differentially Private Natural Language Models: Recent Advances and Future Directions Lijie Hu Ivan Habernal Lei Shen Di Wang AAML 37 19 0 22 Jan 2023
BERT-ERC: Fine-tuning BERT is Enough for Emotion Recognition in Conversation Xiangyu Qin Zhiyu Wu J. Cui Ting Zhang Yanran Li Jian Luan Bin Wang L. xilinx Wang 40 26 0 17 Jan 2023
Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models Zhiqiu Lin Samuel Yu Zhiyi Kuang Deepak Pathak Deva Ramana VLM 22 102 0 16 Jan 2023
FUN with Fisher: Improving Generalization of Adapter-Based Cross-lingual Transfer with Scheduled Unfreezing Chen Cecilia Liu Jonas Pfeiffer Ivan Vulić Iryna Gurevych CLL 34 9 0 13 Jan 2023
Multimodal Inverse Cloze Task for Knowledge-based Visual Question Answering Paul Lerner O. Ferret C. Guinaudeau 29 9 0 11 Jan 2023
Exploring Efficient Few-shot Adaptation for Vision Transformers C. Xu Siqian Yang Yabiao Wang Zhanxiong Wang Yanwei Fu Xiangyang Xue 35 16 0 06 Jan 2023
Parameter-Efficient Fine-Tuning Design Spaces Jiaao Chen Aston Zhang Xingjian Shi Mu Li Alexander J. Smola Diyi Yang 54 59 0 04 Jan 2023
Sample-Efficient Unsupervised Domain Adaptation of Speech Recognition Systems A case study for Modern Greek Georgios Paraskevopoulos Theodoros Kouzelis Georgios Rouvalis Athanasios Katsamanis Vassilis Katsouros Alexandros Potamianos VLM 41 7 0 31 Dec 2022
Principled and Efficient Transfer Learning of Deep Models via Neural Collapse Xiao Li Sheng Liu Jin-li Zhou Xin Lu C. Fernandez‐Granda Zhihui Zhu Q. Qu AAML 33 19 0 23 Dec 2022
MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning Zhiyang Xu Ying Shen Lifu Huang MLLM 34 110 0 21 Dec 2022
KronA: Parameter Efficient Tuning with Kronecker Adapter Ali Edalati Marzieh S. Tahaei I. Kobyzev V. Nia J. Clark Mehdi Rezagholizadeh 45 95 0 20 Dec 2022
Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Training Kelly Marchisio Patrick Lewis Yihong Chen Mikel Artetxe 40 16 0 20 Dec 2022
HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation Hamish Ivison Akshita Bhagia Yizhong Wang Hannaneh Hajishirzi Matthew E. Peters 58 16 0 20 Dec 2022
When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods Zhuo Zhang Yuanhang Yang Yong Dai Lizhen Qu Zenglin Xu FedML 53 66 0 20 Dec 2022
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting Zheng-Xin Yong Hailey Schoelkopf Niklas Muennighoff Alham Fikri Aji David Ifeoluwa Adelani ... Genta Indra Winata Stella Biderman Edward Raff Dragomir R. Radev Vassilina Nikoulina CLL VLM AI4CE LRM 40 81 0 19 Dec 2022
PoE: a Panel of Experts for Generalized Automatic Dialogue Assessment Chen Zhang L. F. D’Haro Qiquan Zhang Thomas Friedrichs Haizhou Li 38 7 0 18 Dec 2022
Decoder Tuning: Efficient Language Understanding as Decoding Ganqu Cui Wentao Li Ning Ding Longtao Huang Zhiyuan Liu Maosong Sun 31 6 0 16 Dec 2022
Injecting Domain Knowledge in Language Models for Task-Oriented Dialogue Systems Denis Emelin Daniele Bonadiman Sawsan Alqahtani Yi Zhang Saab Mansour 28 17 0 15 Dec 2022
Federated Few-Shot Learning for Mobile NLP Dongqi Cai Shangguang Wang Yaozong Wu F. Lin Mengwei Xu FedML 15 12 0 12 Dec 2022
Parameter-Efficient Finetuning of Transformers for Source Code Shamil Ayupov Nadezhda Chirkova 22 17 0 12 Dec 2022
Searching for Effective Multilingual Fine-Tuning Methods: A Case Study in Summarization Yiwei Qin Graham Neubig Pengfei Liu 29 3 0 12 Dec 2022
Federated Learning for Inference at Anytime and Anywhere Zicheng Liu Da Li Javier Fernandez-Marques Stefanos Laskaridis Yan Gao Łukasz Dudziak Stan Z. Li S. Hu Timothy M. Hospedales FedML 37 5 0 08 Dec 2022
Parameter Efficient Transfer Learning for Various Speech Processing Tasks Shinta Otake Rei Kawakami Nakamasa Inoue 24 16 0 06 Dec 2022
Continual Learning for On-Device Speech Recognition using Disentangled Conformers Anuj Diwan Ching-Feng Yeh Wei-Ning Hsu Paden Tomasello Eunsol Choi David Harwath Abdel-rahman Mohamed CLL BDL 35 8 0 02 Dec 2022
CHAPTER: Exploiting Convolutional Neural Network Adapters for Self-supervised Speech Models Zih-Ching Chen Yu-Shun Sung Hung-yi Lee 29 16 0 01 Dec 2022
Data-Efficient Finetuning Using Cross-Task Nearest Neighbors Hamish Ivison Noah A. Smith Hannaneh Hajishirzi Pradeep Dasigi 38 20 0 01 Dec 2022
Learning Label Modular Prompts for Text Classification in the Wild Hailin Chen Amrita Saha Chenyu You Steven C. H. Hoi OOD VLM 28 5 0 30 Nov 2022
SPARTAN: Sparse Hierarchical Memory for Parameter-Efficient Transformers Ameet Deshpande Md Arafat Sultan Anthony Ferritto Ashwin Kalyan Karthik Narasimhan Avirup Sil MoE 51 1 0 29 Nov 2022
Few-shot Query-Focused Summarization with Prefix-Merging Ruifeng Yuan Zili Wang Ziqiang Cao Wenjie Li 37 7 0 29 Nov 2022
Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models Peter Henderson E. Mitchell Christopher D. Manning Dan Jurafsky Chelsea Finn 27 47 0 27 Nov 2022
Alignment-Enriched Tuning for Patch-Level Pre-trained Document Image Models Lei Wang Jian He Xingdong Xu Ning Liu Hui-juan Liu 41 2 0 27 Nov 2022
SciRepEval: A Multi-Format Benchmark for Scientific Document Representations Amanpreet Singh Mike DÁrcy Arman Cohan Doug Downey Sergey Feldman 34 82 0 23 Nov 2022
Semi-Supervised Lifelong Language Learning Ying Zhao Yinhe Zheng Yu Bowen Zhiliang Tian Dongkyu Lee Jian Sun Haiyang Yu Yongbin Li N. Zhang CLL KELM 54 3 0 23 Nov 2022
VoP: Text-Video Co-operative Prompt Tuning for Cross-Modal Retrieval Siteng Huang Biao Gong Yulin Pan Jianwen Jiang Yiliang Lv Yuyuan Li Donglin Wang VLM VPVLM 22 41 0 23 Nov 2022
Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans? Neeraj Varshney Man Luo Chitta Baral RALM 23 11 0 23 Nov 2022
Continual Learning of Natural Language Processing Tasks: A Survey Zixuan Ke Bin Liu KELM CLL VLM 39 69 0 23 Nov 2022
HyperTuning: Toward Adapting Large Language Models without Back-propagation Jason Phang Yi Mao Pengcheng He Weizhu Chen 46 30 0 22 Nov 2022
Linear Interpolation In Parameter Space is Good Enough for Fine-Tuned Language Models Mark Rofin Nikita Balagansky Daniil Gavrilov MoMe KELM 38 5 0 22 Nov 2022