Jointly Reparametrized Multi-Layer Adaptation for Efficient and Private Tuning

30 May 2023

Papers citing "Jointly Reparametrized Multi-Layer Adaptation for Efficient and Private Tuning"

31 / 31 papers shown

Title
Efficiently Tuned Parameters are Task Embeddings Wangchunshu Zhou Canwen Xu Julian McAuley 37 8 0 21 Oct 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 100 114 0 31 Aug 2022
Prompt Tuning for Generative Multimodal Pretrained Models Han Yang Junyang Lin An Yang Peng Wang Chang Zhou Hongxia Yang VLM LRM VPVLM 66 31 0 04 Aug 2022
Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices Mingbin Xu Congzheng Song Ye Tian Neha Agrawal Filip Granqvist ... Shiyi Han Yaqiao Deng Leo Liu Anmol Walia Alex Jin FedML 85 21 0 18 Jul 2022
Scaling Language Model Size in Cross-Device Federated Learning Jae Hun Ro Theresa Breiner Lara McConnaughey Mingqing Chen A. Suresh Shankar Kumar Rajiv Mathews FedML 59 24 0 31 Mar 2022
Quantifying Memorization Across Neural Language Models Nicholas Carlini Daphne Ippolito Matthew Jagielski Katherine Lee Florian Tramèr Chiyuan Zhang PILM 124 628 0 15 Feb 2022
UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning Yuning Mao Lambert Mathias Rui Hou Amjad Almahairi Hao Ma Jiawei Han Wen-tau Yih Madian Khabsa 63 190 0 14 Oct 2021
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 236 369 0 13 Oct 2021
Towards a Unified View of Parameter-Efficient Transfer Learning Junxian He Chunting Zhou Xuezhe Ma Taylor Berg-Kirkpatrick Graham Neubig AAML 129 952 0 08 Oct 2021
Opacus: User-Friendly Differential Privacy Library in PyTorch Ashkan Yousefpour I. Shilov Alexandre Sablayrolles Davide Testuggine Karthik Prasad ... Sayan Gosh Akash Bharadwaj Jessica Zhao Graham Cormode Ilya Mironov VLM 284 368 0 25 Sep 2021
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models Elad Ben-Zaken Shauli Ravfogel Yoav Goldberg 171 1,240 0 18 Jun 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 477 10,367 0 17 Jun 2021
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers Rabeeh Karimi Mahabadi James Henderson Sebastian Ruder MoE 111 486 0 08 Jun 2021
Numerical Composition of Differential Privacy Sivakanth Gopi Y. Lee Lukas Wutschitz 58 183 0 05 Jun 2021
Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation Mozhdeh Gheini Xiang Ren Jonathan May LRM 64 112 0 18 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 579 4,077 0 18 Apr 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 248 4,298 0 01 Jan 2021
WARP: Word-level Adversarial ReProgramming Karen Hambardzumyan Hrant Khachatrian Jonathan May AAML 324 351 0 01 Jan 2021
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning Armen Aghajanyan Luke Zettlemoyer Sonal Gupta 101 568 1 22 Dec 2020
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown Basel Alomair Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 501 1,943 0 14 Dec 2020
AdapterDrop: On the Efficiency of Adapters in Transformers Andreas Rucklé Gregor Geigle Max Glockner Tilman Beck Jonas Pfeiffer Nils Reimers Iryna Gurevych 121 266 0 22 Oct 2020
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners Timo Schick Hinrich Schütze 130 974 0 15 Sep 2020
Discovering Useful Sentence Representations from Large Pretrained Language Models Nishant Subramani Nivedita Suresh 24 5 0 20 Aug 2020
AdapterFusion: Non-Destructive Task Composition for Transfer Learning Jonas Pfeiffer Aishwarya Kamath Andreas Rucklé Kyunghyun Cho Iryna Gurevych CLL MoMe 139 858 0 01 May 2020
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 671 24,528 0 26 Jul 2019
Parameter-Efficient Transfer Learning for NLP N. Houlsby A. Giurgiu Stanislaw Jastrzebski Bruna Morrone Quentin de Laroussilhe Andrea Gesmundo Mona Attariyan Sylvain Gelly 217 4,499 0 02 Feb 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,114 0 11 Oct 2018
Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization Shashi Narayan Shay B. Cohen Mirella Lapata AILaw 146 1,682 0 27 Aug 2018
Measuring the Intrinsic Dimension of Objective Landscapes Chunyuan Li Heerad Farkhoor Rosanne Liu J. Yosinski 86 414 0 24 Apr 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,182 0 20 Apr 2018
Deep Learning with Differential Privacy Martín Abadi Andy Chu Ian Goodfellow H. B. McMahan Ilya Mironov Kunal Talwar Li Zhang FedML SyDa 216 6,155 0 01 Jul 2016