v1v2 (latest)

Step-by-Step Unmasking for Parameter-Efficient Fine-tuning of Large Language Models

26 August 2024

Aradhye Agarwal

Suhas Kamasetty Ramesh

Ayan Sengupta

Tanmoy Chakraborty

ArXiv (abs)PDF HTML

Papers citing "Step-by-Step Unmasking for Parameter-Efficient Fine-tuning of Large Language Models"

21 / 21 papers shown

Title
Parameter-Efficient Fine-Tuning without Introducing New Latency Baohao Liao Yan Meng Christof Monz 51 55 0 26 May 2023
Training Neural Networks with Fixed Sparse Masks Yi-Lin Sung Varun Nair Colin Raffel FedML 88 204 0 18 Nov 2021
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 306 4,408 0 27 Oct 2021
Towards a Unified View of Parameter-Efficient Transfer Learning Junxian He Chunting Zhou Xuezhe Ma Taylor Berg-Kirkpatrick Graham Neubig AAML 129 937 0 08 Oct 2021
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models Elad Ben-Zaken Shauli Ravfogel Yoav Goldberg 168 1,223 0 18 Jun 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 477 10,367 0 17 Jun 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 246 4,261 0 01 Jan 2021
Parameter-Efficient Transfer Learning with Diff Pruning Demi Guo Alexander M. Rush Yoon Kim 82 400 0 14 Dec 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 814 42,055 0 28 May 2020
AdapterFusion: Non-Destructive Task Composition for Transfer Learning Jonas Pfeiffer Aishwarya Kamath Andreas Rucklé Kyunghyun Cho Iryna Gurevych CLL MoMe 139 852 0 01 May 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 520 42,449 0 03 Dec 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 445 20,181 0 23 Oct 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 665 24,464 0 26 Jul 2019
Importance Estimation for Neural Network Pruning Pavlo Molchanov Arun Mallya Stephen Tyree I. Frosio Jan Kautz 3DPC 78 882 0 25 Jun 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 94,891 0 11 Oct 2018
Neural Network Acceptability Judgments Alex Warstadt Amanpreet Singh Samuel R. Bowman 233 1,407 0 31 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,182 0 20 Apr 2018
Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems Wang Ling Dani Yogatama Chris Dyer Phil Blunsom AIMat 88 734 0 11 May 2017
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 524 4,479 0 18 Apr 2017
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 286 8,134 0 16 Jun 2016
Automatic differentiation in machine learning: a survey A. G. Baydin Barak A. Pearlmutter Alexey Radul J. Siskind PINN AI4CE ODL 166 2,808 0 20 Feb 2015