Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Delta Tuning

24 October 2022

Yankai Lin

Xu Han

Zhiyuan Liu

Maosong Sun

Jie Zhou

ArXiv (abs)PDF HTML

Papers citing "Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Delta Tuning"

50 / 61 papers shown

Title
Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models Ning Ding Yujia Qin Guang Yang Fu Wei Zonghan Yang ... Jianfei Chen Yang Liu Jie Tang Juan Li Maosong Sun 89 204 0 14 Mar 2022
UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning Yuning Mao Lambert Mathias Rui Hou Amjad Almahairi Hao Ma Jiawei Han Wen-tau Yih Madian Khabsa 66 192 0 14 Oct 2021
Towards a Unified View of Parameter-Efficient Transfer Learning Junxian He Chunting Zhou Xuezhe Ma Taylor Berg-Kirkpatrick Graham Neubig AAML 135 952 0 08 Oct 2021
Datasets: A Community Library for Natural Language Processing Quentin Lhoest Albert Villanova del Moral Yacine Jernite A. Thakur Patrick von Platen ... Thibault Goehringer Victor Mustar François Lagunas Alexander M. Rush Thomas Wolf 218 614 0 07 Sep 2021
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models Elad Ben-Zaken Shauli Ravfogel Yoav Goldberg 184 1,243 0 18 Jun 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 490 10,496 0 17 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 149 855 0 14 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 589 4,093 0 18 Apr 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 252 4,299 0 01 Jan 2021
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning Armen Aghajanyan Luke Zettlemoyer Sonal Gupta 107 570 1 22 Dec 2020
HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection Binny Mathew Punyajoy Saha Seid Muhie Yimam Chris Biemann Pawan Goyal Animesh Mukherjee 122 581 0 18 Dec 2020
Semi-supervised URL Segmentation with Recurrent Neural Networks Pre-trained on Knowledge Graph Entities Hao Zhang Jae Hun Ro R. Sproat 44 13 0 05 Nov 2020
What Does This Acronym Mean? Introducing a New Dataset for Acronym Identification and Disambiguation Amir Pouran Ben Veyseh Franck Dernoncourt Quan Hung Tran Thien Huu Nguyen 50 55 0 28 Oct 2020
TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification Francesco Barbieri Jose Camacho-Collados Leonardo Neves Luis Espinosa-Anke VLM 84 725 0 23 Oct 2020
Explainable Automated Fact-Checking for Public Health Claims Neema Kotonya Francesca Toni 259 261 0 19 Oct 2020
"I'd rather just go to bed": Understanding Indirect Answers Annie Louis Dan Roth Filip Radlinski 60 48 0 07 Oct 2020
Effective Transfer Learning for Identifying Similar Questions: Matching User Questions to COVID-19 FAQs Clara H. McCreery Namit Katariya A. Kannan Manish Chablani X. Amatriain MedIm OOD 44 76 0 04 Aug 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 880 42,463 0 28 May 2020
How Context Affects Language Models' Factual Predictions Fabio Petroni Patrick Lewis Aleksandra Piktus Tim Rocktaschel Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM 62 239 0 10 May 2020
Neural CRF Model for Sentence Alignment in Text Simplification Chao Jiang Mounica Maddela Wuwei Lan Yang Zhong Wenyuan Xu 84 162 0 05 May 2020
ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning Michael Boratko Xiang Lorraine Li Rajarshi Das Timothy J. O'Gorman Daniel Le Andrew McCallum 97 58 0 02 May 2020
Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models Bill Yuchen Lin Seyeon Lee Rahul Khanna Xiang Ren AIMat 75 158 0 02 May 2020
AdapterFusion: Non-Destructive Task Composition for Transfer Learning Jonas Pfeiffer Aishwarya Kamath Andreas Rucklé Kyunghyun Cho Iryna Gurevych CLL MoMe 147 858 0 01 May 2020
BLiMP: The Benchmark of Linguistic Minimal Pairs for English Alex Warstadt Alicia Parrish Haokun Liu Anhad Mohananey Wei Peng Sheng-Fu Wang Samuel R. Bowman 87 493 0 02 Dec 2019
QASC: A Dataset for Question Answering via Sentence Composition Tushar Khot Peter Clark Michal Guerquin Peter Alexander Jansen Ashish Sabharwal CoGe 76 330 0 25 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 488 20,342 0 23 Oct 2019
QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questions Oyvind Tafjord Matt Gardner Kevin Lin Peter Clark 61 111 0 08 Sep 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 576 2,677 0 03 Sep 2019
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning Lifu Huang Ronan Le Bras Chandra Bhagavatula Yejin Choi AIMat RALM LRM 115 458 0 31 Aug 2019
Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning Pradeep Dasigi Nelson F. Liu Ana Marasović Noah A. Smith Matt Gardner RALM 73 174 0 16 Aug 2019
ELI5: Long Form Question Answering Angela Fan Yacine Jernite Ethan Perez David Grangier Jason Weston Michael Auli AI4MH ELM 106 624 0 22 Jul 2019
Explain Yourself! Leveraging Language Models for Commonsense Reasoning Nazneen Rajani Bryan McCann Caiming Xiong R. Socher ReLM LRM 89 566 0 06 Jun 2019
Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model Alexander R. Fabbri Irene Li Tianwei She Suyi Li Dragomir R. Radev 86 590 0 04 Jun 2019
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions Christopher Clark Kenton Lee Ming-Wei Chang Tom Kwiatkowski Michael Collins Kristina Toutanova 244 1,560 0 24 May 2019
HellaSwag: Can a Machine Really Finish Your Sentence? Rowan Zellers Ari Holtzman Yonatan Bisk Ali Farhadi Yejin Choi 182 2,532 0 19 May 2019
PAWS: Paraphrase Adversaries from Word Scrambling Yuan Zhang Jason Baldridge Luheng He 77 544 0 01 Apr 2019
Parameter-Efficient Transfer Learning for NLP N. Houlsby A. Giurgiu Stanislaw Jastrzebski Bruna Morrone Quentin de Laroussilhe Andrea Gesmundo Mona Attariyan Sylvain Gelly 221 4,518 0 02 Feb 2019
QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships Oyvind Tafjord Peter Clark Matt Gardner Wen-tau Yih Ashish Sabharwal AIMat 69 80 0 20 Nov 2018
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge Alon Talmor Jonathan Herzig Nicholas Lourie Jonathan Berant RALM 144 1,752 0 02 Nov 2018
ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension Sheng Zhang Xiaodong Liu Jingjing Liu Jianfeng Gao Kevin Duh Benjamin Van Durme 83 314 0 30 Oct 2018
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Zhilin Yang Peng Qi Saizheng Zhang Yoshua Bengio William W. Cohen Ruslan Salakhutdinov Christopher D. Manning RALM 191 2,700 0 25 Sep 2018
Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task Tao Yu Rui Zhang Kai-Chou Yang Michihiro Yasunaga Dongxu Wang ... Irene Li Qingning Yao Shanelle Roman Zilin Zhang Dragomir R. Radev RALM 116 1,245 0 24 Sep 2018
Identifying Well-formed Natural Language Questions Manaal Faruqui Dipanjan Das 96 50 0 28 Aug 2018
WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations Mohammad Taher Pilehvar Jose Camacho-Collados 199 492 0 28 Aug 2018
Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization Shashi Narayan Shay B. Cohen Mirella Lapata AILaw 149 1,684 0 27 Aug 2018
Measuring the Intrinsic Dimension of Objective Landscapes Chunyuan Li Heerad Farkhoor Rosanne Liu J. Yosinski 86 415 0 24 Apr 2018
DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension Amrita Saha Rahul Aralikatte Mitesh M. Khapra Karthik Sankaranarayanan 78 197 0 21 Apr 2018
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost Noam M. Shazeer Mitchell Stern ODL 84 1,052 0 11 Apr 2018
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge Peter Clark Isaac Cowhey Oren Etzioni Tushar Khot Ashish Sabharwal Carissa Schoenick Oyvind Tafjord ELM RALM LRM 172 2,670 0 14 Mar 2018
Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning Victor Zhong Caiming Xiong R. Socher RALM 134 1,205 0 31 Aug 2017