v1v2 (latest)

What to Pre-Train on? Efficient Intermediate Task Selection

16 April 2021

Papers citing "What to Pre-Train on? Efficient Intermediate Task Selection"

50 / 62 papers shown

Title
Model-Based Transfer Learning for Contextual Reinforcement Learning Jung-Hoon Cho Vindula Jayawardana Sirui Li Cathy Wu OffRL 143 0 0 08 Aug 2024
A linearized framework and a new benchmark for model selection for fine-tuning Aditya Deshpande Alessandro Achille Avinash Ravichandran Hao Li Luca Zancato Charless C. Fowlkes Rahul Bhotika Stefano Soatto Pietro Perona ALM 157 47 0 29 Jan 2021
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models Phillip Rust Jonas Pfeiffer Ivan Vulić Sebastian Ruder Iryna Gurevych 128 254 0 31 Dec 2020
UNKs Everywhere: Adapting Multilingual Language Models to New Scripts Jonas Pfeiffer Ivan Vulić Iryna Gurevych Sebastian Ruder 78 133 0 31 Dec 2020
Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer M. Vidoni Ivan Vulić Goran Glavaš 79 27 0 11 Dec 2020
Ranking Neural Checkpoints Yandong Li Xuhui Jia Ruoxin Sang Yukun Zhu Bradley Green Liqiang Wang Boqing Gong FedML ELM UQCV 62 48 0 23 Nov 2020
AdapterDrop: On the Efficiency of Adapters in Transformers Andreas Rucklé Gregor Geigle Max Glockner Tilman Beck Jonas Pfeiffer Nils Reimers Iryna Gurevych 121 262 0 22 Oct 2020
Which Model to Transfer? Finding the Needle in the Growing Haystack Cédric Renggli André Susano Pinto Luka Rimanic J. Puigcerver C. Riquelme Ce Zhang Mario Lucic 88 27 0 13 Oct 2020
MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale Andreas Rucklé Jonas Pfeiffer Iryna Gurevych 66 37 0 02 Oct 2020
Scalable Transfer Learning with Expert Models J. Puigcerver C. Riquelme Basil Mustafa Cédric Renggli André Susano Pinto Sylvain Gelly Daniel Keysers N. Houlsby 123 62 0 28 Sep 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 795 42,055 0 28 May 2020
Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work? Yada Pruksachatkun Jason Phang Haokun Liu Phu Mon Htut Xiaoyi Zhang Richard Yuanzhe Pang Clara Vania Katharina Kann Samuel R. Bowman CLL LRM 62 197 0 01 May 2020
AdapterFusion: Non-Destructive Task Composition for Transfer Learning Jonas Pfeiffer Aishwarya Kamath Andreas Rucklé Kyunghyun Cho Iryna Gurevych CLL MoMe 136 852 0 01 May 2020
MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer Jonas Pfeiffer Ivan Vulić Iryna Gurevych Sebastian Ruder 103 627 0 30 Apr 2020
UDapter: Language Adaptation for Truly Universal Dependency Parsing Ahmet Üstün Arianna Bisazza G. Bouma Gertjan van Noord 56 116 0 29 Apr 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 152 2,428 0 23 Apr 2020
The State and Fate of Linguistic Diversity and Inclusion in the NLP World Pratik M. Joshi Sebastin Santy A. Budhiraja Kalika Bali Monojit Choudhury LMTD 117 850 0 20 Apr 2020
LEEP: A New Measure to Evaluate Transferability of Learned Representations Cuong V Nguyen Tal Hassner Matthias Seeger Cédric Archambeau 77 217 0 27 Feb 2020
Decision-Making with Auto-Encoding Variational Bayes Romain Lopez Pierre Boyeau Nir Yosef Michael I. Jordan Jeffrey Regier BDL 407 10,591 0 17 Feb 2020
Adversarial NLI: A New Benchmark for Natural Language Understanding Yixin Nie Adina Williams Emily Dinan Joey Tianyi Zhou Jason Weston Douwe Kiela 125 1,006 0 31 Oct 2019
QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questions Oyvind Tafjord Matt Gardner Kevin Lin Peter Clark 59 108 0 08 Sep 2019
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning Lifu Huang Ronan Le Bras Chandra Bhagavatula Yejin Choi AIMat RALM LRM 112 454 0 31 Aug 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 1.3K 12,226 0 27 Aug 2019
Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning Pradeep Dasigi Nelson F. Liu Ana Marasović Noah A. Smith Matt Gardner RALM 73 173 0 16 Aug 2019
Abductive Commonsense Reasoning Chandra Bhagavatula Ronan Le Bras Chaitanya Malaviya Keisuke Sakaguchi Ari Holtzman Hannah Rashkin Doug Downey Scott Yih Yejin Choi ReLM LRM 78 461 0 15 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 659 24,464 0 26 Jul 2019
Energy and Policy Considerations for Deep Learning in NLP Emma Strubell Ananya Ganesh Andrew McCallum 69 2,657 0 05 Jun 2019
MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension Alon Talmor Jonathan Berant 81 173 0 31 May 2019
Dataset2Vec: Learning Dataset Meta-Features H. Jomaa Lars Schmidt-Thieme Josif Grabocka SSL 84 62 0 27 May 2019
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions Christopher Clark Kenton Lee Ming-Wei Chang Tom Kwiatkowski Michael Collins Kristina Toutanova 224 1,527 0 24 May 2019
HellaSwag: Can a Machine Really Finish Your Sentence? Rowan Zellers Ari Holtzman Yonatan Bisk Ali Farhadi Yejin Choi 174 2,485 0 19 May 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 265 2,315 0 02 May 2019
Structural Scaffolds for Citation Intent Classification in Scientific Publications Arman Cohan Bridger Waleed Ammar Madeleine van Zuylen Field Cady 52 252 0 02 Apr 2019
Linguistic Knowledge and Transferability of Contextual Representations Nelson F. Liu Matt Gardner Yonatan Belinkov Matthew E. Peters Noah A. Smith 130 733 0 21 Mar 2019
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs Dheeru Dua Yizhong Wang Pradeep Dasigi Gabriel Stanovsky Sameer Singh Matt Gardner AIMat 98 955 0 01 Mar 2019
Task2Vec: Task Embedding for Meta-Learning Alessandro Achille Michael Lam Rahul Tewari Avinash Ravichandran Subhransu Maji Charless C. Fowlkes Stefano Soatto Pietro Perona SSL 77 315 0 10 Feb 2019
Parameter-Efficient Transfer Learning for NLP N. Houlsby A. Giurgiu Stanislaw Jastrzebski Bruna Morrone Quentin de Laroussilhe Andrea Gesmundo Mona Attariyan Sylvain Gelly 210 4,460 0 02 Feb 2019
Learning and Evaluating General Linguistic Intelligence Dani Yogatama Cyprien de Masson dÁutume Jerome T. Connor Tomás Kociský Mike Chrzanowski ... Angeliki Lazaridou Wang Ling Lei Yu Chris Dyer Phil Blunsom ELM AI4CE 156 210 0 31 Jan 2019
Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks Jason Phang Thibault Févry Samuel R. Bowman 94 468 0 02 Nov 2018
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge Alon Talmor Jonathan Herzig Nicholas Lourie Jonathan Berant RALM 140 1,733 0 02 Nov 2018
ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension Sheng Zhang Xiaodong Liu Jingjing Liu Jianfeng Gao Kevin Duh Benjamin Van Durme 74 314 0 30 Oct 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 94,891 0 11 Oct 2018
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Zhilin Yang Peng Qi Saizheng Zhang Yoshua Bengio William W. Cohen Ruslan Salakhutdinov Christopher D. Manning RALM 171 2,655 0 25 Sep 2018
WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations Mohammad Taher Pilehvar Jose Camacho-Collados 195 489 0 28 Aug 2018
SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference Rowan Zellers Yonatan Bisk Roy Schwartz Yejin Choi 104 718 0 16 Aug 2018
Know What You Don't Know: Unanswerable Questions for SQuAD Pranav Rajpurkar Robin Jia Percy Liang RALM ELM 279 2,845 0 11 Jun 2018
Neural Network Acceptability Judgments Alex Warstadt Amanpreet Singh Samuel R. Bowman 230 1,407 0 31 May 2018
Taskonomy: Disentangling Task Transfer Learning Amir Zamir Alexander Sax Bokui (William) Shen Leonidas Guibas Jitendra Malik Silvio Savarese 120 1,220 0 23 Apr 2018
DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension Amrita Saha Rahul Aralikatte Mitesh M. Khapra Karthik Sankaranarayanan 78 197 0 21 Apr 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,159 0 20 Apr 2018