SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems

2 May 2019

Amanpreet Singh

Papers citing "SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems"

50 / 507 papers shown

Title
Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data Lingkai Kong Haoming Jiang Yuchen Zhuang Jie Lyu T. Zhao Chao Zhang OODD 24 26 0 22 Oct 2020
Geometry matters: Exploring language examples at the decision boundary Debajyoti Datta Shashwat Kumar Laura E. Barnes Tom Fletcher AAML 4 3 0 14 Oct 2020
SMYRF: Efficient Attention using Asymmetric Clustering Giannis Daras Nikita Kitaev Augustus Odena A. Dimakis 28 44 0 11 Oct 2020
Precise Task Formalization Matters in Winograd Schema Evaluations Haokun Liu William Huang Dhara Mungra Samuel R. Bowman ReLM 22 12 0 08 Oct 2020
On Losses for Modern Language Models Stephane Aroca-Ouellette Frank Rudzicz 14 33 0 04 Oct 2020
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention Ikuya Yamada Akari Asai Hiroyuki Shindo Hideaki Takeda Yuji Matsumoto 22 662 0 02 Oct 2020
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models Nikita Nangia Clara Vania Rasika Bhalerao Samuel R. Bowman 20 645 0 30 Sep 2020
Utility is in the Eye of the User: A Critique of NLP Leaderboards Kawin Ethayarajh Dan Jurafsky ELM 24 51 0 29 Sep 2020
DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue Shikib Mehri Mihail Eric Dilek Z. Hakkani-Tür ELM 26 136 0 28 Sep 2020
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics Swabha Swayamdipta Roy Schwartz Nicholas Lourie Yizhong Wang Hannaneh Hajishirzi Noah A. Smith Yejin Choi 44 429 0 22 Sep 2020
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data Jonathan Pilault Amine Elhattami C. Pal CLL MoE 24 89 0 19 Sep 2020
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners Timo Schick Hinrich Schütze 36 955 0 15 Sep 2020
Analysis and Evaluation of Language Models for Word Sense Disambiguation Daniel Loureiro Kiamehr Rezaee Mohammad Taher Pilehvar Jose Camacho-Collados 24 13 0 26 Aug 2020
Investigating Pretrained Language Models for Graph-to-Text Generation Leonardo F. R. Ribeiro Martin Schmitt Hinrich Schütze Iryna Gurevych 19 215 0 16 Jul 2020
Targeting the Benchmark: On Methodology in Current Natural Language Processing Research David Schlangen 25 57 0 07 Jul 2020
Data Movement Is All You Need: A Case Study on Optimizing Transformers A. Ivanov Nikoli Dryden Tal Ben-Nun Shigang Li Torsten Hoefler 36 131 0 30 Jun 2020
Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures Julien Launay Iacopo Poli Franccois Boniface Florent Krzakala 36 62 0 23 Jun 2020
Revisiting Few-sample BERT Fine-tuning Tianyi Zhang Felix Wu Arzoo Katiyar Kilian Q. Weinberger Yoav Artzi 35 441 0 10 Jun 2020
Probing Neural Dialog Models for Conversational Understanding Abdelrhman Saleh Tovly Deutsch Stephen Casper Yonatan Belinkov Stuart M. Shieber 21 13 0 07 Jun 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 64 2,622 0 05 Jun 2020
A Survey on Transfer Learning in Natural Language Processing Zaid Alyafeai Maged S. Alshaibani Irfan Ahmad 30 72 0 31 May 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 41 40,200 0 28 May 2020
NILE : Natural Language Inference with Faithful Natural Language Explanations Sawan Kumar Partha P. Talukdar XAI LRM 13 159 0 25 May 2020
Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers Anne Lauscher Olga Majewska Leonardo F. R. Ribeiro Iryna Gurevych Nikolai Rozanov Goran Glavavs KELM 31 79 0 24 May 2020
INFOTABS: Inference on Tables as Semi-structured Data Vivek Gupta Maitrey Mehta Pegah Nokhiz Vivek Srikumar LMTD 14 100 0 13 May 2020
Beyond Accuracy: Behavioral Testing of NLP models with CheckList Marco Tulio Ribeiro Tongshuang Wu Carlos Guestrin Sameer Singh ELM 28 1,081 0 08 May 2020
A Systematic Assessment of Syntactic Generalization in Neural Language Models Jennifer Hu Jon Gauthier Peng Qian Ethan Gotlieb Wilcox R. Levy ELM 35 212 0 07 May 2020
Spying on your neighbors: Fine-grained probing of contextual embeddings for information about surrounding words Josef Klafka Allyson Ettinger 51 42 0 04 May 2020
To Test Machine Comprehension, Start by Defining Comprehension Jesse Dunietz Greg Burnham Akash Bharadwaj Owen Rambow Jennifer Chu-Carroll D. Ferrucci FaML 54 65 0 04 May 2020
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations Mostafa Abdou Vinit Ravishankar Maria Barrett Yonatan Belinkov Desmond Elliott Anders Søgaard ReLM LRM 62 34 0 04 May 2020
From SPMRL to NMRL: What Did We Learn (and Unlearn) in a Decade of Parsing Morphologically-Rich Languages (MRLs)? Reut Tsarfaty Dan Bareket Stav Klein Amit Seker 16 39 0 04 May 2020
Probing Contextual Language Models for Common Ground with Visual Representations Gabriel Ilharco Rowan Zellers Ali Farhadi Hannaneh Hajishirzi 30 14 0 01 May 2020
Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning Alexandre Tamborrino Nicola Pellicanò B. Pannier Pascal Voitot Louise Naudin LRM 14 62 0 29 Apr 2020
A Review of Winograd Schema Challenge Datasets and Approaches Vid Kocijan Thomas Lukasiewicz E. Davis G. Marcus L. Morgenstern 25 43 0 23 Apr 2020
Experience Grounds Language Yonatan Bisk Ari Holtzman Jesse Thomason Jacob Andreas Yoshua Bengio ... Angeliki Lazaridou Jonathan May Aleksandr Nisnevich Nicolas Pinto Joseph P. Turian 21 351 0 21 Apr 2020
DIET: Lightweight Language Understanding for Dialogue Systems Tanja Bunk Daksh Varshneya Vladimir Vlasov Alan Nichol 27 160 0 21 Apr 2020
Are we pretraining it right? Digging deeper into visio-linguistic pretraining Amanpreet Singh Vedanuj Goswami Devi Parikh VLM 40 48 0 19 Apr 2020
CLUE: A Chinese Language Understanding Evaluation Benchmark Liang Xu Hai Hu Xuanwei Zhang Lu Li Chenjie Cao ... Cong Yue Xinrui Zhang Zhen-Yi Yang Kyle Richardson Zhenzhong Lan ELM 45 377 0 13 Apr 2020
Calibrating Structured Output Predictors for Natural Language Processing Abhyuday N. Jagannatha Hong-ye Yu 22 28 0 09 Apr 2020
Unsupervised Domain Clusters in Pretrained Language Models Roee Aharoni Yoav Goldberg 35 244 0 05 Apr 2020
XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization Junjie Hu Sebastian Ruder Aditya Siddhant Graham Neubig Orhan Firat Melvin Johnson ELM 62 955 0 24 Mar 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 243 1,452 0 18 Mar 2020
Towards Learning a Universal Non-Semantic Representation of Speech Joel Shor A. Jansen Ronnie Maor Oran Lang Omry Tuval Félix de Chaumont Quitry Marco Tagliasacchi Ira Shavitt Dotan Emanuel Yinnon A. Haviv SSL 44 155 0 25 Feb 2020
GLU Variants Improve Transformer Noam M. Shazeer 54 924 0 12 Feb 2020
Feature Importance Estimation with Self-Attention Networks Blaž Škrlj S. Džeroski Nada Lavrac Matej Petković FAtt MILM 34 51 0 11 Feb 2020
FlauBERT: Unsupervised Language Model Pre-training for French Hang Le Loïc Vial Jibril Frej Vincent Segonne Maximin Coavoux Benjamin Lecouteux A. Allauzen Benoît Crabbé Laurent Besacier D. Schwab AI4CE 40 395 0 11 Dec 2019
What do you mean, BERT? Assessing BERT as a Distributional Semantics Model Timothee Mickus Denis Paperno Mathieu Constant Kees van Deemter 23 45 0 13 Nov 2019
Generalizing Natural Language Analysis through Span-relation Representations Zhengbao Jiang Wenyuan Xu Jun Araki Graham Neubig 30 60 0 10 Nov 2019
When Choosing Plausible Alternatives, Clever Hans can be Clever Pride Kavumba Naoya Inoue Benjamin Heinzerling Keshav Singh Paul Reisert Kentaro Inui 16 50 0 01 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 94 19,493 0 23 Oct 2019