SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems

2 May 2019

Amanpreet Singh

Papers citing "SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems"

50 / 507 papers shown

Title
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark Ningyu Zhang Mosha Chen Zhen Bi Xiaozhuan Liang Lei Li ... Jun Yan Hongying Zan Kunli Zhang Buzhou Tang Qingcai Chen LM&MA ELM 34 179 0 15 Jun 2021
Improving Paraphrase Detection with the Adversarial Paraphrasing Task Animesh Nighojkar John Licato 25 39 0 14 Jun 2021
Probing Pre-Trained Language Models for Disease Knowledge Israa Alghanmi Luis Espinosa-Anke Steven Schockaert LM&MA ELM 27 13 0 14 Jun 2021
Schema-Guided Paradigm for Zero-Shot Dialog Shikib Mehri M. Eskénazi 24 16 0 13 Jun 2021
Can Transformer Language Models Predict Psychometric Properties? Antonio Laverghetta Animesh Nighojkar Jamshidbek Mirzakhalov John Licato LM&MA 38 14 0 12 Jun 2021
Prompting Contrastive Explanations for Commonsense Reasoning Tasks Bhargavi Paranjape Julian Michael Marjan Ghazvininejad Luke Zettlemoyer Hannaneh Hajishirzi ReLM LRM 22 66 0 12 Jun 2021
A Discussion on Building Practical NLP Leaderboards: The Case of Machine Translation Sebastin Santy Prasanta Bhattacharya LLMAG 33 2 0 11 Jun 2021
Generate, Annotate, and Learn: NLP with Synthetic Text Xuanli He Islam Nassar J. Kiros Gholamreza Haffari Mohammad Norouzi 39 51 0 11 Jun 2021
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers Rabeeh Karimi Mahabadi James Henderson Sebastian Ruder MoE 67 468 0 08 Jun 2021
Reordering Examples Helps during Priming-based Few-Shot Learning Sawan Kumar Partha P. Talukdar 18 58 0 03 Jun 2021
SyGNS: A Systematic Generalization Testbed Based on Natural Language Semantics Hitomi Yanaka K. Mineshima Kentaro Inui NAI AI4CE 38 11 0 02 Jun 2021
LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond Daniel Loureiro A. Jorge Jose Camacho-Collados 33 26 0 26 May 2021
DaLAJ - a dataset for linguistic acceptability judgments for Swedish: Format, baseline, sharing Elena Volodina Yousuf Ali Mohammed Julia Klezl 11 21 0 14 May 2021
Designing Multimodal Datasets for NLP Challenges James Pustejovsky E. Holderness Jingxuan Tu Parker Glenn Kyeongmin Rim Kelley Lynch R. Brutti 23 5 0 12 May 2021
Entailment as Few-Shot Learner Sinong Wang Han Fang Madian Khabsa Hanzi Mao Hao Ma 35 183 0 29 Apr 2021
MOROCCO: Model Resource Comparison Framework Valentin Malykh Alexander Kukushkin Ekaterina Artemova Vladislav Mikhailov Maria Tikhonova Tatiana Shavrina 21 0 0 29 Apr 2021
Cross-Task Generalization via Natural Language Crowdsourcing Instructions Swaroop Mishra Daniel Khashabi Chitta Baral Hannaneh Hajishirzi LRM 28 718 0 18 Apr 2021
Membership Inference Attack Susceptibility of Clinical Language Models Abhyuday N. Jagannatha Bhanu Pratap Singh Rawat Hong-ye Yu MIACV 29 61 0 16 Apr 2021
AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models Katikapalli Subramanyam Kalyan A. Rajasekharan S. Sangeetha LM&MA MedIm 26 164 0 16 Apr 2021
Language Models are Few-Shot Butlers Vincent Micheli Franccois Fleuret 25 31 0 16 Apr 2021
XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation Sebastian Ruder Noah Constant Jan A. Botha Aditya Siddhant Orhan Firat ... Pengfei Liu Junjie Hu Dan Garrette Graham Neubig Melvin Johnson ELM AAML LRM 24 184 0 15 Apr 2021
ExplainaBoard: An Explainable Leaderboard for NLP Pengfei Liu Jinlan Fu Yanghua Xiao Weizhe Yuan Shuaichen Chang Junqi Dai Yixin Liu Zihuiwen Ye Zi-Yi Dou Graham Neubig XAI LRM ELM 28 54 0 13 Apr 2021
Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages Gowtham Ramesh Sumanth Doddapaneni Aravinth Bheemaraj Mayank Jobanputra AK Raghavan ... K. Deepak Vivek Raghavan Anoop Kunchukuttan Pratyush Kumar Mitesh Khapra LRM 37 229 0 12 Apr 2021
NLI Data Sanity Check: Assessing the Effect of Data Corruption on Model Performance Aarne Talman Marianna Apidianaki S. Chatzikyriakidis Jörg Tiedemann 30 10 0 10 Apr 2021
What Will it Take to Fix Benchmarking in Natural Language Understanding? Samuel R. Bowman George E. Dahl ELM ALM 30 156 0 05 Apr 2021
EnergyVis: Interactively Tracking and Exploring Energy Consumption for ML Models Omar Shaikh Jon Saad-Falcon Austin P. Wright Nilaksh Das Scott Freitas O. Asensio Duen Horng Chau 27 18 0 30 Mar 2021
FastMoE: A Fast Mixture-of-Expert Training System Jiaao He J. Qiu Aohan Zeng Zhilin Yang Jidong Zhai Jie Tang ALM MoE 30 94 0 24 Mar 2021
Representing Numbers in NLP: a Survey and a Vision Avijit Thawani Jay Pujara Pedro A. Szekely Filip Ilievski 32 114 0 24 Mar 2021
UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark Nicholas Lourie Ronan Le Bras Chandra Bhagavatula Yejin Choi LRM 30 137 0 24 Mar 2021
The NLP Cookbook: Modern Recipes for Transformer based Deep Learning Architectures Sushant Singh A. Mahmood AI4TS 60 92 0 23 Mar 2021
Improving and Simplifying Pattern Exploiting Training Derek Tam Rakesh R Menon Joey Tianyi Zhou Shashank Srivastava Colin Raffel 18 149 0 22 Mar 2021
GPT Understands, Too Xiao Liu Yanan Zheng Zhengxiao Du Ming Ding Yujie Qian Zhilin Yang Jie Tang VLM 66 1,144 0 18 Mar 2021
Structural Adapters in Pretrained Language Models for AMR-to-text Generation Leonardo F. R. Ribeiro Yue Zhang Iryna Gurevych 41 69 0 16 Mar 2021
How Many Data Points is a Prompt Worth? Teven Le Scao Alexander M. Rush VLM 66 296 0 15 Mar 2021
CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review Dan Hendrycks Collin Burns Anya Chen Spencer Ball ELM AILaw 23 180 0 10 Mar 2021
Measuring Mathematical Problem Solving With the MATH Dataset Dan Hendrycks Collin Burns Saurav Kadavath Akul Arora Steven Basart Eric Tang D. Song Jacob Steinhardt ReLM FaML 84 1,840 0 05 Mar 2021
Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development Kexin Huang Tianfan Fu Wenhao Gao Yue Zhao Yusuf Roohani J. Leskovec Connor W. Coley Cao Xiao Jimeng Sun Marinka Zitnik OOD LM&MA 33 263 0 18 Feb 2021
A Metamodel and Framework for Artificial General Intelligence From Theory to Practice Hugo Latapie Özkan Kiliç Gaowen Liu Yan Yan Ramana Rao Kompella Pei Wang K. Thórisson Adam Lawrence Yuhong Sun Jayanth Srinivasa AI4CE 22 9 0 11 Feb 2021
Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models Hannah Rose Kirk Yennie Jun Haider Iqbal Elias Benussi Filippo Volpin F. Dreyer Aleksandar Shtedritski Yuki M. Asano 19 179 0 08 Feb 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics Sebastian Gehrmann Tosin P. Adewumi Karmanya Aggarwal Pawan Sasanka Ammanamanchi Aremu Anuoluwapo ... Nishant Subramani Wei-ping Xu Diyi Yang Akhila Yerukola Jiawei Zhou VLM 260 285 0 02 Feb 2021
Muppet: Massive Multi-task Representations with Pre-Finetuning Armen Aghajanyan Anchit Gupta Akshat Shrivastava Xilun Chen Luke Zettlemoyer Sonal Gupta 33 266 0 26 Jan 2021
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity W. Fedus Barret Zoph Noam M. Shazeer MoE 11 2,075 0 11 Jan 2021
WARP: Word-level Adversarial ReProgramming Karen Hambardzumyan Hrant Khachatrian Jonathan May AAML 254 342 0 01 Jan 2021
Out of Order: How Important Is The Sequential Order of Words in a Sentence in Natural Language Understanding Tasks? Thang M. Pham Trung Bui Long Mai Anh Totti Nguyen 217 122 0 30 Dec 2020
Seeing past words: Testing the cross-modal capabilities of pretrained V&L models on counting tasks Letitia Parcalabescu Albert Gatt Anette Frank Iacer Calixto LRM 33 48 0 22 Dec 2020
Exploring Fluent Query Reformulations with Text-to-Text Transformers and Reinforcement Learning Jerry Zikun Chen S. Yu Haoran Wang 163 5 0 18 Dec 2020
WILDS: A Benchmark of in-the-Wild Distribution Shifts Pang Wei Koh Shiori Sagawa Henrik Marklund Sang Michael Xie Marvin Zhang ... A. Kundaje Emma Pierson Sergey Levine Chelsea Finn Percy Liang OOD 71 1,377 0 14 Dec 2020
GLGE: A New General Language Generation Evaluation Benchmark Dayiheng Liu Yu Yan Yeyun Gong Weizhen Qi Hang Zhang ... Jiancheng Lv Ruofei Zhang Winnie Wu Ming Zhou Nan Duan ELM 40 66 0 24 Nov 2020
Learn to Bind and Grow Neural Structures Azhar Shaikh Nishant Sinha CLL 21 0 0 21 Nov 2020
When Do You Need Billions of Words of Pretraining Data? Yian Zhang Alex Warstadt Haau-Sing Li Samuel R. Bowman 29 136 0 10 Nov 2020