Adversarially Constructed Evaluation Sets Are More Challenging, but May
Not Be Fair

Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair

16 November 2021

Samuel R. Bowman

Papers citing "Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair"

17 / 17 papers shown

Title
Improving Model Evaluation using SMART Filtering of Benchmark Datasets Vipul Gupta Candace Ross David Pantoja R. Passonneau Megan Ung Adina Williams 240 2 0 26 Oct 2024
Comparing Test Sets with Item Response Theory Clara Vania Phu Mon Htut William Huang Dhara Mungra Richard Yuanzhe Pang Jason Phang Haokun Liu Kyunghyun Cho Sam Bowman 56 42 0 01 Jun 2021
Dynabench: Rethinking Benchmarking in NLP Douwe Kiela Max Bartolo Yixin Nie Divyansh Kaushik Atticus Geiger ... Pontus Stenetorp Robin Jia Joey Tianyi Zhou Christopher Potts Adina Williams 198 407 0 07 Apr 2021
ANLIzing the Adversarial Natural Language Inference Dataset Adina Williams Tristan Thrush Douwe Kiela AAML 214 47 0 24 Oct 2020
What Can We Learn from Collective Human Opinions on Natural Language Inference Data? Yixin Nie Xiang Zhou Joey Tianyi Zhou 79 138 0 07 Oct 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 137 2,731 0 05 Jun 2020
Adversarial Filters of Dataset Biases Ronan Le Bras Swabha Swayamdipta Chandra Bhagavatula Rowan Zellers Matthew E. Peters Ashish Sabharwal Yejin Choi 92 222 0 10 Feb 2020
Unsupervised Cross-lingual Representation Learning at Scale Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzmán Edouard Grave Myle Ott Luke Zettlemoyer Veselin Stoyanov 212 6,555 0 05 Nov 2019
Adversarial NLI: A New Benchmark for Natural Language Understanding Yixin Nie Adina Williams Emily Dinan Joey Tianyi Zhou Jason Weston Douwe Kiela 118 1,005 0 31 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 356 6,449 0 26 Sep 2019
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning Lifu Huang Ronan Le Bras Chandra Bhagavatula Yejin Choi AIMat RALM LRM 106 454 0 31 Aug 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 256 2,312 0 02 May 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,154 0 20 Apr 2018
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 520 4,476 0 18 Apr 2017
Bidirectional Attention Flow for Machine Comprehension Minjoon Seo Aniruddha Kembhavi Ali Farhadi Hannaneh Hajishirzi 131 2,090 0 05 Nov 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 274 8,127 0 16 Jun 2016
A large annotated corpus for learning natural language inference Samuel R. Bowman Gabor Angeli Christopher Potts Christopher D. Manning 310 4,284 0 21 Aug 2015