How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks

14 August 2018

Papers citing "How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks"

50 / 53 papers shown

Title
Task Calibration: Calibrating Large Language Models on Inference Tasks Yingjie Li Yun Luo Xiaotian Xie Yue Zhang LRM 21 0 0 24 Oct 2024
Desiderata for the Context Use of Question Answering Systems Sagi Shaier Lawrence E Hunter K. Wense 28 4 0 31 Jan 2024
Measuring and Improving Attentiveness to Partial Inputs with Counterfactuals Yanai Elazar Bhargavi Paranjape Hao Peng Sarah Wiegreffe Khyathi Raghavi Vivek Srikumar Sameer Singh Noah A. Smith AAML OOD 34 0 0 16 Nov 2023
Analyzing Multiple-Choice Reading and Listening Comprehension Tests Vatsal Raina Adian Liusie Mark Gales ELM 43 2 0 03 Jul 2023
Out-of-Distribution Generalization in Text Classification: Past, Present, and Future Linyi Yang Yangqiu Song Xuan Ren Chenyang Lyu Yidong Wang Lingqiao Liu Jindong Wang Jennifer Foster Yue Zhang OOD 37 2 0 23 May 2023
What's the Meaning of Superhuman Performance in Today's NLU? Simone Tedeschi Johan Bos T. Declerck Jan Hajic Daniel Hershcovich ... Simon Krek Steven Schockaert Rico Sennrich Ekaterina Shutova Roberto Navigli ELM LM&MA VLM ReLM LRM 39 26 0 15 May 2023
SkillQG: Learning to Generate Question for Reading Comprehension Assessment Xiaoqiang Wang Bang Liu Siliang Tang Lingfei Wu 25 3 0 08 May 2023
DISCO: Distilling Counterfactuals with Large Language Models Zeming Chen Qiyue Gao Antoine Bosselut Ashish Sabharwal Kyle Richardson 37 25 0 20 Dec 2022
Evaluation for Change Rishi Bommasani ELM 40 0 0 20 Dec 2022
Feature-Level Debiased Natural Language Understanding Yougang Lyu Piji Li Yechang Yang Maarten de Rijke Pengjie Ren Yukun Zhao Dawei Yin Z. Ren 32 10 0 11 Dec 2022
GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective Linyi Yang Shuibai Zhang Libo Qin Yafu Li Yidong Wang Hanmeng Liu Jindong Wang Xingxu Xie Yue Zhang ELM 46 79 0 15 Nov 2022
CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation Abhilasha Ravichander Matt Gardner Ana Marasović 33 34 0 01 Nov 2022
On Feature Learning in the Presence of Spurious Correlations Pavel Izmailov Polina Kirichenko Nate Gruver A. Wilson 36 118 0 20 Oct 2022
State-of-the-art generalisation research in NLP: A taxonomy and review Dieuwke Hupkes Mario Giulianelli Verna Dankers Mikel Artetxe Yanai Elazar ... Leila Khalatbari Maria Ryskina Rita Frieske Ryan Cotterell Zhijing Jin 127 94 0 06 Oct 2022
Understanding Prior Bias and Choice Paralysis in Transformer-based Language Representation Models through Four Experimental Probes Ke Shen Mayank Kejriwal 27 4 0 03 Oct 2022
A Survey on Measuring and Mitigating Reasoning Shortcuts in Machine Reading Comprehension Xanh Ho Johannes Mario Meissner Saku Sugawara Akiko Aizawa OffRL 35 4 0 05 Sep 2022
MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Data Yilun Zhao Yunxiang Li Chenying Li Rui Zhang AIMat 39 97 0 03 Jun 2022
What Makes Reading Comprehension Questions Difficult? Saku Sugawara Nikita Nangia Alex Warstadt Sam Bowman ELM RALM 20 13 0 12 Mar 2022
Dyna-bAbI: unlocking bAbI's potential with dynamic synthetic benchmarking Ronen Tamari Kyle Richardson Aviad Sar-Shalom Noam Kahlon Nelson F. Liu Reut Tsarfaty Dafna Shahaf 43 5 0 30 Nov 2021
Attacking Open-domain Question Answering by Injecting Misinformation Liangming Pan Wenhu Chen Min-Yen Kan Wei Wang HILM AAML 206 22 0 15 Oct 2021
Text-based NP Enrichment Yanai Elazar Victoria Basmov Yoav Goldberg Reut Tsarfaty 60 15 0 24 Sep 2021
ParaShoot: A Hebrew Question Answering Dataset Omri Keren Omer Levy 37 17 0 23 Sep 2021
Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning Prasetya Ajie Utama N. Moosavi Victor Sanh Iryna Gurevych AAML 61 35 0 09 Sep 2021
MuSiQue: Multihop Questions via Single-hop Question Composition H. Trivedi Niranjan Balasubramanian Tushar Khot Ashish Sabharwal LRM 20 229 0 02 Aug 2021
On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study Divyansh Kaushik Douwe Kiela Zachary Chase Lipton Wen-tau Yih AAML 11 36 0 02 Jun 2021
Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema Yanai Elazar Hongming Zhang Yoav Goldberg Dan Roth ReLM LRM 45 44 0 16 Apr 2021
MultiModalQA: Complex Question Answering over Text, Tables and Images Alon Talmor Ori Yoran Amnon Catav Dan Lahav Yizhong Wang Akari Asai Gabriel Ilharco Hannaneh Hajishirzi Jonathan Berant LMTD 32 149 0 13 Apr 2021
DynaSent: A Dynamic Benchmark for Sentiment Analysis Christopher Potts Zhengxuan Wu Atticus Geiger Douwe Kiela 230 77 0 30 Dec 2020
Out of Order: How Important Is The Sequential Order of Words in a Sentence in Natural Language Understanding Tasks? Thang M. Pham Trung Bui Long Mai Anh Totti Nguyen 220 122 0 30 Dec 2020
Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrieval Akari Asai Eunsol Choi RALM 45 51 0 22 Oct 2020
Geometry matters: Exploring language examples at the decision boundary Debajyoti Datta Shashwat Kumar Laura E. Barnes Tom Fletcher AAML 9 3 0 14 Oct 2020
Counterfactual Variable Control for Robust and Interpretable Question Answering S. Yu Yulei Niu Shuohang Wang Jing Jiang Qianru Sun AAML OOD 42 9 0 12 Oct 2020
A Survey on Explainability in Machine Reading Comprehension Mokanarangan Thayaparan Marco Valentino André Freitas FaML 12 50 0 01 Oct 2020
Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets Patrick Lewis Pontus Stenetorp Sebastian Riedel OOD ELM 18 184 0 06 Aug 2020
Learning Reasoning Strategies in End-to-End Differentiable Proving Pasquale Minervini Sebastian Riedel Pontus Stenetorp Edward Grefenstette Tim Rocktaschel LRM 45 96 0 13 Jul 2020
To Test Machine Comprehension, Start by Defining Comprehension Jesse Dunietz Greg Burnham Akash Bharadwaj Owen Rambow Jennifer Chu-Carroll D. Ferrucci FaML 54 65 0 04 May 2020
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations Mostafa Abdou Vinit Ravishankar Maria Barrett Yonatan Belinkov Desmond Elliott Anders Søgaard ReLM LRM 62 34 0 04 May 2020
DQI: Measuring Data Quality in NLP Swaroop Mishra Anjana Arunkumar Bhavdeep Singh Sachdeva Chris Bryan Chitta Baral 36 30 0 02 May 2020
HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data Wenhu Chen Hanwen Zha Zhiyu Zoey Chen Wenhan Xiong Hong Wang Wei Wang 38 293 0 15 Apr 2020
Translation Artifacts in Cross-lingual Transfer Learning Mikel Artetxe Gorka Labaka Eneko Agirre 27 115 0 09 Apr 2020
Evaluating Models' Local Decision Boundaries via Contrast Sets Matt Gardner Yoav Artzi Victoria Basmova Jonathan Berant Ben Bogin ... Sanjay Subramanian Reut Tsarfaty Eric Wallace Ally Zhang Ben Zhou ELM 43 84 0 06 Apr 2020
Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension Max Bartolo A. Roberts Johannes Welbl Sebastian Riedel Pontus Stenetorp AAML 28 167 0 02 Feb 2020
What Question Answering can Learn from Trivia Nerds Jordan L. Boyd-Graber Benjamin Borschinger 24 36 0 31 Oct 2019
On the Cross-lingual Transferability of Monolingual Representations Mikel Artetxe Sebastian Ruder Dani Yogatama 28 774 0 25 Oct 2019
Learning the Difference that Makes a Difference with Counterfactually-Augmented Data Divyansh Kaushik Eduard H. Hovy Zachary Chase Lipton CML 28 560 0 26 Sep 2019
An Empirical Study of Content Understanding in Conversational Question Answering Ting-Rui Chiang Hao-Tong Ye Yun-Nung (Vivian) Chen ELM 31 8 0 24 Sep 2019
Neural Text Summarization: A Critical Evaluation Wojciech Kry'sciñski N. Keskar Bryan McCann Caiming Xiong R. Socher 22 361 0 23 Aug 2019
Towards Debiasing Fact Verification Models Tal Schuster Darsh J. Shah Yun Jie Serene Yeo Daniel Filizzola Enrico Santus Regina Barzilay 36 209 0 14 Aug 2019
AmazonQA: A Review-Based Question Answering Task Mansi Gupta Nitish Kulkarni Raghuveer Chanda Anirudha Rayasam Zachary Chase Lipton RALM 19 67 0 12 Aug 2019
Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference Yonatan Belinkov Adam Poliak Stuart M. Shieber Benjamin Van Durme Alexander M. Rush 27 94 0 09 Jul 2019