Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets

6 August 2020

Patrick Lewis

Papers citing "Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets"

50 / 132 papers shown

Title
A Survey on Measuring and Mitigating Reasoning Shortcuts in Machine Reading Comprehension Xanh Ho Johannes Mario Meissner Saku Sugawara Akiko Aizawa OffRL 35 4 0 05 Sep 2022
Investigating data partitioning strategies for crosslinguistic low-resource ASR evaluation Zoey Liu J. Spence Emily Tucker Prudhommeaux 32 8 0 26 Aug 2022
Towards Generating Robust, Fair, and Emotion-Aware Explanations for Recommender Systems Bingbing Wen Yunhe Feng Yongfeng Zhang Chirag Shah 19 1 0 17 Aug 2022
MIA 2022 Shared Task: Evaluating Cross-lingual Open-Retrieval Question Answering for 16 Diverse Languages Akari Asai Shayne Longpre Jungo Kasai Chia-Hsuan Lee Rui Zhang Junjie Hu Ikuya Yamada J. Clark Eunsol Choi LRM 19 14 0 02 Jul 2022
An Open-Domain QA System for e-Governance Radu Ion Andrei-Marius Avram Vasile Puaics Maria Mitrofan V. Mititelu Elena Irimia Valentin Badea 11 1 0 16 Jun 2022
Optimizing Test-Time Query Representations for Dense Retrieval Mujeen Sung Jungsoo Park Jaewoo Kang Danqi Chen Jinhyuk Lee 33 5 0 25 May 2022
QAMPARI: An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphs S. Amouyal Tomer Wolfson Ohad Rubin Ori Yoran Jonathan Herzig Jonathan Berant RALM VLM 33 21 0 25 May 2022
Policy Compliance Detection via Expression Tree Inference Neema Kotonya Andreas Vlachos Majid Yazdani Lambert Mathias Marzieh Saeidi 27 2 0 24 May 2022
Two-Step Question Retrieval for Open-Domain QA Yeon Seonwoo Juhee Son Jiho Jin Sang-Woo Lee Ji-Hoon Kim Jung-Woo Ha Alice Oh RALM LRM 40 5 0 19 May 2022
Asking for Knowledge: Training RL Agents to Query External Knowledge Using Language Iou-Jen Liu Xingdi Yuan Marc-Alexandre Côté Pierre-Yves Oudeyer A. Schwing RALM 19 12 0 12 May 2022
TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models Joel Jang Seonghyeon Ye Changho Lee Sohee Yang Joongbo Shin Janghoon Han Gyeonghun Kim Minjoon Seo CLL KELM 27 92 0 29 Apr 2022
Entity-Conditioned Question Generation for Robust Attention Distribution in Neural Information Retrieval R. Reddy Md Arafat Sultan M. Franz Avirup Sil Heng Ji 32 7 0 24 Apr 2022
Autoregressive Search Engines: Generating Substrings as Document Identifiers Michele Bevilacqua G. Ottaviano Patrick Lewis Wen-tau Yih Sebastian Riedel Fabio Petroni KELM RALM 30 156 0 22 Apr 2022
The Risks of Machine Learning Systems Samson Tan Araz Taeihagh K. Baxter 12 5 0 21 Apr 2022
Knowledgeable Salient Span Mask for Enhancing Language Models as Knowledge Base Cunxiang Wang Fuli Luo Yanyang Li Runxin Xu Fei Huang Yue Zhang KELM 36 2 0 17 Apr 2022
Towards Fine-grained Causal Reasoning and QA Linyi Yang Zhen Wang Yuxiang Wu Jie Yang Yue Zhang 41 15 0 15 Apr 2022
Towards Generalizable Semantic Product Search by Text Similarity Pre-training on Search Click Logs Zheng-Wang Liu Wei Zhang Yan Chen Weiyi Sun Tianchuan Du Benjamin Schroeder 34 3 0 11 Apr 2022
Entity-driven Fact-aware Abstractive Summarization of Biomedical Literature Amanuel Alambo Tanvi Banerjee K. Thirunarayan M. Raymer MedIm 18 7 0 30 Mar 2022
An Empirical Study of Memorization in NLP Xiaosen Zheng Jing Jiang TDI 17 24 1 23 Mar 2022
Teaching language models to support answers with verified quotes Jacob Menick Maja Trebacz Vladimir Mikulik John Aslanides Francis Song ... Mia Glaese Susannah Young Lucy Campbell-Gillingham G. Irving Nat McAleese ELM RALM 246 259 0 21 Mar 2022
DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Engine Yifu Qiu Hongyu Li Yingqi Qu Ying-Cong Chen Qiaoqiao She Jing Liu Hua Wu Haifeng Wang RALM 38 15 0 19 Mar 2022
Iteratively Prompt Pre-trained Language Models for Chain of Thought Boshi Wang Xiang Deng Huan Sun KELM ReLM LRM 29 95 0 16 Mar 2022
Trajectory Test-Train Overlap in Next-Location Prediction Datasets Massimiliano Luca Luca Pappalardo Bruno Lepri Gianni Barlacchi 30 5 0 07 Mar 2022
Impact of Pretraining Term Frequencies on Few-Shot Reasoning Yasaman Razeghi Robert L Logan IV Matt Gardner Sameer Singh ReLM LRM 32 150 0 15 Feb 2022
TimeLMs: Diachronic Language Models from Twitter Daniel Loureiro Francesco Barbieri Leonardo Neves Luis Espinosa Anke Jose Camacho-Collados 41 249 0 08 Feb 2022
Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classification Takashi Ishida Ikko Yamane Nontawat Charoenphakdee Gang Niu Masashi Sugiyama BDL UQCV 48 14 0 01 Feb 2022
Artefact Retrieval: Overview of NLP Models with Knowledge Base Access Vilém Zouhar Marius Mosbach Debanjali Biswas Dietrich Klakow KELM 27 4 0 24 Jan 2022
Improving Biomedical Information Retrieval with Neural Retrievers Man Luo Arindam Mitra Tejas Gokhale Chitta Baral 35 33 0 19 Jan 2022
WebGPT: Browser-assisted question-answering with human feedback Reiichiro Nakano Jacob Hilton S. Balaji Jeff Wu Ouyang Long ... Gretchen Krueger Kevin Button Matthew Knight B. Chess John Schulman ALM RALM 108 1,202 0 17 Dec 2021
Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks Akari Asai Matt Gardner Hannaneh Hajishirzi RALM 21 45 0 16 Dec 2021
Measure and Improve Robustness in NLP Models: A Survey Xuezhi Wang Haohan Wang Diyi Yang 139 130 0 15 Dec 2021
Decomposing Natural Logic Inferences in Neural NLI Julia Rozanova Deborah Ferreira Marco Valentino Mokanrarangan Thayaparan André Freitas NAI LRM 17 4 0 15 Dec 2021
Learning to Retrieve Passages without Supervision Ori Ram Gal Shachaf Omer Levy Jonathan Berant Amir Globerson RALM 11 61 0 14 Dec 2021
Improving language models by retrieving from trillions of tokens Sebastian Borgeaud A. Mensch Jordan Hoffmann Trevor Cai Eliza Rutherford ... Simon Osindero Karen Simonyan Jack W. Rae Erich Elsen Laurent Sifre KELM RALM 90 1,024 0 08 Dec 2021
Ethical and social risks of harm from Language Models Laura Weidinger John F. J. Mellor Maribeth Rauh Conor Griffin J. Uesato ... Lisa Anne Hendricks William S. Isaac Sean Legassick G. Irving Iason Gabriel PILM 35 982 0 08 Dec 2021
Hey AI, Can You Solve Complex Tasks by Talking to Agents? Tushar Khot Kyle Richardson Daniel Khashabi Ashish Sabharwal RALM LRM 13 14 0 16 Oct 2021
CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training Patrick Huber Armen Aghajanyan Barlas Oğuz Dmytro Okhonko Wen-tau Yih Sonal Gupta Xilun Chen 43 15 0 14 Oct 2021
Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One? Xilun Chen Kushal Lakhotia Barlas Oğuz Anchit Gupta Patrick Lewis Stanislav Peshterliev Yashar Mehdad Sonal Gupta Wen-tau Yih 56 68 0 13 Oct 2021
Attention-guided Generative Models for Extractive Question Answering Peng Xu Davis Liang Zhiheng Huang Bing Xiang 38 18 0 12 Oct 2021
A Few More Examples May Be Worth Billions of Parameters Yuval Kirstain Patrick Lewis Sebastian Riedel Omer Levy 30 21 0 08 Oct 2021
Towards Continual Knowledge Learning of Language Models Joel Jang Seonghyeon Ye Sohee Yang Joongbo Shin Janghoon Han Gyeonghun Kim Stanley Jungkyu Choi Minjoon Seo CLL KELM 233 151 0 07 Oct 2021
How BPE Affects Memorization in Transformers Eugene Kharitonov Marco Baroni Dieuwke Hupkes 169 32 0 06 Oct 2021
Towards Universal Dense Retrieval for Open-domain Question Answering Christopher Sciavolino RALM 10 1 0 23 Sep 2021
Simple Entity-Centric Questions Challenge Dense Retrievers Christopher Sciavolino Zexuan Zhong Jinhyuk Lee Danqi Chen RALM 27 160 0 17 Sep 2021
Entity-Based Knowledge Conflicts in Question Answering Shayne Longpre Kartik Perisetla Anthony Chen Nikhil Ramesh Chris DuBois Sameer Singh HILM 245 239 0 10 Sep 2021
R2-D2: A Modular Baseline for Open-Domain Question Answering Martin Fajcik Martin Docekal Karel Ondrej Pavel Smrz 33 46 0 08 Sep 2021
Boosting Search Engines with Interactive Agents Leonard Adolphs Benjamin Boerschinger Christian Buck Michelle Chen Huebscher Massimiliano Ciaramita ... Thomas Hofmann Yannic Kilcher Sascha Rothe Pier Giuseppe Sessa Lierni Sestorain Saralegui LLMAG 26 24 0 01 Sep 2021
Interactive Machine Comprehension with Dynamic Knowledge Graphs Xingdi Yuan 34 3 0 31 Aug 2021
Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning Cunxiang Wang Boyuan Zheng Y. Niu Yue Zhang LRM 33 22 0 15 Aug 2021
Is My Model Using The Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning Vivek Gupta Riyaz Ahmad Bhat Atreya Ghosal Manisha Srivastava M. Singh Vivek Srikumar LMTD 17 18 0 02 Aug 2021