KMIR: A Benchmark for Evaluating Knowledge Memorization, Identification and Reasoning Abilities of Language Models

28 February 2022

Papers citing "KMIR: A Benchmark for Evaluating Knowledge Memorization, Identification and Reasoning Abilities of Language Models"

31 / 31 papers shown

Title
Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text Siyuan Wang Wanjun Zhong Duyu Tang Zhongyu Wei Zhihao Fan Daxin Jiang Ming Zhou Nan Duan NAI 107 72 0 08 May 2021
JAKET: Joint Pre-training of Knowledge Graph and Language Understanding Donghan Yu Chenguang Zhu Yiming Yang Michael Zeng KELM 45 144 0 02 Oct 2020
LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning Jian Liu Leyang Cui Hanmeng Liu Dandan Huang Yile Wang Yue Zhang RALM 95 369 0 16 Jul 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 743 41,932 0 28 May 2020
The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence G. Marcus VLM 66 365 0 14 Feb 2020
How Much Knowledge Can You Pack Into the Parameters of a Language Model? Adam Roberts Colin Raffel Noam M. Shazeer KELM 107 890 0 10 Feb 2020
How Can We Know What Language Models Know? Zhengbao Jiang Frank F. Xu Jun Araki Graham Neubig KELM 130 1,405 0 28 Nov 2019
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 246 10,819 0 29 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 419 20,127 0 23 Oct 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 230 7,504 0 02 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 356 6,449 0 26 Sep 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 571 2,664 0 03 Sep 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 1.3K 12,193 0 27 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 615 24,431 0 26 Jul 2019
ELI5: Long Form Question Answering Angela Fan Yacine Jernite Ethan Perez David Grangier Jason Weston Michael Auli AI4MH ELM 82 617 0 22 Jul 2019
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions Christopher Clark Kenton Lee Ming-Wei Chang Tom Kwiatkowski Michael Collins Kristina Toutanova 217 1,517 0 24 May 2019
Cognitive Graph for Multi-Hop Reading Comprehension at Scale Ming Ding Chang Zhou Qibin Chen Hongxia Yang Jie Tang 72 226 0 14 May 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 256 2,312 0 02 May 2019
Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches Shane Storks Qiaozi Gao J. Chai 57 132 0 02 Apr 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 219 3,726 0 09 Jan 2019
Wizard of Wikipedia: Knowledge-Powered Conversational agents Emily Dinan Stephen Roller Kurt Shuster Angela Fan Michael Auli Jason Weston RALM KELM 113 947 0 03 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.7K 94,770 0 11 Oct 2018
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Zhilin Yang Peng Qi Saizheng Zhang Yoshua Bengio William W. Cohen Ruslan Salakhutdinov Christopher D. Manning RALM 166 2,647 0 25 Sep 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,154 0 20 Apr 2018
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 206 11,549 0 15 Feb 2018
Zero-Shot Relation Extraction via Reading Comprehension Omer Levy Minjoon Seo Eunsol Choi Luke Zettlemoyer ReLM 68 693 0 13 Jun 2017
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer RALM 204 2,646 0 09 May 2017
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 520 4,476 0 18 Apr 2017
ConceptNet 5.5: An Open Multilingual Graph of General Knowledge R. Speer Joshua Chin Catherine Havasi 193 2,896 0 12 Dec 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 274 8,127 0 16 Jun 2016
Reasoning about Entailment with Neural Attention Tim Rocktaschel Edward Grefenstette Karl Moritz Hermann Tomás Kociský Phil Blunsom NAI 62 762 0 22 Sep 2015