How Much Knowledge Can You Pack Into the Parameters of a Language Model?

How Much Knowledge Can You Pack Into the Parameters of a Language Model?

10 February 2020

Noam M. Shazeer

Papers citing "How Much Knowledge Can You Pack Into the Parameters of a Language Model?"

17 / 217 papers shown

Title
Differentiable Open-Ended Commonsense Reasoning Bill Yuchen Lin Haitian Sun Bhuwan Dhingra Manzil Zaheer Xiang Ren William W. Cohen ReLM LRM 13 41 0 24 Oct 2020
Is Retriever Merely an Approximator of Reader? Sohee Yang Minjoon Seo RALM 21 39 0 21 Oct 2020
X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models Zhengbao Jiang Antonios Anastasopoulos Jun Araki Haibo Ding Graham Neubig HILM KELM 15 136 0 13 Oct 2020
Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems Andrea Madotto Samuel Cahyawijaya Genta Indra Winata Yan Xu Zihan Liu Zhaojiang Lin Pascale Fung 36 59 0 28 Sep 2020
Generation-Augmented Retrieval for Open-domain Question Answering Yuning Mao Pengcheng He Xiaodong Liu Yelong Shen Jianfeng Gao Jiawei Han Weizhu Chen RALM 42 238 0 17 Sep 2020
Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets Patrick Lewis Pontus Stenetorp Sebastian Riedel OOD ELM 12 184 0 06 Aug 2020
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering Shayne Longpre Yi Lu Joachim Daiber ELM HILM 41 152 0 30 Jul 2020
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering Gautier Izacard Edouard Grave RALM 25 1,108 0 02 Jul 2020
Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge Pat Verga Haitian Sun Livio Baldini Soares William W. Cohen KELM 35 50 0 02 Jul 2020
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval Lee Xiong Chenyan Xiong Ye Li Kwok-Fung Tang Jialin Liu Paul N. Bennett Junaid Ahmed Arnold Overwijk 11 1,180 0 01 Jul 2020
A Survey on Transfer Learning in Natural Language Processing Zaid Alyafeai Maged S. Alshaibani Irfan Ahmad 30 72 0 31 May 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 50 40,200 0 28 May 2020
UnifiedQA: Crossing Format Boundaries With a Single QA System Daniel Khashabi Sewon Min Tushar Khot Ashish Sabharwal Oyvind Tafjord Peter Clark Hannaneh Hajishirzi 35 720 0 02 May 2020
Probing Contextual Language Models for Common Ground with Visual Representations Gabriel Ilharco Rowan Zellers Ali Farhadi Hannaneh Hajishirzi 30 14 0 01 May 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 36 1,992 0 10 Feb 2020
How Decoding Strategies Affect the Verifiability of Generated Text Luca Massarelli Fabio Petroni Aleksandra Piktus Myle Ott Tim Rocktaschel Vassilis Plachouras Fabrizio Silvestri Sebastian Riedel 23 50 0 09 Nov 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 425 2,588 0 03 Sep 2019