Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge

16 December 2021

Papers citing "Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge"

22 / 22 papers shown

Title
Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs Peter Hase Mona T. Diab Asli Celikyilmaz Xian Li Zornitsa Kozareva Veselin Stoyanov Joey Tianyi Zhou Srini Iyer KELM LRM 63 79 0 26 Nov 2021
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models Peter West Chandrasekhar Bhagavatula Jack Hessel Jena D. Hwang Liwei Jiang Ronan Le Bras Ximing Lu Sean Welleck Yejin Choi SyDa 97 333 0 14 Oct 2021
Datasets: A Community Library for Natural Language Processing Quentin Lhoest Albert Villanova del Moral Yacine Jernite A. Thakur Patrick von Platen ... Thibault Goehringer Victor Mustar François Lagunas Alexander M. Rush Thomas Wolf 210 610 0 07 Sep 2021
General-Purpose Question-Answering with Macaw Oyvind Tafjord Peter Clark SyDa ELM MLLM 56 60 0 06 Sep 2021
Implicit Representations of Meaning in Neural Language Models Belinda Z. Li Maxwell Nye Jacob Andreas NAI MILM 55 176 0 01 Jun 2021
Do language models learn typicality judgments from text? Kanishka Misra Allyson Ettinger Julia Taylor Rayz 40 34 0 06 May 2021
Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand? William Merrill Yoav Goldberg Roy Schwartz Noah A. Smith 67 69 0 22 Apr 2021
Editing Factual Knowledge in Language Models Nicola De Cao Wilker Aziz Ivan Titov KELM 110 504 0 16 Apr 2021
Probing Across Time: What Does RoBERTa Know and When? Leo Z. Liu Yizhong Wang Jungo Kasai Hannaneh Hajishirzi Noah A. Smith KELM 76 85 0 16 Apr 2021
ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language Oyvind Tafjord Bhavana Dalvi Peter Clark 70 273 0 24 Dec 2020
Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both? Peter Shaw Ming-Wei Chang Panupong Pasupat Kristina Toutanova CoGe 71 187 0 24 Oct 2020
COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs Jena D. Hwang Chandra Bhagavatula Ronan Le Bras Jeff Da Keisuke Sakaguchi Antoine Bosselut Yejin Choi 75 409 0 12 Oct 2020
Measuring Systematic Generalization in Neural Proof Generation with Transformers Nicolas Angelard-Gontier Koustuv Sinha Siva Reddy C. Pal LRM 81 64 0 30 Sep 2020
RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms Pei Zhou Rahul Khanna Seyeon Lee Bill Yuchen Lin Daniel E. Ho Jay Pujara Xiang Ren ReLM 47 37 0 02 May 2020
TransOMCS: From Linguistic Graphs to Commonsense Knowledge Hongming Zhang Daniel Khashabi Yangqiu Song Dan Roth 50 94 0 01 May 2020
Transformers as Soft Reasoners over Language Peter Clark Oyvind Tafjord Kyle Richardson ReLM OffRL LRM 96 358 0 14 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 564 4,797 0 23 Jan 2020
oLMpics -- On what Language Model Pre-training Captures Alon Talmor Yanai Elazar Yoav Goldberg Jonathan Berant LRM 98 304 0 31 Dec 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 351 6,448 0 26 Sep 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 322 1,899 0 17 Sep 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 568 2,664 0 03 Sep 2019
COMET: Commonsense Transformers for Automatic Knowledge Graph Construction Antoine Bosselut Hannah Rashkin Maarten Sap Chaitanya Malaviya Asli Celikyilmaz Yejin Choi 82 910 0 12 Jun 2019