On the Inductive Bias of Masked Language Modeling: From Statistical to
Syntactic Dependencies

On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies

12 April 2021

Tatsunori Hashimoto

Papers citing "On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies"

8 / 8 papers shown

Title
Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph Completion Cunhang Fan Yujie Chen Jun Xue Yonghui Kong Jianhua Tao Zhao Lv 28 2 0 19 Jan 2024
Syntactic Substitutability as Unsupervised Dependency Syntax Jasper Jian Siva Reddy 27 3 0 29 Nov 2022
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models Hong Liu Sang Michael Xie Zhiyuan Li Tengyu Ma AI4CE 40 49 0 25 Oct 2022
Metadata Shaping: Natural Language Annotations for the Tail Simran Arora Sen Wu Enci Liu Christopher Ré 32 0 0 16 Oct 2021
How Does Adversarial Fine-Tuning Benefit BERT? J. Ebrahimi Hao Yang Wei Zhang AAML 26 4 0 31 Aug 2021
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning Colin Wei Sang Michael Xie Tengyu Ma 24 97 0 17 Jun 2021
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 446 2,589 0 03 Sep 2019
Efficient Estimation of Word Representations in Vector Space Tomáš Mikolov Kai Chen G. Corrado J. Dean 3DV 296 31,267 0 16 Jan 2013