Syntactic Structure Distillation Pretraining For Bidirectional Encoders

27 May 2020

A. Kuncoro

Lingpeng Kong

Daniel Fried

Papers citing "Syntactic Structure Distillation Pretraining For Bidirectional Encoders"

48 / 48 papers shown

Title
A Systematic Assessment of Syntactic Generalization in Neural Language Models Jennifer Hu Jon Gauthier Peng Qian Ethan Gotlieb Wilcox R. Levy ELM 64 215 0 07 May 2020
Syntax-Infused Transformer and BERT models for Machine Translation and Natural Language Understanding Dhanasekar Sundararaman Vivek Subramanian Guoyin Wang Shijing Si Dinghan Shen Dong Wang Lawrence Carin 19 40 0 10 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 254 19,824 0 23 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 246 6,420 0 26 Sep 2019
Designing and Interpreting Probes with Control Tasks John Hewitt Percy Liang 56 531 0 08 Sep 2019
BERT for Coreference Resolution: Baselines and Analysis Mandar Joshi Omer Levy Daniel S. Weld Luke Zettlemoyer 54 321 0 24 Aug 2019
StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding Wei Wang Bin Bi Ming Yan Chen Henry Wu Zuyi Bao Jiangnan Xia Liwei Peng Luo Si 42 260 0 13 Aug 2019
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding Yu Sun Shuohuan Wang Yukun Li Shikun Feng Hao Tian Hua Wu Haifeng Wang CLL 75 804 0 29 Jul 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 387 24,160 0 26 Jul 2019
SpanBERT: Improving Pre-training by Representing and Predicting Spans Mandar Joshi Danqi Chen Yinhan Liu Daniel S. Weld Luke Zettlemoyer Omer Levy 109 1,953 0 24 Jul 2019
Cross-Domain Generalization of Neural Constituency Parsers Daniel Fried Nikita Kitaev Dan Klein NAI AI4CE 27 36 0 09 Jul 2019
Head-Driven Phrase Structure Grammar Parsing on Penn Treebank Junru Zhou Zhao Hai 56 144 0 05 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 173 8,386 0 19 Jun 2019
What do you learn from context? Probing for sentence structure in contextualized word representations Ian Tenney Patrick Xia Berlin Chen Alex Jinpeng Wang Adam Poliak ... Najoung Kim Benjamin Van Durme Samuel R. Bowman Dipanjan Das Ellie Pavlick 157 852 0 15 May 2019
BERT Rediscovers the Classical NLP Pipeline Ian Tenney Dipanjan Das Ellie Pavlick MILM SSeg 100 1,458 0 15 May 2019
Simple BERT Models for Relation Extraction and Semantic Role Labeling Peng Shi Jimmy J. Lin VLM 45 445 0 10 Apr 2019
Unsupervised Recurrent Neural Network Grammars Yoon Kim Alexander M. Rush Lei Yu A. Kuncoro Chris Dyer Gábor Melis LRM RALM SSL 44 115 0 07 Apr 2019
Linguistic Knowledge and Transferability of Contextual Representations Nelson F. Liu Matt Gardner Yonatan Belinkov Matthew E. Peters Noah A. Smith 90 728 0 21 Mar 2019
Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State Richard Futrell Ethan Gotlieb Wilcox Takashi Morita Peng Qian Miguel Ballesteros R. Levy MILM 116 193 0 08 Mar 2019
Structural Supervision Improves Learning of Non-Local Grammatical Dependencies Ethan Gotlieb Wilcox Peng Qian Richard Futrell Miguel Ballesteros R. Levy 37 56 0 03 Mar 2019
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference R. Thomas McCoy Ellie Pavlick Tal Linzen 111 1,226 0 04 Feb 2019
Learning and Evaluating General Linguistic Intelligence Dani Yogatama Cyprien de Masson dÁutume Jerome T. Connor Tomás Kociský Mike Chrzanowski ... Angeliki Lazaridou Wang Ling Lei Yu Chris Dyer Phil Blunsom ELM AI4CE 124 210 0 31 Jan 2019
Assessing BERT's Syntactic Abilities Yoav Goldberg 50 494 0 16 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 870 93,936 0 11 Oct 2018
Syntactic Scaffolds for Semantic Structures Swabha Swayamdipta Sam Thomson Kenton Lee Luke Zettlemoyer Chris Dyer Noah A. Smith 61 97 0 30 Aug 2018
Targeted Syntactic Evaluation of Language Models Rebecca Marvin Tal Linzen 58 414 0 27 Aug 2018
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing Taku Kudo John Richardson 142 3,490 0 19 Aug 2018
Neural Network Acceptability Judgments Alex Warstadt Amanpreet Singh Samuel R. Bowman 155 1,390 0 31 May 2018
Born Again Neural Networks Tommaso Furlanello Zachary Chase Lipton Michael Tschannen Laurent Itti Anima Anandkumar 60 1,030 0 12 May 2018
Linguistically-Informed Self-Attention for Semantic Role Labeling Emma Strubell Pat Verga D. Andor David J. Weiss Andrew McCallum OffRL 58 379 0 23 Apr 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 574 7,080 0 20 Apr 2018
Higher-order Coreference Resolution with Coarse-to-fine Inference Kenton Lee Luheng He Luke Zettlemoyer BDL 51 470 0 15 Apr 2018
AllenNLP: A Deep Semantic Natural Language Processing Platform Matt Gardner Joel Grus Mark Neumann Oyvind Tafjord Pradeep Dasigi Nelson F. Liu Matthew E. Peters Michael Schmitz Luke Zettlemoyer VLM 45 1,280 0 20 Mar 2018
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 99 11,520 0 15 Feb 2018
In-Order Transition-based Constituent Parsing Jiangming Liu Yue Zhang 59 66 0 17 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 422 129,831 0 12 Jun 2017
On-the-fly Operation Batching in Dynamic Computation Graphs Graham Neubig Yoav Goldberg Chris Dyer 43 60 0 22 May 2017
What do Neural Machine Translation Models Learn about Morphology? Yonatan Belinkov Nadir Durrani Fahim Dalvi Hassan Sajjad James R. Glass 85 414 0 11 Apr 2017
DyNet: The Dynamic Neural Network Toolkit Graham Neubig Chris Dyer Yoav Goldberg Austin Matthews Bridger Waleed Ammar ... Yusuke Oda Matthew Richardson Naomi Saphra Swabha Swayamdipta Pengcheng Yin 68 386 0 15 Jan 2017
What Do Recurrent Neural Network Grammars Learn About Syntax? Noah A. Smith Miguel Ballesteros Lingpeng Kong Chris Dyer Graham Neubig A. Kuncoro GNN 45 147 0 17 Nov 2016
Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies Tal Linzen Emmanuel Dupoux Yoav Goldberg 73 898 0 04 Nov 2016
Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks Yossi Adi Einat Kermany Yonatan Belinkov Ofer Lavi Yoav Goldberg 51 543 0 15 Aug 2016
Sequence-Level Knowledge Distillation Yoon Kim Alexander M. Rush 81 1,109 0 25 Jun 2016
Globally Normalized Transition-Based Neural Networks D. Andor Chris Alberti David J. Weiss Aliaksei Severyn Alessandro Presta Kuzman Ganchev Slav Petrov Michael Collins 75 568 0 19 Mar 2016
Recurrent Neural Network Grammars Chris Dyer A. Kuncoro Miguel Ballesteros Noah A. Smith GNN 58 524 0 25 Feb 2016
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 151 7,683 0 31 Aug 2015
Transition-Based Dependency Parsing with Stack Long Short-Term Memory Chris Dyer Miguel Ballesteros Wang Ling Austin Matthews Noah A. Smith 100 801 0 29 May 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 191 19,448 0 09 Mar 2015