Variance-reduced Language Pretraining via a Mask Proposal Network

v1v2 (latest)

Variance-reduced Language Pretraining via a Mask Proposal Network

12 August 2020

ArXiv (abs)PDF HTML

Papers citing "Variance-reduced Language Pretraining via a Mask Proposal Network"

8 / 8 papers shown

Title
Unified Language Model Pre-training for Natural Language Understanding and Generation Li Dong Nan Yang Wenhui Wang Furu Wei Xiaodong Liu Yu Wang Jianfeng Gao M. Zhou H. Hon ELM AI4CE 236 1,560 0 08 May 2019
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes Yang You Jing Li Sashank J. Reddi Jonathan Hseu Sanjiv Kumar Srinadh Bhojanapalli Xiaodan Song J. Demmel Kurt Keutzer Cho-Jui Hsieh ODL 274 998 0 01 Apr 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,201 0 20 Apr 2018
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 235 11,569 0 15 Feb 2018
Gaussian Error Linear Units (GELUs) Dan Hendrycks Kevin Gimpel 178 5,056 0 27 Jun 2016
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 244 7,765 0 31 Aug 2015
Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks Kai Sheng Tai R. Socher Christopher D. Manning AIMat 146 3,123 0 28 Feb 2015
A Convolutional Neural Network for Modelling Sentences Nal Kalchbrenner Edward Grefenstette Phil Blunsom 111 3,562 0 08 Apr 2014