v1v2 (latest)

Taking Notes on the Fly Helps BERT Pre-training

4 August 2020

Papers citing "Taking Notes on the Fly Helps BERT Pre-training"

24 / 24 papers shown

Title
Entities as Experts: Sparse Memory Access with Entity Supervision Thibault Févry Livio Baldini Soares Nicholas FitzGerald Eunsol Choi Tom Kwiatkowski RALM 87 153 0 15 Apr 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 145 2,116 0 10 Feb 2020
Generalization through Memorization: Nearest Neighbor Language Models Urvashi Khandelwal Omer Levy Dan Jurafsky Luke Zettlemoyer M. Lewis RALM 171 842 0 01 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 485 20,317 0 23 Oct 2019
Parsimonious Morpheme Segmentation with an Application to Enriching Word Embeddings Ahmed El-Kishky Frank F. Xu Aston Zhang Jiawei Han 38 4 0 18 Aug 2019
Representation Degeneration Problem in Training Natural Language Generation Models Jun Gao Di He Xu Tan Tao Qin Liwei Wang Tie-Yan Liu 62 270 0 28 Jul 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 677 24,541 0 26 Jul 2019
Energy and Policy Considerations for Deep Learning in NLP Emma Strubell Ananya Ganesh Andrew McCallum 73 2,660 0 05 Jun 2019
Rare Words: A Major Problem for Contextualized Embeddings And How to Fix it by Attentive Mimicking Timo Schick Hinrich Schütze 62 97 0 14 Apr 2019
Enriching Rare Word Representations in Neural Language Models by Embedding Matrix Augmentation Yerbolat Khassanov Zhiping Zeng Van Tung Pham Haihua Xu Chng Eng Siong 27 10 0 08 Apr 2019
fairseq: A Fast, Extensible Toolkit for Sequence Modeling Myle Ott Sergey Edunov Alexei Baevski Angela Fan Sam Gross Nathan Ng David Grangier Michael Auli VLM FaML 116 3,156 0 01 Apr 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,175 0 11 Oct 2018
FRAGE: Frequency-Agnostic Word Representation Chengyue Gong Di He Xu Tan Tao Qin Liwei Wang Tie-Yan Liu OOD 69 144 0 18 Sep 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,196 0 20 Apr 2018
Analyzing Uncertainty in Neural Machine Translation Myle Ott Michael Auli David Grangier MarcÁurelio Ranzato UQLM 110 275 0 28 Feb 2018
Mixed Precision Training Paulius Micikevicius Sharan Narang Jonah Alben G. Diamos Erich Elsen ... Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh Hao Wu 174 1,805 0 10 Oct 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 786 132,363 0 12 Jun 2017
Learning to Compute Word Embeddings On the Fly Dzmitry Bahdanau Tom Bosc Stanislaw Jastrzebski Edward Grefenstette Pascal Vincent Yoshua Bengio 77 83 0 01 Jun 2017
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 432 10,531 0 21 Jul 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,426 0 10 Dec 2015
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 228 7,757 0 31 Aug 2015
Character-Aware Neural Language Models Yoon Kim Yacine Jernite David Sontag Alexander M. Rush 107 1,669 0 26 Aug 2015
Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books Yukun Zhu Ryan Kiros R. Zemel Ruslan Salakhutdinov R. Urtasun Antonio Torralba Sanja Fidler 133 2,554 0 22 Jun 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.0K 150,312 0 22 Dec 2014