A Survey on Transformers in NLP with Focus on Efficiency

A Survey on Transformers in NLP with Focus on Efficiency

15 May 2024

Saptarsi Goswami

Amlan Chakrabarti

Papers citing "A Survey on Transformers in NLP with Focus on Efficiency"

19 / 69 papers shown

Title
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 612 24,431 0 26 Jul 2019
Green AI Roy Schwartz Jesse Dodge Noah A. Smith Oren Etzioni 102 1,144 0 22 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 227 8,426 0 19 Jun 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 112 1,896 0 23 Apr 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 216 3,726 0 09 Jan 2019
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 204 11,549 0 15 Feb 2018
Get To The Point: Summarization with Pointer-Generator Networks A. See Peter J. Liu Christopher D. Manning 3DPC 291 4,019 0 14 Apr 2017
A Structured Self-attentive Sentence Embedding Zhouhan Lin Minwei Feng Cicero Nogueira dos Santos Mo Yu Bing Xiang Bowen Zhou Yoshua Bengio 113 2,139 0 09 Mar 2017
Structured Attention Networks Yoon Kim Carl Denton Luong Hoang Alexander M. Rush 106 463 0 03 Feb 2017
SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents Ramesh Nallapati Feifei Zhai Bowen Zhou 341 1,262 0 14 Nov 2016
Bag of Tricks for Efficient Text Classification Armand Joulin Edouard Grave Piotr Bojanowski Tomas Mikolov VLM 168 4,619 0 06 Jul 2016
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 377 7,959 0 17 Aug 2015
Neural Turing Machines Alex Graves Greg Wayne Ivo Danihelka 97 2,327 0 20 Oct 2014
Sequence to Sequence Learning with Neural Networks Ilya Sutskever Oriol Vinyals Quoc V. Le AIMat 434 20,541 0 10 Sep 2014
On the Properties of Neural Machine Translation: Encoder-Decoder Approaches Kyunghyun Cho B. V. Merrienboer Dzmitry Bahdanau Yoshua Bengio AI4CE AIMat 237 6,775 0 03 Sep 2014
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 546 27,300 0 01 Sep 2014
Distributed Representations of Sentences and Documents Quoc V. Le Tomas Mikolov FaML 251 9,242 0 16 May 2014
Distributed Representations of Words and Phrases and their Compositionality Tomas Mikolov Ilya Sutskever Kai Chen G. Corrado J. Dean NAI OCL 392 33,521 0 16 Oct 2013
Natural Language Processing (almost) from Scratch R. Collobert Jason Weston Léon Bottou Michael Karlen Koray Kavukcuoglu Pavel P. Kuksa 186 7,725 0 02 Mar 2011