v1v2 (latest)

On the validity of pre-trained transformers for natural language processing in the software engineering domain

10 September 2021

Papers citing "On the validity of pre-trained transformers for natural language processing in the software engineering domain"

30 / 30 papers shown

Title
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity W. Fedus Barret Zoph Noam M. Shazeer MoE 88 2,220 0 11 Jan 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 556 2,099 0 28 Jul 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 877 42,379 0 28 May 2020
Code and Named Entity Recognition in StackOverflow Jeniya Tabassum Mounica Maddela Wei Xu Alan Ritter 122 119 0 04 May 2020
CodeBERT: A Pre-Trained Model for Programming and Natural Languages Zhangyin Feng Daya Guo Duyu Tang Nan Duan Xiaocheng Feng ... Linjun Shou Bing Qin Ting Liu Daxin Jiang Ming Zhou 165 2,667 0 19 Feb 2020
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 264 10,851 0 29 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 373 6,467 0 26 Sep 2019
FinBERT: Financial Sentiment Analysis with Pre-trained Language Models Dogu Araci AIFin 120 648 0 27 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 677 24,541 0 26 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 236 8,447 0 19 Jun 2019
A Multiscale Visualization of Attention in the Transformer Model Jesse Vig ViT 79 582 0 12 Jun 2019
Energy and Policy Considerations for Deep Learning in NLP Emma Strubell Ananya Ganesh Andrew McCallum 73 2,660 0 05 Jun 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 274 2,323 0 02 May 2019
ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission Kexin Huang Jaan Altosaar Rajesh Ranganath OOD 122 909 0 10 Apr 2019
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes Yang You Jing Li Sashank J. Reddi Jonathan Hseu Sanjiv Kumar Srinadh Bhojanapalli Xiaodan Song J. Demmel Kurt Keutzer Cho-Jui Hsieh ODL 261 999 0 01 Apr 2019
Import2vec - Learning Embeddings for Software Libraries B. Theeten Frederik Vandeputte Tom Van Cutsem SSL 41 34 0 27 Mar 2019
SciBERT: A Pretrained Language Model for Scientific Text Iz Beltagy Kyle Lo Arman Cohan 168 2,985 0 26 Mar 2019
BioBERT: a pre-trained biomedical language representation model for biomedical text mining Jinhyuk Lee Wonjin Yoon Sungdong Kim Donghyeon Kim Sunkyu Kim Chan Ho So Jaewoo Kang OOD 180 5,672 0 25 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,175 0 11 Oct 2018
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems S. Kiritchenko Saif M. Mohammad FaML 86 440 0 11 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,196 0 20 Apr 2018
code2vec: Learning Distributed Representations of Code Uri Alon Meital Zilberstein Omer Levy Eran Yahav 53 1,181 0 26 Mar 2018
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 233 11,566 0 15 Feb 2018
Mixed Precision Training Paulius Micikevicius Sharan Narang Jonah Alben G. Diamos Erich Elsen ... Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh Hao Wu 174 1,805 0 10 Oct 2017
Sentiment Polarity Detection for Software Development Fabio Calefato F. Lanubile Federico Maiorano Nicole Novielli 45 228 0 09 Sep 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 786 132,363 0 12 Jun 2017
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 316 8,169 0 16 Jun 2016
Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis A. Benavoli Giorgio Corani J. Demšar Marco Zaffalon BDL 67 424 0 14 Jun 2016
Characterizing Diseases from Unstructured Text: A Vocabulary Driven Word2vec Approach Saurav Ghosh Prithwish Chakraborty E. Cohn J. Brownstein Naren Ramakrishnan 58 33 0 01 Mar 2016
Efficient Estimation of Word Representations in Vector Space Tomas Mikolov Kai Chen G. Corrado J. Dean 3DV 684 31,544 0 16 Jan 2013