Title
Protein sequence classification using natural language processing techniques Huma Perveen Julie Weeds 34 0 0 06 Sep 2024
A Text-guided Protein Design Framework Shengchao Liu Yanjing Li Zhuoxinran Li A. Gitter Yutao Zhu ... Arvind Ramanathan Chaowei Xiao Jian Tang Hongyu Guo Anima Anandkumar 98 62 0 09 Feb 2023
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 480 2,051 0 28 Jul 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 498 41,106 0 28 May 2020
ProGen: Language Modeling for Protein Generation Ali Madani Bryan McCann Nikhil Naik N. Keskar N. Anand Raphael R. Eguchi Po-Ssu Huang R. Socher 40 278 0 08 Mar 2020
Reformer: The Efficient Transformer Nikita Kitaev Lukasz Kaiser Anselm Levskaya VLM 159 2,279 0 13 Jan 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 261 42,038 0 03 Dec 2019
Pre-Training of Deep Bidirectional Protein Sequence Representations with Structural Information Seonwoo Min Seunghyun Park Siwon Kim Hyun-Soo Choi Byunghan Lee Sungroh Yoon SSL 36 62 0 25 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 265 19,824 0 23 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 266 6,420 0 26 Sep 2019
Attention Interpretability Across NLP Tasks Shikhar Vashishth Shyam Upadhyay Gaurav Singh Tomar Manaal Faruqui XAI MILM 76 176 0 24 Sep 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 288 1,861 0 17 Sep 2019
On the Variance of the Adaptive Learning Rate and Beyond Liyuan Liu Haoming Jiang Pengcheng He Weizhu Chen Xiaodong Liu Jianfeng Gao Jiawei Han ODL 152 1,894 0 08 Aug 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 183 8,386 0 19 Jun 2019
Evaluating Protein Transfer Learning with TAPE Roshan Rao Nicholas Bhattacharya Neil Thomas Yan Duan Xi Chen John F. Canny Pieter Abbeel Yun S. Song SSL 79 786 0 19 Jun 2019
A Multiscale Visualization of Attention in the Transformer Model Jesse Vig ViT 58 571 0 12 Jun 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 73 1,880 0 23 Apr 2019
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes Yang You Jing Li Sashank J. Reddi Jonathan Hseu Sanjiv Kumar Srinadh Bhojanapalli Xiaodan Song J. Demmel Kurt Keutzer Cho-Jui Hsieh ODL 163 991 0 01 Apr 2019
Learning protein sequence embeddings using information from structure Tristan Bepler Bonnie Berger 51 286 0 22 Feb 2019
ProteinNet: a standardized data set for machine learning of protein structure Mohammed AlQuraishi PINN 28 136 0 01 Feb 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 138 3,714 0 09 Jan 2019
A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation Akhilesh Deepak Gotmare N. Keskar Caiming Xiong R. Socher ODL 53 275 0 29 Oct 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 953 93,936 0 11 Oct 2018
TFLMS: Large Model Support in TensorFlow by Graph Rewriting Tung D. Le Haruki Imai Yasushi Negishi K. Kawachiya GNN 53 45 0 05 Jul 2018
Horovod: fast and easy distributed deep learning in TensorFlow Alexander Sergeev Mike Del Balso 59 1,218 0 15 Feb 2018
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 109 11,520 0 15 Feb 2018
PowerAI DDL Minsik Cho Ulrich Finkler Sameer Kumar David S. Kung Vaibhav Saxena D. Sreedhar AI4CE 21 47 0 07 Aug 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 443 129,831 0 12 Jun 2017
In-Datacenter Performance Analysis of a Tensor Processing Unit N. Jouppi C. Young Nishant Patil David Patterson Gaurav Agrawal ... Vijay Vasudevan Richard Walter Walter Wang Eric Wilcox Doe Hyun Yoon 181 4,619 0 16 Apr 2017
FastText.zip: Compressing text classification models Armand Joulin Edouard Grave Piotr Bojanowski Matthijs Douze Hervé Jégou Tomas Mikolov MQ 67 1,201 0 12 Dec 2016
Extreme Scale-out SuperMUC Phase 2 - lessons learned Nicolay J. Hammer F. Jamitzky H. Satzger M. Allalen A. Block ... H. Stüben A. Deshmukh Tobias Falkenstein K. Dolag Margarita Petkova 16 7 0 06 Sep 2016
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems Martín Abadi Ashish Agarwal P. Barham E. Brevdo Zhiwen Chen ... Pete Warden Martin Wattenberg Martin Wicke Yuan Yu Xiaoqiang Zheng 176 11,135 0 14 Mar 2016
Protein secondary structure prediction using deep convolutional neural fields Sheng Wang Jian-wei Peng Jianzhu Ma Jinbo Xu 3DV 47 514 0 02 Dec 2015
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 388 27,205 0 01 Sep 2014
One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling Ciprian Chelba Tomas Mikolov M. Schuster Qi Ge T. Brants P. Koehn T. Robinson 118 1,099 0 11 Dec 2013
Distributed Representations of Words and Phrases and their Compositionality Tomas Mikolov Ilya Sutskever Kai Chen G. Corrado J. Dean NAI OCL 296 33,445 0 16 Oct 2013