Title
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics Kai He Rui Mao Qika Lin Yucheng Ruan Xiang Lan Mengling Feng Min Zhang LM&MA AILaw 154 166 0 28 Jan 2025
Revisiting Reliability in Large-Scale Machine Learning Research Clusters Apostolos Kokolis Michael Kuchnik John Hoffman Adithya Kumar Parth Malani Faye Ma Zachary DeVito Siyang Song Kalyan Saladi Carole-Jean Wu 259 8 0 29 Oct 2024
Glauber Generative Model: Discrete Diffusion Models via Binary Classification Harshit Varma Dheeraj M. Nagaraj Karthikeyan Shanmugam VLM 93 3 0 27 May 2024
FlexLLM: A System for Co-Serving Large Language Model Inference and Parameter-Efficient Finetuning Xupeng Miao Gabriele Oliaro Xinhao Cheng Vineeth Kada Ruohan Gao ... April Yang Yingcheng Wang Mengdi Wu Colin Unger Zhihao Jia MoE 112 10 0 29 Feb 2024
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 328 6,132 0 05 Apr 2022
ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning Samyam Rajbhandari Olatunji Ruwase Jeff Rasley Shaden Smith Yuxiong He GNN 55 376 0 16 Apr 2021
Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM Deepak Narayanan Mohammad Shoeybi Jared Casper P. LeGresley M. Patwary ... Prethvi Kashinkunti J. Bernauer Bryan Catanzaro Amar Phanishayee Matei A. Zaharia MoE 68 667 0 09 Apr 2021
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity W. Fedus Barret Zoph Noam M. Shazeer MoE 55 2,136 0 11 Jan 2021
Nimble: Lightweight and Parallel GPU Task Scheduling for Deep Learning Woosuk Kwon Gyeong-In Yu Eunji Jeong Byung-Gon Chun 36 68 0 04 Dec 2020
Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads Deepak Narayanan Keshav Santhanam Fiodar Kazhamiaka Amar Phanishayee Matei A. Zaharia 34 207 0 20 Aug 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Zhiwen Chen MoE 76 1,142 0 30 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 469 41,106 0 28 May 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 242 42,038 0 03 Dec 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 260 19,824 0 23 Oct 2019
PipeMare: Asynchronous Pipeline Parallel DNN Training Bowen Yang Jian Zhang Jonathan Li Christopher Ré Christopher R. Aberger Christopher De Sa 42 110 0 09 Oct 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 283 1,861 0 17 Sep 2019
Themis: Fair and Efficient GPU Cluster Scheduling Kshiteej S. Mahajan Arjun Balasubramanian Arjun Singhvi Shivaram Venkataraman Aditya Akella Amar Phanishayee Shuchi Chawla 41 183 0 02 Jul 2019
Neural Message Passing for Multi-Label Classification Jack Lanchantin Arshdeep Sekhon Yanjun Qi 42 38 0 17 Apr 2019
TensorFlow Eager: A Multi-Stage, Python-Embedded DSL for Machine Learning Akshay Agrawal A. Modi Alexandre Passos Allen Lavoie Ashish Agarwal ... Igor Ganichev J. Levenberg Mingsheng Hong R. Monga Shanqing Cai 57 79 0 27 Feb 2019
Parameter-Efficient Transfer Learning for NLP N. Houlsby A. Giurgiu Stanislaw Jastrzebski Bruna Morrone Quentin de Laroussilhe Andrea Gesmundo Mona Attariyan Sylvain Gelly 179 4,368 0 02 Feb 2019
Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads Myeongjae Jeon Shivaram Venkataraman Amar Phanishayee Junjie Qian Wencong Xiao Fan Yang GNN 53 353 0 17 Jan 2019
Measuring the Effects of Data Parallelism on Neural Network Training Christopher J. Shallue Jaehoon Lee J. Antognini J. Mamou J. Ketterling Yao Wang 68 408 0 08 Nov 2018
Mesh-TensorFlow: Deep Learning for Supercomputers Noam M. Shazeer Youlong Cheng Niki Parmar Dustin Tran Ashish Vaswani ... HyoukJoong Lee O. Milenkovic C. Young Ryan Sepassi Blake Hechtman GNN MoE AI4CE 49 387 0 05 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 943 93,936 0 11 Oct 2018
Dynamic Control Flow in Large-Scale Machine Learning Yuan Yu Martín Abadi P. Barham E. Brevdo M. Burrows ... Michael Isard M. Kudlur R. Monga D. Murray Xiaoqiang Zheng AI4CE 50 106 0 04 May 2018
Shampoo: Preconditioned Stochastic Tensor Optimization Vineet Gupta Tomer Koren Y. Singer ODL 48 214 0 26 Feb 2018
Efficient Neural Architecture Search via Parameter Sharing Hieu H. Pham M. Guan Barret Zoph Quoc V. Le J. Dean 76 2,761 0 09 Feb 2018
Ray: A Distributed Framework for Emerging AI Applications Philipp Moritz Robert Nishihara Stephanie Wang Alexey Tumanov Richard Liaw ... Melih Elibol Zongheng Yang William Paul Michael I. Jordan Ion Stoica GNN 58 1,241 0 16 Dec 2017
Large Batch Training of Convolutional Networks Yang You Igor Gitman Boris Ginsburg ODL 106 844 0 13 Aug 2017
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer Noam M. Shazeer Azalia Mirhoseini Krzysztof Maziarz Andy Davis Quoc V. Le Geoffrey E. Hinton J. Dean MoE 158 2,614 0 23 Jan 2017
Clipper: A Low-Latency Online Prediction Serving System D. Crankshaw Xin Wang Giulio Zhou Michael Franklin Joseph E. Gonzalez Ion Stoica 50 673 0 09 Dec 2016
TensorFlow: A system for large-scale machine learning Martín Abadi P. Barham Jianmin Chen Zhiwen Chen Andy Davis ... Vijay Vasudevan Pete Warden Martin Wicke Yuan Yu Xiaoqiang Zhang GNN AI4CE 331 18,300 0 27 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.4K 192,638 0 10 Dec 2015