Efficiently Scaling Transformer Inference

Efficiently Scaling Transformer Inference

9 November 2022

Aakanksha Chowdhery

Anselm Levskaya

Shivani Agrawal

Papers citing "Efficiently Scaling Transformer Inference"

12 / 62 papers shown

Title
Learning to Compress Prompts with Gist Tokens Jesse Mu Xiang Lisa Li Noah D. Goodman VLM 53 206 0 17 Apr 2023
STen: Productive and Efficient Sparsity in PyTorch Andrei Ivanov Nikoli Dryden Tal Ben-Nun Saleh Ashkboos Torsten Hoefler 34 4 0 15 Apr 2023
CoLT5: Faster Long-Range Transformers with Conditional Computation Joshua Ainslie Tao Lei Michiel de Jong Santiago Ontañón Siddhartha Brahma ... Mandy Guo James Lee-Thorp Yi Tay Yun-hsuan Sung Sumit Sanghai LLMAG 36 63 0 17 Mar 2023
With Shared Microexponents, A Little Shifting Goes a Long Way Bita Darvish Rouhani Ritchie Zhao V. Elango Rasoul Shafipour Mathew Hall ... Eric S. Chung Zhaoxia Deng S. Naghshineh Jongsoo Park Maxim Naumov MQ 43 36 0 16 Feb 2023
Scaling Vision Transformers to 22 Billion Parameters Mostafa Dehghani Josip Djolonga Basil Mustafa Piotr Padlewski Jonathan Heek ... Mario Luvcić Xiaohua Zhai Daniel Keysers Jeremiah Harmsen N. Houlsby MLLM 90 572 0 10 Feb 2023
Accelerating Large Language Model Decoding with Speculative Sampling Charlie Chen Sebastian Borgeaud G. Irving Jean-Baptiste Lespiau Laurent Sifre J. Jumper BDL LRM 8 384 0 02 Feb 2023
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models Guangxuan Xiao Ji Lin Mickael Seznec Hao Wu Julien Demouth Song Han MQ 86 741 0 18 Nov 2022
Desiderata for next generation of ML model serving Sherif Akoush Andrei Paleyes A. V. Looveren Clive Cox 38 5 0 26 Oct 2022
Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference Sneha Kudugunta Yanping Huang Ankur Bapna M. Krikun Dmitry Lepikhin Minh-Thang Luong Orhan Firat MoE 119 107 0 24 Sep 2021
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 252 580 0 12 Mar 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 264 4,505 0 23 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,833 0 17 Sep 2019