Title
AttentionLego: An Open-Source Building Block For Spatially-Scalable Large Language Model Accelerator With Processing-In-Memory Technology Rongqing Cong Wenyang He Mingxuan Li Bangning Luo Zebin Yang Yuchao Yang Ru Huang Bonan Yan 23 3 0 21 Jan 2024
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads Tianle Cai Yuhong Li Zhengyang Geng Hongwu Peng Jason D. Lee De-huai Chen Tri Dao 172 314 0 19 Jan 2024
A Survey on Hardware Accelerators for Large Language Models C. Kachris 72 15 0 18 Jan 2024
Mixtral of Experts Albert Q. Jiang Alexandre Sablayrolles Antoine Roux A. Mensch Blanche Savary ... Théophile Gervet Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LLMAG 164 1,123 0 08 Jan 2024
FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAs Shulin Zeng Jun Liu Guohao Dai Xinhao Yang Tianyu Fu ... Zehao Wang Ruoyu Zhang Kairui Wen Xuefei Ning Yu Wang 96 65 0 08 Jan 2024
Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems Xupeng Miao Gabriele Oliaro Zhihao Zhang Xinhao Cheng Hongyi Jin Tianqi Chen Zhihao Jia 129 84 0 23 Dec 2023
Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inference Hongzheng Chen Jiahao Zhang Yixiao Du Shaojie Xiang Zichao Yue Niansong Zhang Yaohui Cai Zhiru Zhang 94 39 0 23 Dec 2023
A Heterogeneous Chiplet Architecture for Accelerating End-to-End Transformer Models Harsh Sharma Pratyush Dhingra J. Doppa Ümit Y. Ogras P. Pande 61 7 0 18 Dec 2023
PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU Yixin Song Zeyu Mi Haotong Xie Haibo Chen BDL 158 135 0 16 Dec 2023
Mamba: Linear-Time Sequence Modeling with Selective State Spaces Albert Gu Tri Dao Mamba 150 2,786 0 01 Dec 2023
The Falcon Series of Open Language Models Ebtesam Almazrouei Hamza Alobeidli Abdulaziz Alshamsi Alessandro Cappelli Ruxandra-Aimée Cojocaru ... Quentin Malartic Daniele Mazzotta Badreddine Noune B. Pannier Guilherme Penedo AI4TS ALM 166 449 0 28 Nov 2023
FlashDecoding++: Faster Large Language Model Inference on GPUs Ke Hong Guohao Dai Jiaming Xu Qiuli Mao Xiuhong Li Jun Liu Kangdi Chen Yuhan Dong Yu Wang 74 77 0 02 Nov 2023
Efficient LLM Inference on CPUs Haihao Shen Hanwen Chang Bo Dong Yu Luo Hengyu Meng MQ 59 18 0 01 Nov 2023
Atom: Low-bit Quantization for Efficient and Accurate LLM Serving Yilong Zhao Chien-Yu Lin Kan Zhu Zihao Ye Lequn Chen Wenlei Bao Luis Ceze Arvind Krishnamurthy Tianqi Chen Baris Kasikci MQ 104 148 0 29 Oct 2023
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time Zichang Liu Jue Wang Tri Dao Dinesh Manocha Binhang Yuan ... Anshumali Shrivastava Ce Zhang Yuandong Tian Christopher Ré Beidi Chen BDL 110 218 0 26 Oct 2023
LLM-FP4: 4-Bit Floating-Point Quantized Transformers Shih-yang Liu Zechun Liu Xijie Huang Pingcheng Dong Kwang-Ting Cheng MQ 76 63 0 25 Oct 2023
E-Sparse: Boosting the Large Language Model Inference through Entropy-based N:M Sparsity Yun Li Lin Niu Xipeng Zhang Kai Liu Jianchen Zhu Zhanhui Kang MoE 64 13 0 24 Oct 2023
TEQ: Trainable Equivalent Transformation for Quantization of LLMs Wenhua Cheng Yiyang Cai Kaokao Lv Haihao Shen MQ 79 7 0 17 Oct 2023
QUIK: Towards End-to-End 4-Bit Inference on Generative Large Language Models Saleh Ashkboos Ilia Markov Elias Frantar Tingxuan Zhong Xincheng Wang Jie Ren Torsten Hoefler Dan Alistarh MQ SyDa 179 22 0 13 Oct 2023
Mistral 7B Albert Q. Jiang Alexandre Sablayrolles A. Mensch Chris Bamford Devendra Singh Chaplot ... Teven Le Scao Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LRM 110 2,246 0 10 Oct 2023
Efficient Streaming Language Models with Attention Sinks Michel Lang Yuandong Tian Beidi Chen Song Han Mike Lewis AI4TS RALM 138 790 0 29 Sep 2023
Qwen Technical Report Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang ... Zhenru Zhang Chang Zhou Jingren Zhou Xiaohuan Zhou Tianhang Zhu OSLM 268 1,908 0 28 Sep 2023
Efficient Memory Management for Large Language Model Serving with PagedAttention Woosuk Kwon Zhuohan Li Siyuan Zhuang Ying Sheng Lianmin Zheng Cody Hao Yu Joseph E. Gonzalez Haotong Zhang Ion Stoica VLM 196 2,322 0 12 Sep 2023
A Survey on Model Compression for Large Language Models Xunyu Zhu Jian Li Yong Liu Can Ma Weiping Wang 101 229 0 15 Aug 2023
VeriGen: A Large Language Model for Verilog Code Generation Shailja Thakur Baleegh Ahmad Hammond Pearce Benjamin Tan Brendan Dolan-Gavitt Ramesh Karri S. Garg 101 169 0 28 Jul 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 413 12,076 0 18 Jul 2023
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning Tri Dao LRM 117 1,332 0 17 Jul 2023
H $_2$ O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models Zhenyu Zhang Ying Sheng Dinesh Manocha Tianlong Chen Lianmin Zheng ... Yuandong Tian Christopher Ré Clark W. Barrett Zhangyang Wang Beidi Chen VLM 151 313 0 24 Jun 2023
A Simple and Effective Pruning Approach for Large Language Models Mingjie Sun Zhuang Liu Anna Bair J. Zico Kolter 156 440 0 20 Jun 2023
Block-State Transformers Mahan Fathi Jonathan Pilault Orhan Firat C. Pal Pierre-Luc Bacon Ross Goroshin 83 18 0 15 Jun 2023
SqueezeLLM: Dense-and-Sparse Quantization Sehoon Kim Coleman Hooper A. Gholami Zhen Dong Xiuyu Li Sheng Shen Michael W. Mahoney Kurt Keutzer MQ 108 197 0 13 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 450 4,444 0 09 Jun 2023
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression Tim Dettmers Ruslan Svirschevski Vage Egiazarian Denis Kuznedelev Elias Frantar Saleh Ashkboos Alexander Borzunov Torsten Hoefler Dan Alistarh MQ 71 254 0 05 Jun 2023
Faster Causal Attention Over Large Sequences Through Sparse Flash Attention Matteo Pagliardini Daniele Paliotta Martin Jaggi Franccois Fleuret LRM 69 25 0 01 Jun 2023
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration Ji Lin Jiaming Tang Haotian Tang Shang Yang Wei-Ming Chen Wei-Chen Wang Guangxuan Xiao Xingyu Dang Chuang Gan Song Han EDL MQ 104 578 0 01 Jun 2023
RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting Lei Shu Liangchen Luo Jayakumar Hoskere Yun Zhu Canoee Liu Simon Tong Jindong Chen Lei Meng KELM LRM 86 50 0 25 May 2023
RWKV: Reinventing RNNs for the Transformer Era Bo Peng Eric Alcaide Quentin G. Anthony Alon Albalak Samuel Arcadinho ... Qihang Zhao P. Zhou Qinghua Zhou Jian Zhu Rui-Jie Zhu 235 609 0 22 May 2023
LLM-Pruner: On the Structural Pruning of Large Language Models Xinyin Ma Gongfan Fang Xinchao Wang 142 441 0 19 May 2023
Inference with Reference: Lossless Acceleration of Large Language Models Nan Yang Tao Ge Liang Wang Binxing Jiao Daxin Jiang Linjun Yang Rangan Majumder Furu Wei 63 63 0 10 Apr 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.5K 14,761 0 15 Mar 2023
Hyena Hierarchy: Towards Larger Convolutional Language Models Michael Poli Stefano Massaroli Eric Q. Nguyen Daniel Y. Fu Tri Dao S. Baccus Yoshua Bengio Stefano Ermon Christopher Ré VLM 134 310 0 21 Feb 2023
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot Elias Frantar Dan Alistarh VLM 113 734 0 02 Jan 2023
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models Guangxuan Xiao Ji Lin Mickael Seznec Hao Wu Julien Demouth Song Han MQ 207 839 0 18 Nov 2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop : Teven Le Scao Angela Fan Christopher Akiki ... Zhongli Xie Zifan Ye M. Bras Younes Belkada Thomas Wolf VLM 414 2,393 0 09 Nov 2022
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers Elias Frantar Saleh Ashkboos Torsten Hoefler Dan Alistarh MQ 152 1,008 0 31 Oct 2022
ByteTransformer: A High-Performance Transformer Boosted for Variable-Length Inputs Yujia Zhai Chengquan Jiang Leyuan Wang Xiaoying Jia Shang Zhang Zizhong Chen Xin Liu Yibo Zhu 110 51 0 06 Oct 2022
DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation Seongmin Hong Seungjae Moon Junsoo Kim Sungjae Lee Minsub Kim Dongsoo Lee Joo-Young Kim 148 82 0 22 Sep 2022
DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale Reza Yazdani Aminabadi Samyam Rajbhandari Minjia Zhang A. A. Awan Cheng-rong Li ... Elton Zheng Jeff Rasley Shaden Smith Olatunji Ruwase Yuxiong He 85 369 0 30 Jun 2022
Long Range Language Modeling via Gated State Spaces Harsh Mehta Ankit Gupta Ashok Cutkosky Behnam Neyshabur Mamba 109 242 0 27 Jun 2022
LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models Gunho Park Baeseong Park Minsub Kim Sungjae Lee Jeonghoon Kim Beomseok Kwon S. Kwon Byeongwook Kim Youngjoo Lee Dongsoo Lee MQ 62 84 0 20 Jun 2022