Title
The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization Jae-Won Chung Jiachen Liu Jeff J. Ma Ruofan Wu Oh Jun Kweon Yuxuan Xia Zhiyu Wu Mosharaf Chowdhury 28 0 0 09 May 2025
AccLLM: Accelerating Long-Context LLM Inference Via Algorithm-Hardware Co-Design Yanbiao Liang Huihong Shi Haikuo Shao Zhongfeng Wang 28 0 0 07 Apr 2025
LLMPi: Optimizing LLMs for High-Throughput on Raspberry Pi Mahsa Ardakani Jinendra Malekar Ramtin Zand MQ 37 0 0 02 Apr 2025
PIM-LLM: A High-Throughput Hybrid PIM Architecture for 1-bit LLMs Jinendra Malekar Peyton S. Chandarana Md Hasibul Amin Mohammed E. Elbtity Ramtin Zand 26 1 0 31 Mar 2025
Adaptive Rank Allocation: Speeding Up Modern Transformers with RaNA Adapters Roberto Garcia Jerry Liu Daniel Sorvisto Sabri Eyuboglu 90 0 0 23 Mar 2025
Changing Base Without Losing Pace: A GPU-Efficient Alternative to MatMul in DNNs Nir Ailon Akhiad Bercovich Omri Weinstein 54 0 0 15 Mar 2025
A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks Thomas Schmied Thomas Adler Vihang Patil M. Beck Korbinian Poppel Johannes Brandstetter G. Klambauer Razvan Pascanu Sepp Hochreiter 75 4 0 21 Feb 2025
QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache Rishabh Tiwari Haocheng Xi Aditya Tomar Coleman Hooper Sehoon Kim Maxwell Horton Mahyar Najibi Michael W. Mahoney K. K. Amir Gholami MQ 56 1 0 05 Feb 2025
SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantization Runsheng Bai Qiang Liu B. Liu MQ 59 1 0 05 Dec 2024
Interactions Across Blocks in Post-Training Quantization of Large Language Models Khasmamad Shabanovi Lukas Wiest Vladimir Golkov Daniel Cremers Thomas Pfeil MQ 31 1 0 06 Nov 2024
Computational Bottlenecks of Training Small-scale Large Language Models Saleh Ashkboos Iman Mirzadeh Keivan Alizadeh Mohammad Hossein Sekhavat Moin Nabi Mehrdad Farajtabar Fartash Faghri 21 0 0 25 Oct 2024
Dynamic Vocabulary Pruning in Early-Exit LLMs Jort Vincenti Karim Abdel Sadek Joan Velja Matteo Nulli Metod Jazbec 24 0 0 24 Oct 2024
BATON: Enhancing Batch-wise Inference Efficiency for Large Language Models via Dynamic Re-batching Peizhuang Cong Qizhi Chen Haochen Zhao Tong Yang KELM 23 1 0 24 Oct 2024
On-Chip Learning via Transformer In-Context Learning Jan Finkbeiner Emre Neftci 26 0 0 11 Oct 2024
Matmul or No Matmal in the Era of 1-bit LLMs Jinendra Malekar Mohammed E. Elbtity Ramtin Zand MQ 24 2 0 21 Aug 2024
BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts Qizhen Zhang Nikolas Gritsch Dwaraknath Gnaneshwar Simon Guo David Cairuz ... Jakob N. Foerster Phil Blunsom Sebastian Ruder A. Ustun Acyr F. Locatelli MoMe MoE 48 5 0 15 Aug 2024
Inference Optimizations for Large Language Models: Effects, Challenges, and Practical Considerations Leo Donisch Sigurd Schacht Carsten Lanquillon 22 2 0 06 Aug 2024
GreenStableYolo: Optimizing Inference Time and Image Quality of Text-to-Image Generation Jingzhi Gong Sisi Li Giordano dÁloisio Zishuo Ding Yulong Ye William B. Langdon Federica Sarro 45 2 0 20 Jul 2024
LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference Qichen Fu Minsik Cho Thomas Merth Sachin Mehta Mohammad Rastegari Mahyar Najibi 38 25 0 19 Jul 2024
Lean Attention: Hardware-Aware Scalable Attention Mechanism for the Decode-Phase of Transformers Rya Sanovar Srikant Bharadwaj Renée St. Amant Victor Rühle Saravan Rajmohan 49 6 0 17 May 2024
Characterizing the Accuracy - Efficiency Trade-off of Low-rank Decomposition in Language Models Chakshu Moar Michael Pellauer Hyoukjun Kwon 30 1 0 10 May 2024
Trio-ViT: Post-Training Quantization and Acceleration for Softmax-Free Efficient Vision Transformer Huihong Shi Haikuo Shao Wendong Mao Zhongfeng Wang ViT MQ 36 3 0 06 May 2024
Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey Dayou Du Gu Gong Xiaowen Chu MQ 32 7 0 01 May 2024
Mapping Parallel Matrix Multiplication in GotoBLAS2 to the AMD Versal ACAP for Deep Learning Jie Lei Enrique S. Quintana-Ortí 21 1 0 23 Apr 2024
Towards Coarse-to-Fine Evaluation of Inference Efficiency for Large Language Models Yushuo Chen Tianyi Tang Erge Xiang Linjiang Li Wayne Xin Zhao Jing Wang Yunpeng Chai Ji-Rong Wen 16 1 0 17 Apr 2024
CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models Je-Yong Lee Donghyun Lee Genghan Zhang Mo Tiwari Azalia Mirhoseini 36 13 0 12 Apr 2024
Towards Pareto Optimal Throughput in Small Language Model Serving Pol G. Recasens Yue Zhu Chen Wang Eun Kyung Lee Olivier Tardieu Alaa Youssef Jordi Torres Josep Ll. Berral 38 4 0 04 Apr 2024
Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference Jovan Stojkovic Esha Choukse Chaojie Zhang Inigo Goiri Josep Torrellas 41 36 0 29 Mar 2024
AI and Memory Wall A. Gholami Z. Yao Sehoon Kim Coleman Hooper Michael W. Mahoney Kurt Keutzer 25 141 0 21 Mar 2024
LLM Inference Unveiled: Survey and Roofline Model Insights Zhihang Yuan Yuzhang Shang Yang Zhou Zhen Dong Zhe Zhou ... Yong Jae Lee Yan Yan Beidi Chen Guangyu Sun Kurt Keutzer 37 79 0 26 Feb 2024
ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition Lu Ye Ze Tao Yong Huang Yang Li 32 26 0 23 Feb 2024
ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decoding Shuzhang Zhong Zebin Yang Meng Li Ruihao Gong Runsheng Wang Ru Huang 32 6 0 21 Feb 2024
Psychological Assessments with Large Language Models: A Privacy-Focused and Cost-Effective Approach Sergi Blanco-Cuaresma 23 1 0 05 Feb 2024
KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization Coleman Hooper Sehoon Kim Hiva Mohammadzadeh Michael W. Mahoney Y. Shao Kurt Keutzer A. Gholami MQ 23 174 0 31 Jan 2024
A Comprehensive Survey of Compression Algorithms for Language Models Seungcheol Park Jaehyeon Choi Sojin Lee U. Kang MQ 24 12 0 27 Jan 2024
FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design Haojun Xia Zhen Zheng Xiaoxia Wu Shiyang Chen Zhewei Yao ... Donglin Zhuang Zhongzhu Zhou Olatunji Ruwase Yuxiong He S. Song MQ 25 14 0 25 Jan 2024
The What, Why, and How of Context Length Extension Techniques in Large Language Models -- A Detailed Survey Saurav Pawar S.M. Towhidul Islam Tonmoy S. M. M. Zaman Vinija Jain Aman Chadha Amitava Das 35 27 0 15 Jan 2024
Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems Xupeng Miao Gabriele Oliaro Zhihao Zhang Xinhao Cheng Hongyi Jin Tianqi Chen Zhihao Jia 65 76 0 23 Dec 2023
Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inference Hongzheng Chen Jiahao Zhang Yixiao Du Shaojie Xiang Zichao Yue Niansong Zhang Yaohui Cai Zhiru Zhang 48 34 0 23 Dec 2023
Accelerator-driven Data Arrangement to Minimize Transformers Run-time on Multi-core Architectures Alireza Amirshahi G. Ansaloni David Atienza 18 0 0 20 Dec 2023
Conformer-Based Speech Recognition On Extreme Edge-Computing Devices Mingbin Xu Alex Jin Sicheng Wang Mu Su Tim Ng ... Shiyi Han Zhihong Lei Yaqiao Deng Zhen Huang Mahesh Krishnamoorthy 17 4 0 16 Dec 2023
Language Modeling on a SpiNNaker 2 Neuromorphic Chip Khaleelulla Khan Nazeer Mark Schöne Rishav Mukherji Bernhard Vogginger Christian Mayr David Kappel Anand Subramoney 32 5 0 14 Dec 2023
ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks Xiaoxia Wu Haojun Xia Stephen Youn Zhen Zheng Shiyang Chen ... Reza Yazdani Aminabadi Yuxiong He Olatunji Ruwase Leon Song Zhewei Yao 66 8 0 14 Dec 2023
EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism Yanxi Chen Xuchen Pan Yaliang Li Bolin Ding Jingren Zhou LRM 35 31 0 08 Dec 2023
MatFormer: Nested Transformer for Elastic Inference Devvrit Sneha Kudugunta Aditya Kusupati Tim Dettmers Kaifeng Chen ... Yulia Tsvetkov Hannaneh Hajishirzi Sham Kakade Ali Farhadi Prateek Jain 37 22 0 11 Oct 2023
ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models Iman Mirzadeh Keivan Alizadeh-Vahid Sachin Mehta C. C. D. Mundo Oncel Tuzel Golnoosh Samei Mohammad Rastegari Mehrdad Farajtabar 118 60 0 06 Oct 2023
GrowLength: Accelerating LLMs Pretraining by Progressively Growing Training Length Hongye Jin Xiaotian Han Jingfeng Yang Zhimeng Jiang Chia-Yuan Chang Xia Hu 33 11 0 01 Oct 2023
LLMCad: Fast and Scalable On-device Large Language Model Inference Daliang Xu Wangsong Yin Xin Jin Y. Zhang Shiyun Wei Mengwei Xu Xuanzhe Liu 17 43 0 08 Sep 2023
Predictive Pipelined Decoding: A Compute-Latency Trade-off for Exact LLM Decoding Seongjun Yang Gibbeum Lee Jaewoong Cho Dimitris Papailiopoulos Kangwook Lee 21 32 0 12 Jul 2023
SqueezeLLM: Dense-and-Sparse Quantization Sehoon Kim Coleman Hooper A. Gholami Zhen Dong Xiuyu Li Sheng Shen Michael W. Mahoney Kurt Keutzer MQ 24 167 0 13 Jun 2023