LLM in a flash: Efficient Large Language Model Inference with Limited
Memory

v1v2 (latest)

LLM in a flash: Efficient Large Language Model Inference with Limited Memory

12 December 2023

Keivan Alizadeh-Vahid

Karen Khatamifard

Minsik Cho

Mohammad Rastegari

Mehrdad Farajtabar

ArXiv (abs)PDF HTML

Papers citing "LLM in a flash: Efficient Large Language Model Inference with Limited Memory"

18 / 18 papers shown

Title
COUNTDOWN: Contextually Sparse Activation Filtering Out Unnecessary Weights in Down Projection Jaewon Cheon Pilsung Kang 80 0 0 23 May 2025
KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments Junyoung Park Dalton Jones Matthew J Morse Raghavv Goel Mingu Lee Chris Lott 89 1 0 21 Apr 2025
PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing Cheng Deng Luoyang Sun Jiwen Jiang Yongcheng Zeng Xinjian Wu ... Haoyang Li Lei Chen Lionel M. Ni Jun Wang Jun Wang 411 0 0 15 Mar 2025
When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models Weilan Wang Yu Mao Dongdong Tang Hongchao Du Nan Guan Chun Jason Xue MQ 120 2 0 24 Feb 2025
Efficient LLM Inference using Dynamic Input Pruning and Cache-Aware Masking Marco Federici Davide Belli M. V. Baalen Amir Jalalirad Andrii Skliar Bence Major Markus Nagel Paul N. Whatmough 197 1 0 02 Dec 2024
Small Language Models: Survey, Measurements, and Insights Zhenyan Lu Xiang Li Dongqi Cai Rongjie Yi Fangming Liu Xiwen Zhang Nicholas D. Lane Mengwei Xu ObjD LRM 140 58 0 24 Sep 2024
Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models Keisuke Kamahori Tian Tang Yile Gu Kan Zhu Baris Kasikci 134 24 0 10 Feb 2024
A Hardware Evaluation Framework for Large Language Model Inference Hengrui Zhang August Ning R. Prabhakar D. Wentzlaff ELM 84 17 0 05 Dec 2023
Intriguing Properties of Quantization at Scale Arash Ahmadian Saurabh Dash Hongyu Chen Bharat Venkitesh Stephen Gou Phil Blunsom Ahmet Üstün Sara Hooker MQ 118 38 0 30 May 2023
Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt Zhaozhuo Xu Zirui Liu Beidi Chen Yuxin Tang Jue Wang Kaixiong Zhou Helen Zhou Anshumali Shrivastava MQ 93 31 0 17 May 2023
Alternating Updates for Efficient Transformers Cenk Baykal D. Cutler Nishanth Dikkala Nikhil Ghosh Rina Panigrahy Xin Wang MoE 76 5 0 30 Jan 2023
Fast Inference from Transformers via Speculative Decoding Yaniv Leviathan Matan Kalman Yossi Matias LRM 151 736 0 30 Nov 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 537 6,301 0 05 Apr 2022
SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning Hanrui Wang Zhekai Zhang Song Han 138 397 0 17 Dec 2020
Sparse GPU Kernels for Deep Learning Trevor Gale Matei A. Zaharia C. Young Erich Elsen 90 234 0 18 Jun 2020
Adaptive Computation Time for Recurrent Neural Networks Alex Graves 125 552 0 29 Mar 2016
EIE: Efficient Inference Engine on Compressed Deep Neural Network Song Han Xingyu Liu Huizi Mao Jing Pu A. Pedram M. Horowitz W. Dally 134 2,461 0 04 Feb 2016
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han Huizi Mao W. Dally 3DGS 263 8,864 0 01 Oct 2015

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.