Efficient Memory Management for Large Language Model Serving with PagedAttention

12 September 2023

Papers citing "Efficient Memory Management for Large Language Model Serving with PagedAttention"

50 / 412 papers shown

Title
Multi-agents based User Values Mining for Recommendation L. Chen Wei Yuan Tong Chen Xiangyu Zhao Nguyen Quoc Viet Hung Hongzhi Yin OffRL 55 0 0 02 May 2025
PipeSpec: Breaking Stage Dependencies in Hierarchical LLM Decoding Bradley McDanel Shanghang Zhang Y. Hu Zining Liu MoE 202 0 0 02 May 2025
Phantora: Live GPU Cluster Simulation for Machine Learning System Performance Estimation Jianxing Qin Jingrong Chen Xinhao Kong Yongji Wu Liang Luo Zhilin Wang Ying Zhang Tingjun Chen Alvin R. Lebeck Danyang Zhuo 169 0 0 02 May 2025
NeMo-Inspector: A Visualization Tool for LLM Generation Analysis Daria Gitman Igor Gitman Evelina Bakhturina SyDa 49 0 0 01 May 2025
Scaling On-Device GPU Inference for Large Generative Models Jiuqiang Tang Raman Sarokin Ekaterina Ignasheva Grant Jensen Lin Chen Juhyun Lee Andrei Kulik Matthias Grundmann 198 1 0 01 May 2025
Patchwork: A Unified Framework for RAG Serving Bodun Hu Luis Pabon Saurabh Agarwal Aditya Akella 26 0 0 01 May 2025
GPU Performance Portability needs Autotuning Burkhard Ringlein Thomas Parnell Radu Stoica 194 0 0 30 Apr 2025
DNB-AI-Project at SemEval-2025 Task 5: An LLM-Ensemble Approach for Automated Subject Indexing Lisa Kluge Maximilian Kähler 186 1 0 30 Apr 2025
Reinforcement Learning for Reasoning in Large Language Models with One Training Example Yiping Wang Qing Yang Zhiyuan Zeng Liliang Ren L. Liu ... Jianfeng Gao Weizhu Chen Shuaiqiang Wang Simon S. Du Yelong Shen OffRL ReLM LRM 131 9 0 29 Apr 2025
Beyond the Last Answer: Your Reasoning Trace Uncovers More than You Think Hasan Hammoud Hani Itani Guohao Li ReLM LRM 80 1 0 29 Apr 2025
Computational Reasoning of Large Language Models Haitao Wu Zongbo Han Joey Tianyi Zhou Huaxi Huang Changqing Zhang ELM LRM 62 0 0 29 Apr 2025
DMDTEval: An Evaluation and Analysis of LLMs on Disambiguation in Multi-domain Translation Zhibo Man Yuanmeng Chen Yuyao Zhang Jinan Xu 62 0 0 29 Apr 2025
FineQ: Software-Hardware Co-Design for Low-Bit Fine-Grained Mixed-Precision Quantization of LLMs Xilong Xie Liang Wang Limin Xiao Meng Han Lin Sun S. Zheng Xiangrong Xu MQ 31 0 0 28 Apr 2025
R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference Zhenyu Zhang Zechun Liu Yuandong Tian Harshit Khaitan Zhilin Wang Steven Li 68 0 0 28 Apr 2025
BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice Text Jiageng Wu Bowen Gu Ren Zhou Kevin Xie Doug Snyder ... Shri Kiran Srinivasan Jonathan H. Chen Santiago Romero-Brufau K. J. Lin Jie Yang LM&MA ELM 103 0 0 28 Apr 2025
Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Report Paul Kassianik Baturay Saglam Alexander Chen Blaine Nelson Anu Vellore ... Hyrum Anderson Kojin Oshiba Omar Santos Yaron Singer Amin Karbasi PILM 66 1 0 28 Apr 2025
Taming the Titans: A Survey of Efficient LLM Inference Serving Ranran Zhen J. Li Yixin Ji Zheng Yang Tong Liu Qingrong Xia Xinyu Duan Zehao Wang Baoxing Huai Hao Fei LLMAG 77 0 0 28 Apr 2025
Muyan-TTS: A Trainable Text-to-Speech Model Optimized for Podcast Scenarios with a $50K Budget$ Xin Li Kaikai Jia Hao Sun Jun Dai Z. L. Jiang 203 0 0 27 Apr 2025
GenTorrent: Scaling Large Language Model Serving with An Overley Network Fei Fang Yifan Hua Shengze Wang Ruilin Zhou Y. Liu Chen Qian Jiahui Geng 63 0 0 27 Apr 2025
Fast-Slow Thinking for Large Vision-Language Model Reasoning W. L. Xiao Leilei Gan Weilong Dai Wanggui He Ziwei Huang ... Fangxun Shu Zhelun Yu Peng Zhang Hao Jiang Fei Wu ReLM LRM AI4CE 239 1 0 25 Apr 2025
DualRAG: A Dual-Process Approach to Integrate Reasoning and Retrieval for Multi-Hop Question Answering Rong Cheng Qingbin Liu Yan Zheng Fei Ni Jiazhen Du Hangyu Mao Fuzheng Zhang Bo-Lan Wang Jianye Hao LRM 67 0 0 25 Apr 2025
Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers Jared Moore Declan Grabb William Agnew Kevin Klyman Stevie Chancellor Desmond C. Ong Nick Haber AI4MH 49 1 0 25 Apr 2025
Tempo: Application-aware LLM Serving with Mixed SLO Requirements Wei Zhang Zhiyu Wu Yi Mu Banruo Liu Myungjin Lee Fan Lai 60 0 0 24 Apr 2025
Circinus: Efficient Query Planner for Compound ML Serving Banruo Liu Wei-Yu Lin Minghao Fang Yihan Jiang Fan Lai LRM 39 0 0 23 Apr 2025
PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation Zihao An Huajun Bai Zichen Liu Dong Li E. Barsoum 61 0 0 23 Apr 2025
Dynamic Early Exit in Reasoning Models Chenxu Yang Qingyi Si Yongjie Duan Zheliang Zhu Chenyu Zhu Zheng Lin Zheng Lin Li Cao Weiping Wang ReLM LRM 53 6 0 22 Apr 2025
CAPO: Cost-Aware Prompt Optimization Tom Zehle Moritz Schlager Timo Heiß Matthias Feurer VLM 63 0 0 22 Apr 2025
Hexcute: A Tile-based Programming Language with Automatic Layout and Task-Mapping Synthesis Xinsong Zhang Yaoyao Ding Yang Hu Gennady Pekhimenko 49 0 0 22 Apr 2025
LLM-Assisted Translation of Legacy FORTRAN Codes to C++: A Cross-Platform Study Nishath Rajiv Ranasinghe Shawn M. Jones Michal Kucer Ayan Biswas Daniel O'Malley Alexander Buschmann Most Selma Liliane Wanna Ajay Sreekumar 32 0 0 21 Apr 2025
Efficient Pretraining Length Scaling Bohong Wu Shen Yan Sijun Zhang Jianqiao Lu Yutao Zeng Ya Wang Xun Zhou 209 0 0 21 Apr 2025
Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators Yilun Zhou Austin Xu Peifeng Wang Caiming Xiong Chenyu You ELM ALM LRM 58 3 0 21 Apr 2025
KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments Junyoung Park Dalton Jones Matthew J Morse Raghavv Goel Mingu Lee Chris Lott 29 0 0 21 Apr 2025
Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management Hang Zhang Jiuchen Shi Yixiao Wang Quan Chen Yizhou Shan Minyi Guo 38 0 0 19 Apr 2025
SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM X. Zhang Rongxiang Weng Zifei Cheng Wenhao Zhuang Zheng Lin ... Shouyu Yin Chaohang Wen Haotian Zhang Bin Chen Bing Yu LRM 43 5 0 19 Apr 2025
Compile Scene Graphs with Reinforcement Learning Zuyao Chen Jinlin Wu Zhen Lei Marc Pollefeys Chang Wen Chen OffRL LRM 59 0 0 18 Apr 2025
From Large to Super-Tiny: End-to-End Optimization for Cost-Efficient LLMs Jiliang Ni Jiachen Pu Zhongyi Yang Kun Zhou Hui Wang Xiaoliang Xiao Dakui Wang Xin Li Jingfeng Luo Conggang Hu 39 0 0 18 Apr 2025
$D$^{2}$MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving$ D $^{2}$ MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving Haodong Wang Qihua Zhou Zicong Hong Song Guo MoE 58 0 0 17 Apr 2025
Tilus: A Virtual Machine for Arbitrary Low-Precision GPGPU Computation in LLM Serving Yaoyao Ding Bohan Hou X. Zhang Allan Lin Tianqi Chen Cody Yu Hao Yida Wang Gennady Pekhimenko 50 0 0 17 Apr 2025
The Digital Cybersecurity Expert: How Far Have We Come? Dawei Wang Geng Zhou Xianglong Li Yu Bai Li Chen Ting Qin Jian Sun Didong Li ELM 72 0 0 16 Apr 2025
Teaching Large Language Models to Reason through Learning and Forgetting Tianwei Ni Allen Nie Sapana Chaudhary Yao Liu Huzefa Rangwala Rasool Fakoor ReLM CLL LRM 216 0 0 15 Apr 2025
SocioVerse: A World Model for Social Simulation Powered by LLM Agents and A Pool of 10 Million Real-World Users Jiahui Geng Jiayu Lin Xinyi Mou Shiyue Yang Xiawei Liu ... Jiebo Luo Shiping Tang Libo Wu Baohua Zhou Zhongyu Wei LLMAG 54 3 0 14 Apr 2025
MSCCL++: Rethinking GPU Communication Abstractions for Cutting-edge AI Applications Aashaka Shah Abhinav Jangda Yangqiu Song Caio Rocha Changho Hwang ... Peng Cheng Qinghua Zhou Roshan Dathathri Saeed Maleki Ziyue Yang GNN 54 0 0 11 Apr 2025
Throughput-Optimal Scheduling Algorithms for LLM Inference and AI Agents Yueying Li Jim Dai Tianyi Peng 171 1 0 10 Apr 2025
FuseRL: Dense Preference Optimization for Heterogeneous Model Fusion Longguang Zhong Fanqi Wan Ziyi Yang Guosheng Liang Tianyuan Shi Xiaojun Quan MoMe 59 0 0 09 Apr 2025
From Superficial to Deep: Integrating External Knowledge for Follow-up Question Generation Using Knowledge Graph and LLM Jianyu Liu Yi Huang Sheng Bi Junlan Feng Guilin Qi 49 2 0 08 Apr 2025
CARE: Aligning Language Models for Regional Cultural Awareness Geyang Guo Tarek Naous Hiromi Wakaki Yukiko Nishimura Yuki Mitsufuji Alan Ritter Wei Xu 56 1 0 07 Apr 2025
Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models Ruikang Liu Yuxuan Sun Manyi Zhang Haoli Bai Xianzhi Yu Tiezheng Yu C. Yuan Lu Hou MQ LRM 41 7 0 07 Apr 2025
Cognitive Memory in Large Language Models Lianlei Shan Shixian Luo Zezhou Zhu Yu Yuan Yong Wu LLMAG KELM 232 1 0 03 Apr 2025
When Reasoning Meets Compression: Benchmarking Compressed Large Reasoning Models on Complex Reasoning Tasks Nan Zhang Yusen Zhang Prasenjit Mitra Rui Zhang MQ LRM 69 2 0 02 Apr 2025
Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training Zhijun Wang Jiahuan Li Hao Zhou Rongxiang Weng Rongxiang Weng Xin Huang Xue Han Junlan Feng Chao Deng Shujian Huang LRM 59 1 0 02 Apr 2025