ELIS: Efficient LLM Iterative Scheduling System with Response Length Predictor

ELIS: Efficient LLM Iterative Scheduling System with Response Length Predictor

14 May 2025

ArXiv (abs)PDF HTML

Papers citing "ELIS: Efficient LLM Iterative Scheduling System with Response Length Predictor"

9 / 9 papers shown

Title
Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction Haoran Qiu Weichao Mao Archit Patke Shengkun Cui Saurabh Jha Chen Wang Hubertus Franke Zbigniew T. Kalbarczyk Tamer Basar Ravishankar K. Iyer 62 26 0 12 Apr 2024
ExeGPT: Constraint-Aware Resource Scheduling for LLM Inference Hyungjun Oh Kihong Kim Jaemin Kim Sungkyun Kim Junyeol Lee Du-Seong Chang Jiwon Seo 69 35 0 15 Mar 2024
$S$^{3}$: Increasing GPU Utilization during Generative Inference for Higher Throughput$ S $^{3}$ : Increasing GPU Utilization during Generative Inference for Higher Throughput Yunho Jin Chun-Feng Wu David Brooks Gu-Yeon Wei 84 68 0 09 Jun 2023
Varuna: Scalable, Low-cost Training of Massive Deep Learning Models Sanjith Athlur Nitika Saran Muthian Sivathanu Ramachandran Ramjee Nipun Kwatra GNN 67 81 0 07 Nov 2021
Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads Deepak Narayanan Keshav Santhanam Fiodar Kazhamiaka Amar Phanishayee Matei A. Zaharia 58 210 0 20 Aug 2020
Serving DNNs like Clockwork: Performance Predictability from the Bottom Up A. Gujarati Reza Karimi Safya Alzayat Wei Hao Antoine Kaufmann Ymir Vigfusson Jonathan Mace 85 280 0 03 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 795 42,055 0 28 May 2020
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 94,891 0 11 Oct 2018
Clipper: A Low-Latency Online Prediction Serving System D. Crankshaw Xin Wang Giulio Zhou Michael Franklin Joseph E. Gonzalez Ion Stoica 61 674 0 09 Dec 2016