Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Execution

29 May 2024

Papers citing "Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Execution"

5 / 5 papers shown

Title
Number Cookbook: Number Understanding of Language Models and How to Improve It Haotong Yang Yi Hu Shijia Kang Zhouchen Lin Muhan Zhang LRM 46 2 0 06 Nov 2024
Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention Bin Gao Zhuomin He Puru Sharma Qingxuan Kang Djordje Jevdjic Junbo Deng Xingkun Yang Zhou Yu Pengfei Zuo 68 45 0 23 Mar 2024
Optimizing LLM Queries in Relational Data Analytics Workloads Shu Liu Asim Biswal Audrey Cheng Xiangxi Mo Shiyi Cao ... Ion Stoica Matei A. Zaharia Ion Stoica Joseph E. Gonzalez Matei Zaharia 71 18 0 09 Mar 2024
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding Yichao Fu Peter Bailis Ion Stoica Hao Zhang 130 141 0 03 Feb 2024
InferCept: Efficient Intercept Support for Augmented Large Language Model Inference Reyna Abhyankar Zijian He Vikranth Srivatsa Hao Zhang Yiying Zhang RALM 38 11 0 02 Feb 2024