Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Execution

29 May 2024

Papers citing "Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Execution"

6 / 6 papers shown

Title
Demystifying and Enhancing the Efficiency of Large Language Model Based Search Agents Tiannuo Yang Zebin Yao Bowen Jin Lixiao Cui Yusen Li Gang Wang Xiaoguang Liu LLMAG 2 0 0 17 May 2025
Number Cookbook: Number Understanding of Language Models and How to Improve It Haotong Yang Yi Hu Shijia Kang Zhouchen Lin Muhan Zhang LRM 46 2 0 06 Nov 2024
Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention Bin Gao Zhuomin He Puru Sharma Qingxuan Kang Djordje Jevdjic Junbo Deng Xingkun Yang Zhou Yu Pengfei Zuo 71 45 0 23 Mar 2024
Optimizing LLM Queries in Relational Data Analytics Workloads Shu Liu Asim Biswal Audrey Cheng Xiangxi Mo Shiyi Cao ... Ion Stoica Matei A. Zaharia Ion Stoica Joseph E. Gonzalez Matei Zaharia 74 18 0 09 Mar 2024
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding Yichao Fu Peter Bailis Ion Stoica Hao Zhang 133 143 0 03 Feb 2024
InferCept: Efficient Intercept Support for Augmented Large Language Model Inference Reyna Abhyankar Zijian He Vikranth Srivatsa Hao Zhang Yiying Zhang RALM 40 13 0 02 Feb 2024