RouterBench: A Benchmark for Multi-LLM Routing System

18 March 2024

Shriyash Kaustubh Upadhyay

ArXiv PDF HTML

Papers citing "RouterBench: A Benchmark for Multi-LLM Routing System"

32 / 32 papers shown

Title
Guarded Query Routing for Large Language Models Richard Šléher William Brach Tibor Sloboda Kristián Košťál Lukas Galke RALM 14 0 0 20 May 2025
COSMOS: Predictable and Cost-Effective Adaptation of LLMs Jiayu Wang Aws Albarghouthi Frederic Sala 57 0 0 30 Apr 2025
Can a Crow Hatch a Falcon? Lineage Matters in Predicting Large Language Model Performance Takuya Tamura Taro Yano Masafumi Enomoto Masafumi Oyamada 50 0 0 28 Apr 2025
Toward Super Agent System with Hybrid AI Routers Yuhang Yao Haixin Wang Yibo Chen Jiawen Wang Min Chang Jordan Ren Bosheng Ding Salman Avestimehr Chaoyang He LLMAG LM&Ro 51 1 0 11 Apr 2025
Resource-efficient Inference with Foundation Model Programs Lunyiu Nie Zhimin Ding Kevin Yu Marco Cheung C. Jermaine S. Chaudhuri 30 0 0 09 Apr 2025
Geo-OLM: Enabling Sustainable Earth Observation Studies with Cost-Efficient Open Language Models & State-Driven Workflows Dimitrios Stamoulis Diana Marculescu 28 0 0 06 Apr 2025
Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Compute Jianhao Chen Zishuo Xun Bocheng Zhou Han Qi Qiaosheng Zhang ... Wei Hu Yuzhong Qu W. Ouyang Wanli Ouyang Shuyue Hu 74 1 0 01 Apr 2025
Improving the End-to-End Efficiency of Offline Inference for Multi-LLM Applications Based on Sampling and Simulation Jingzhi Fang Yanyan Shen Yufei Wang Lei Chen 47 2 0 21 Mar 2025
Ensemble Learning for Large Language Models in Text and Code Generation: A Survey Mari Ashiga Wei Jie Fan Wu Vardan K. Voskanyan Fateme Dinmohammadi P. Brookes Jingzhi Gong Zheng Wang 44 0 0 13 Mar 2025
ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learning Bo Liu Yunxiang Li Yangqiu Song Hanjing Wang Linyi Yang Mark Schmidt Jun Wang Weinan Zhang Shuyue Hu Ying Wen LLMAG KELM LRM AI4CE 94 6 0 12 Mar 2025
Life-Cycle Routing Vulnerabilities of LLM Router Qiqi Lin Xiaoyang Ji Shengfang Zhai Qingni Shen Zhi-Li Zhang Yuejian Fang Yansong Gao AAML 65 1 0 09 Mar 2025
When does a predictor know its own loss? Aravind Gollakota Parikshit Gopalan Aayush Karan Charlotte Peale Udi Wieder UQCV FaML 67 0 0 27 Feb 2025
Aligning Compound AI Systems via System-level DPO Xiangwen Wang Yibo Jacky Zhang Zhoujie Ding Katherine Tsai Sanmi Koyejo 43 0 0 24 Feb 2025
Capability Instruction Tuning: A New Paradigm for Dynamic LLM Routing Yi-Kai Zhang De-Chuan Zhan Han-Jia Ye ALM ELM LRM 44 1 0 24 Feb 2025
Optimizing Model Selection for Compound AI Systems Lingjiao Chen Jared Quincy Davis Boris Hanin Peter Bailis Matei A. Zaharia James Zou Ion Stoica 63 1 0 20 Feb 2025
EvoFlow: Evolving Diverse Agentic Workflows On The Fly Guibin Zhang Kaijie Chen Guancheng Wan Heng Chang Hong Cheng Kaidi Wang Shuyue Hu Lei Bai 97 2 0 11 Feb 2025
Smoothie: Label Free Language Model Routing Neel Guha Mayee F. Chen Trevor Chow Ishan S. Khare Christopher Ré 71 3 0 06 Dec 2024
Learning to Route LLMs with Confidence Tokens Yu-Neng Chuang Helen Zhou Prathusha Kameswara Sarma Parikshit Gopalan John Boccio Sara Bolouki Xia Hu 35 8 0 17 Oct 2024
A Unified Approach to Routing and Cascading for LLMs Jasper Dekoninck Maximilian Baader Martin Vechev 60 2 0 14 Oct 2024
Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmap Ahmed E. Hassan G. Oliva Dayi Lin Boyuan Chen Zhen Ming Jiang 47 5 0 08 Oct 2024
EmbedLLM: Learning Compact Representations of Large Language Models Richard Zhuang Tianhao Wu Zhaojin Wen Andrew Li Jiantao Jiao Kannan Ramchandran AIFin 37 1 0 03 Oct 2024
Efficiently Deploying LLMs with Controlled Risk Michael J. Zellinger Matt Thomson 41 1 0 03 Oct 2024
RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Models Shuhao Chen Weisen Jiang Baijiong Lin James T. Kwok Yu Zhang RALM MQ 48 5 0 30 Sep 2024
Eagle: Efficient Training-Free Router for Multi-LLM Inference Zesen Zhao Shuowei Jin Z. Morley Mao 28 3 0 23 Sep 2024
What is the Role of Small Models in the LLM Era: A Survey Lihu Chen Gaël Varoquaux ALM 66 23 0 10 Sep 2024
MetaLLM: A High-performant and Cost-efficient Dynamic Framework for Wrapping LLMs Quang H. Nguyen Duy C. Hoang Juliette Decugis Saurav Manchanda Nitesh Chawla Khoa D. Doan Khoa D. Doan 45 8 0 15 Jul 2024
RouteLLM: Learning to Route LLMs with Preference Data Isaac Ong Amjad Almahairi Vincent Wu Wei-Lin Chiang Tianhao Wu Joseph E. Gonzalez M. W. Kadous Ion Stoica 81 77 0 26 Jun 2024
LLM-dCache: Improving Tool-Augmented LLMs with GPT-Driven Localized Data Caching Simranjit Singh Michael Fore Andreas Karatzas Chaehong Lee Yanan Jian Longfei Shangguan Fuxun Yu Iraklis Anagnostopoulos Dimitrios Stamoulis RALM 35 2 0 10 Jun 2024
Cascade-Aware Training of Language Models Congchao Wang Sean Augenstein Keith Rush Wittawat Jitkrittum Harikrishna Narasimhan A. S. Rawat A. Menon Alec Go 36 4 0 29 May 2024
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 366 3,029 0 22 Mar 2023
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 349 3,314 0 21 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 447 8,650 0 28 Jan 2022