Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models

Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models

21 May 2025

ArXiv (abs)PDF HTML

Papers citing "Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models"

16 / 16 papers shown

Title
fMoE: Fine-Grained Expert Offloading for Large Mixture-of-Experts Serving Hanfei Yu Xingqi Cui Huatian Zhang Han Wang Hao Wang MoE 103 1 0 07 Feb 2025
DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inference Yujie Zhang Shivam Aggarwal T. Mitra MoE 125 1 0 16 Dec 2024
Mixture of Cache-Conditional Experts for Efficient Mobile Device Inference Andrii Skliar T. V. Rozendaal Romain Lepert Todor Boinovski M. V. Baalen Markus Nagel Paul N. Whatmough B. Bejnordi MoE 115 2 0 27 Nov 2024
ProMoE: Fast MoE-based LLM Serving using Proactive Caching Xiaoniu Song Zihang Zhong Rong Chen Haibo Chen MoE 110 6 0 29 Oct 2024
Mixture Compressor for Mixture-of-Experts LLMs Gains More Wei Huang Yue Liao Jianhui Liu Ruifei He Haoru Tan Shiming Zhang Hongsheng Li Si Liu Xiaojuan Qi MoE 89 4 0 08 Oct 2024
GRIN: GRadient-INformed MoE Liyuan Liu Young Jin Kim Shuohang Wang Chen Liang Yelong Shen ... Chenruidong Zhang Jilong Xue Hany Awadalla Jianfeng Gao Weizhu Chen MoE 64 7 0 18 Sep 2024
Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler Songlin Yang Matthew Stallone Mayank Mishra Gaoyuan Zhang Shawn Tan Aditya Prasad Adriana Meza Soria David D. Cox Yikang Shen 96 15 0 23 Aug 2024
Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models Keisuke Kamahori Tian Tang Yile Gu Kan Zhu Baris Kasikci 122 24 0 10 Feb 2024
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models Fuzhao Xue Zian Zheng Yao Fu Jinjie Ni Zangwei Zheng Wangchunshu Zhou Yang You MoE 82 103 0 29 Jan 2024
Mixtral of Experts Albert Q. Jiang Alexandre Sablayrolles Antoine Roux A. Mensch Blanche Savary ... Théophile Gervet Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LLMAG 155 1,117 0 08 Jan 2024
Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy Pingzhi Li Zhenyu Zhang Prateek Yadav Yi-Lin Sung Yu Cheng Mohit Bansal Tianlong Chen MoMe 69 39 0 02 Oct 2023
Mixture of Attention Heads: Selecting Attention Heads Per Token Xiaofeng Zhang Songlin Yang Zeyu Huang Jie Zhou Wenge Rong Zhang Xiong MoE 142 47 0 11 Oct 2022
No Language Left Behind: Scaling Human-Centered Machine Translation Nllb team Marta R. Costa-jussá James Cross Onur cCelebi Maha Elbayad ... Alexandre Mourachko C. Ropers Safiyyah Saleem Holger Schwenk Jeff Wang MoE 228 1,266 0 11 Jul 2022
Mixture-of-Experts with Expert Choice Routing Yan-Quan Zhou Tao Lei Han-Chu Liu Nan Du Yanping Huang Vincent Zhao Andrew M. Dai Zhifeng Chen Quoc V. Le James Laudon MoE 301 367 0 18 Feb 2022
DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale Samyam Rajbhandari Conglong Li Z. Yao Minjia Zhang Reza Yazdani Aminabadi A. A. Awan Jeff Rasley Yuxiong He 110 302 0 14 Jan 2022
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Zhiwen Chen MoE 124 1,191 0 30 Jun 2020