Title
Qwen3 Technical Report An Yang A. Li Baosong Yang Beichen Zhang Binyuan Hui ... Zekun Wang Zeyu Cui Zhenru Zhang Zhenhong Zhou Zihan Qiu LLMAG OSLM LRM 45 0 0 14 May 2025
Reproducibility Study of "Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents" Pedro M. P. Curvo Mara Dragomir Salvador Torpes Mohammadmahdi Rahimi LLMAG 18 0 0 14 May 2025
Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures Chenggang Zhao Chengqi Deng Chong Ruan Damai Dai Huazuo Gao ... Wenfeng Liang Ying He Yishuo Wang Yuxuan Liu Y. X. Wei MoE 41 0 0 14 May 2025
Aquarius: A Family of Industry-Level Video Generation Models for Marketing Scenarios Huafeng Shi Jianzhong Liang Rongchang Xie Xian Wu Cheng Chen Chang Liu VGen 22 0 0 14 May 2025
Large Language Models for Computer-Aided Design: A Survey Licheng Zhang Bach Le Naveed Akhtar Siew-Kei Lam Tuan Ngo 3DV AI4CE 40 0 0 13 May 2025
UMoE: Unifying Attention and FFN with Shared Experts Yuanhang Yang Chaozheng Wang Jing Li MoE 29 0 0 12 May 2025
The power of fine-grained experts: Granularity boosts expressivity in Mixture of Experts Enric Boix Adserà Philippe Rigollet MoE 28 0 0 11 May 2025
Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free Zihan Qiu Zhaoxiang Wang Bo Zheng Zeyu Huang Kaiyue Wen ... Fei Huang Suozhi Huang Dayiheng Liu Jingren Zhou Junyang Lin MoE 31 0 0 10 May 2025
FloE: On-the-Fly MoE Inference on Memory-constrained GPU Yuxin Zhou Zheng Li J. Zhang Jue Wang Yanjie Wang Zhongle Xie Ke Chen Lidan Shou MoE 52 0 0 09 May 2025
Scalable LLM Math Reasoning Acceleration with Low-rank Distillation Harry Dong Bilge Acun Beidi Chen Yuejie Chi LRM 34 0 0 08 May 2025
Faster MoE LLM Inference for Extremely Large Models Haoqi Yang Luohe Shi Qiwei Li Zuchao Li Ping Wang Bo Du Mengjia Shen Hai Zhao MoE 68 0 0 06 May 2025
MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance Xing Hu Zhixuan Chen Dawei Yang Zukang Xu Chen Xu Zhihang Yuan Sifan Zhou Jiangyong Yu MoE MQ 44 0 0 02 May 2025
MambaMoE: Mixture-of-Spectral-Spatial-Experts State Space Model for Hyperspectral Image Classification Yichu Xu Di Wang Hongzan Jiao Li Zhang Lefei Zhang Mamba 83 0 0 29 Apr 2025
ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Driving Renju Feng Ning Xi Duanfeng Chu Rukang Wang Zejian Deng Anzheng Wang Liping Lu Jinxiang Wang Yanjun Huang 51 0 0 28 Apr 2025
TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate A. Zandieh Majid Daliri Majid Hadian Vahab Mirrokni MQ 74 0 0 28 Apr 2025
BadMoE: Backdooring Mixture-of-Experts LLMs via Optimizing Routing Triggers and Infecting Dormant Experts Qingyue Wang Qi Pang Xixun Lin Shuai Wang Daoyuan Wu MoE 62 0 0 24 Apr 2025
Distribution-aware Forgetting Compensation for Exemplar-Free Lifelong Person Re-identification Shiben Liu Huijie Fan Qiang Wang Baojie Fan Yandong Tang Liangqiong Qu OOD CLL 57 0 0 21 Apr 2025
MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core Dennis Liu Zijie Yan Xin Yao Tong Liu V. Korthikanti ... Jiajie Yao Chandler Zhou David Wu Xipeng Li J. Yang MoE 70 0 0 21 Apr 2025
RouterKT: Mixture-of-Experts for Knowledge Tracing Han Liao Shuaishuai Zu 43 0 0 11 Apr 2025
Investigating and Mitigating Stereotype-aware Unfairness in LLM-based Recommendations Zihuai Zhao Wenqi Fan Yao Wu Qing Li 80 1 0 05 Apr 2025
Advancing MoE Efficiency: A Collaboration-Constrained Routing (C2R) Strategy for Better Expert Parallelism Design Mohan Zhang Pingzhi Li Jie Peng Mufan Qiu Tianlong Chen MoE 50 0 0 02 Apr 2025
Exploiting Mixture-of-Experts Redundancy Unlocks Multimodal Generative Abilities Raman Dutt Harleen Hanspal Guoxuan Xia Petru-Daniel Tudosiu Alexander Black Yongxin Yang Jingyu Sun Sarah Parisot MoE 43 0 0 28 Mar 2025
MoQa: Rethinking MoE Quantization with Multi-stage Data-model Distribution Awareness Zihao Zheng Xiuping Cui Size Zheng Maoliang Li Jiayu Chen Yun Liang Xiang Chen MQ MoE 64 0 0 27 Mar 2025
A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications Siyuan Mu Sen Lin MoE 153 2 0 10 Mar 2025
eMoE: Task-aware Memory Efficient Mixture-of-Experts-Based (MoE) Model Inference Suraiya Tairin Shohaib Mahmud Haiying Shen Anand Iyer MoE 182 0 0 10 Mar 2025
Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs Ling Team B. Zeng Chenyu Huang Chao Zhang Changxin Tian ... Zhaoxin Huan Zujie Wen Zhenhang Sun Zhuoxuan Du Z. He MoE ALM 109 2 0 07 Mar 2025
CoServe: Efficient Collaboration-of-Experts (CoE) Model Inference with Limited Memory Jiashun Suo Xiaojian Liao Limin Xiao Li Ruan Jinquan Wang Xiao Su Zhisheng Huo 69 0 0 04 Mar 2025
Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization Taishi Nakamura Takuya Akiba Kazuki Fujii Yusuke Oda Rio Yokota Jun Suzuki MoMe MoE 94 1 0 26 Feb 2025
Filtered not Mixed: Stochastic Filtering-Based Online Gating for Mixture of Large Language Models Raeid Saqur Anastasis Kratsios Florian Krach Yannick Limmer Jacob-Junqi Tian John Willes Blanka Horvath Frank Rudzicz MoE 53 0 0 24 Feb 2025
MoM: Linear Sequence Modeling with Mixture-of-Memories Jusen Du Weigao Sun Disen Lan Jiaxi Hu Yu-Xi Cheng KELM 75 3 0 19 Feb 2025
Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysis J. Zhao Hao Wu Miao Zhang Yuzhang Shang Xuebo Liu Yaowei Wang Min Zhang Liqiang Nie MQ 60 1 0 18 Feb 2025
DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs Minxuan Lv Zhenpeng Su Leiyu Pan Yizhe Xiong Zijia Lin ... Guiguang Ding Cheng Luo Di Zhang Kun Gai Songlin Hu MoE 41 0 0 18 Feb 2025
Efficient Long-Decoding Inference with Reasoning-Aware Attention Sparsity Junhao Hu Wenrui Huang Weidong Wang Zhenwen Li Tiancheng Hu Zhixia Liu Xusheng Chen Tao Xie Yizhou Shan LRM 51 0 0 16 Feb 2025
Steel-LLM:From Scratch to Open Source -- A Personal Journey in Building a Chinese-Centric LLM Qingshui Gu Shu Li Tianyu Zheng Zhaoxiang Zhang 228 0 0 10 Feb 2025
Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline Zhiyuan Fang Yuegui Huang Zicong Hong Yufeng Lyu Wuhui Chen Yue Yu Fan Yu Zibin Zheng MoE 48 0 0 09 Feb 2025
Dynamic Chain-of-Thought: Towards Adaptive Deep Reasoning Libo Wang LRM 159 1 0 07 Feb 2025
Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning Peizhuang Cong Wenpu Liu Wenhan Yu Haochen Zhao Tong Yang ALM MoE 81 0 0 06 Feb 2025
Brief analysis of DeepSeek R1 and its implications for Generative AI Sarah Mercer Samuel Spillard Daniel P. Martin 76 13 0 04 Feb 2025
MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEs Yuhang Zhou Giannis Karamanolakis Victor Soto Anna Rumshisky Mayank Kulkarni Furong Huang Wei Ai Jianhua Lu MoMe 108 0 0 03 Feb 2025
Soup-of-Experts: Pretraining Specialist Models via Parameters Averaging Pierre Ablin Angelos Katharopoulos Skyler Seto David Grangier MoMe 50 0 0 03 Feb 2025
Position: AI Scaling: From Up to Down and Out Yunke Wang Yanxi Li Chang Xu HAI 88 2 0 02 Feb 2025
UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Models Xin Xu Qiyun Xu Tong Xiao Tianhao Chen Yuchen Yan Jiaxin Zhang Shizhe Diao Can Yang Yang Wang ELM LRM AI4CE 108 3 0 01 Feb 2025
mHumanEval -- A Multilingual Benchmark to Evaluate Large Language Models for Code Generation Nishat Raihan Antonios Anastasopoulos Marcos Zampieri ELM 43 6 0 28 Jan 2025
Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models Zihan Qiu Zeyu Huang Jian Xu Kaiyue Wen Zhaoxiang Wang Rui Men Ivan Titov Dayiheng Liu Jingren Zhou Junyang Lin MoE 57 6 0 21 Jan 2025
Yi-Lightning Technical Report 01. AI : Alan Wake Albert Wang Bei Chen ... Yuxuan Sha Zhaodong Yan Zhiyuan Liu Zirui Zhang Zonghong Dai OSLM 102 3 0 02 Dec 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies Liwen Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 49 3 0 24 Oct 2024
Mixture of Parrots: Experts improve memorization more than reasoning Samy Jelassi Clara Mohri David Brandfonbrener Alex Gu Nikhil Vyas Nikhil Anand David Alvarez-Melis Yuanzhi Li Sham Kakade Eran Malach MoE 33 4 0 24 Oct 2024
ViMoE: An Empirical Study of Designing Vision Mixture-of-Experts Xumeng Han Longhui Wei Zhiyang Dou Zipeng Wang Chenhui Qiang Xin He Yingfei Sun Zhenjun Han Qi Tian MoE 45 3 0 21 Oct 2024
CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts Zhenpeng Su Xing Wu Zijia Lin Yizhe Xiong Minxuan Lv Guangyuan Ma Hui Chen Songlin Hu Guiguang Ding MoE 29 3 0 21 Oct 2024
Router-Tuning: A Simple and Effective Approach for Enabling Dynamic-Depth in Transformers Shwai He Tao Ge Guoheng Sun Bowei Tian Xiaoyang Wang Ang Li MoE 54 1 0 17 Oct 2024