PreMoe: Lightening MoEs on Constrained Memory by Expert Pruning and Retrieval

23 May 2025

Papers citing "PreMoe: Lightening MoEs on Constrained Memory by Expert Pruning and Retrieval"

3 / 3 papers shown

Title
Pangu Ultra MoE: How to Train Your Big MoE on Ascend NPUs Yehui Tang Yichun Yin Yaoyuan Wang Hang Zhou Yu Pan ... Zhe Liu Zhicheng Liu Zhuowen Tu Zilin Ding Zongyuan Zhan MoE 113 2 0 07 May 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 395 2,028 0 22 Jan 2025
FuseGPT: Learnable Layers Fusion of Generative Pre-trained Transformers Zehua Pei Hui-Ling Zhen Xianzhi Yu Sinno Jialin Pan Mingxuan Yuan Bei Yu AI4CE 249 3 0 21 Nov 2024