Training and Serving System of Foundation Models: A Comprehensive Survey

Training and Serving System of Foundation Models: A Comprehensive Survey

5 January 2024

Papers citing "Training and Serving System of Foundation Models: A Comprehensive Survey"

9 / 9 papers shown

Title
Robustness Tokens: Towards Adversarial Robustness of Transformers Brian Pulfer Yury Belousov S. Voloshynovskiy AAML 45 0 0 13 Mar 2025
Mobile Edge Intelligence for Large Language Models: A Contemporary Survey Guanqiao Qu Qiyuan Chen Wei Wei Zheng Lin Xianhao Chen Kaibin Huang 42 43 0 09 Jul 2024
Green Edge AI: A Contemporary Survey Yuyi Mao X. Yu Kaibin Huang Ying-Jun Angela Zhang Jun Zhang 41 17 0 01 Dec 2023
ZeRO++: Extremely Efficient Collective Communication for Giant Model Training Guanhua Wang Heyang Qin S. A. Jacobs Connor Holmes Samyam Rajbhandari Olatunji Ruwase Feng Yan Lei Yang Yuxiong He VLM 59 57 0 16 Jun 2023
PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing Xiaozhe Ren Pingyi Zhou Xinfan Meng Xinjing Huang Yadao Wang ... Jiansheng Wei Xin Jiang Teng Su Qun Liu Jun Yao ALM MoE 75 60 0 20 Mar 2023
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU Ying Sheng Lianmin Zheng Binhang Yuan Zhuohan Li Max Ryabinin ... Joseph E. Gonzalez Percy Liang Christopher Ré Ion Stoica Ce Zhang 149 369 0 13 Mar 2023
Varuna: Scalable, Low-cost Training of Massive Deep Learning Models Sanjith Athlur Nitika Saran Muthian Sivathanu Ramachandran Ramjee Nipun Kwatra GNN 31 80 0 07 Nov 2021
Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines Shigang Li Torsten Hoefler GNN AI4CE LRM 77 131 0 14 Jul 2021
ZeRO-Offload: Democratizing Billion-Scale Model Training Jie Ren Samyam Rajbhandari Reza Yazdani Aminabadi Olatunji Ruwase Shuangyang Yang Minjia Zhang Dong Li Yuxiong He MoE 177 414 0 18 Jan 2021