Model Parallelism on Distributed Infrastructure: A Literature Review from Theory to LLM Case-Studies

6 March 2024

Papers citing "Model Parallelism on Distributed Infrastructure: A Literature Review from Theory to LLM Case-Studies"

4 / 4 papers shown

Title
Watermarking Large Language Models and the Generated Content: Opportunities and Challenges Ruisi Zhang F. Koushanfar WaLM 47 0 0 24 Oct 2024
TiMePReSt: Time and Memory Efficient Pipeline Parallel DNN Training with Removed Staleness Ankita Dutta Nabendu Chaki Rajat K. De 27 0 0 18 Oct 2024
A Survey of Language-Based Communication in Robotics William Hunt Sarvapali D. Ramchurn Mohammad D. Soorati LM&Ro 65 12 0 06 Jun 2024
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,821 0 17 Sep 2019