Accelerated Large Batch Optimization of BERT Pretraining in 54 minutes

Accelerated Large Batch Optimization of BERT Pretraining in 54 minutes

24 June 2020

Sheng Zha

Papers citing "Accelerated Large Batch Optimization of BERT Pretraining in 54 minutes"

4 / 4 papers shown

Title
LLM Content Moderation and User Satisfaction: Evidence from Response Refusals in Chatbot Arena Stefan Pasch 40 0 0 04 Jan 2025
DropCompute: simple and more robust distributed synchronous training via compute variance reduction Niv Giladi Shahar Gottlieb Moran Shkolnik A. Karnieli Ron Banner Elad Hoffer Kfir Y. Levy Daniel Soudry 40 2 0 18 Jun 2023
MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud Zhen Zhang Shuai Zheng Yida Wang Justin Chiu George Karypis Trishul Chilimbi Mu Li Xin Jin 23 39 0 30 Apr 2022
Optimal Distributed Online Prediction using Mini-Batches O. Dekel Ran Gilad-Bachrach Ohad Shamir Lin Xiao 182 683 0 07 Dec 2010