Coupling Adaptive Batch Sizes with Learning Rates

Coupling Adaptive Batch Sizes with Learning Rates

15 December 2016

Papers citing "Coupling Adaptive Batch Sizes with Learning Rates"

14 / 14 papers shown

Title
Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descent Hikaru Umeda Hideaki Iiduka 67 2 0 17 Feb 2025
Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism Tim Tsz-Kit Lau Weijian Li Chenwei Xu Han Liu Mladen Kolar 147 0 0 30 Dec 2024
Multiple Importance Sampling for Stochastic Gradient Estimation Corentin Salaün Xingchang Huang Iliyan Georgiev Niloy J. Mitra Gurprit Singh 32 1 0 22 Jul 2024
Shot Optimization in Quantum Machine Learning Architectures to Accelerate Training Koustubh Phalak Swaroop Ghosh 36 5 0 21 Apr 2023
Dynamic Batch Adaptation Cristian Simionescu George Stoica Robert Herscovici ODL 19 1 0 01 Aug 2022
Dynamic Hard Pruning of Neural Networks at the Edge of the Internet Lorenzo Valerio F. M. Nardini A. Passarella R. Perego 12 12 0 17 Nov 2020
Balancing Rates and Variance via Adaptive Batch-Size for Stochastic Optimization Problems Zhan Gao Alec Koppel Alejandro Ribeiro 14 10 0 02 Jul 2020
Dynamically Adjusting Transformer Batch Size by Monitoring Gradient Direction Change Hongfei Xu Josef van Genabith Deyi Xiong Qiuhui Liu 6 10 0 05 May 2020
Adaptive Regularization via Residual Smoothing in Deep Learning Optimization Jung-Kyun Cho Junseok Kwon Byung-Woo Hong 28 1 0 23 Jul 2019
Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD Jianyu Wang Gauri Joshi FedML 25 231 0 19 Oct 2018
Continuous-time Models for Stochastic Optimization Algorithms Antonio Orvieto Aurelien Lucchi 13 31 0 05 Oct 2018
AdaBatch: Adaptive Batch Sizes for Training Deep Neural Networks Aditya Devarakonda Maxim Naumov M. Garland ODL 19 136 0 06 Dec 2017
Advances in Variational Inference Cheng Zhang Judith Butepage Hedvig Kjellström Stephan Mandt BDL 38 684 0 15 Nov 2017
Probabilistic Line Searches for Stochastic Optimization Maren Mahsereci Philipp Hennig ODL 21 126 0 10 Feb 2015