kDecay: Just adding k-decay items on Learning-Rate Schedule to improve Neural Networks

13 April 2020

Papers citing "kDecay: Just adding k-decay items on Learning-Rate Schedule to improve Neural Networks"

9 / 9 papers shown

Title
X-volution: On the unification of convolution and self-attention Xuanhong Chen Hang Wang Bingbing Ni ViT 42 25 0 04 Jun 2021
AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly Yuchen Jin Dinesh Manocha Liangyu Zhao Yibo Zhu Chuanxiong Guo Marco Canini Arvind Krishnamurthy 71 19 0 22 May 2021
A Comprehensive Survey on Graph Neural Networks Zonghan Wu Shirui Pan Fengwen Chen Guodong Long Chengqi Zhang Philip S. Yu FaML GNN AI4TS AI4CE 783 8,566 0 03 Jan 2019
SGDR: Stochastic Gradient Descent with Warm Restarts I. Loshchilov Frank Hutter ODL 350 8,174 0 13 Aug 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 351 8,000 0 23 May 2016
Cyclical Learning Rates for Training Neural Networks L. Smith ODL 230 2,537 0 03 Jun 2015
Image Super-Resolution Using Deep Convolutional Networks Chao Dong Chen Change Loy Kaiming He Xiaoou Tang SupR 155 8,091 0 31 Dec 2014
Network In Network Min Lin Qiang Chen Shuicheng Yan 294 6,284 0 16 Dec 2013
ADADELTA: An Adaptive Learning Rate Method Matthew D. Zeiler ODL 163 6,632 0 22 Dec 2012