On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019

Xiaodong Liu

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 373 papers shown

Title
Spherical coordinates transformation pre-processing in Deep Convolution Neural Networks for brain tumor segmentation in MRI C. Russo Sidong Liu A. Di Ieva 52 22 0 17 Aug 2020
SynDistNet: Self-Supervised Monocular Fisheye Camera Distance Estimation Synergized with Semantic Segmentation for Autonomous Driving V. Kumar Marvin Klingner S. Yogamani Stefan Milz Tim Fingscheidt Patrick Mäder MDE 28 81 0 10 Aug 2020
Two-Stage Deep Learning for Accelerated 3D Time-of-Flight MRA without Matched Training Data Hyungjin Chung Eunju Cha Leonard Sunwoo J. C. Ye MedIm 39 26 0 04 Aug 2020
Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation Elad Richardson Yuval Alaluf Or Patashnik Yotam Nitzan Yaniv Azar Stav Shapiro Daniel Cohen-Or 79 1,100 0 03 Aug 2020
HMQ: Hardware Friendly Mixed Precision Quantization Block for CNNs H. Habi Roy H. Jennings Arnon Netzer MQ 34 65 0 20 Jul 2020
Tracking-by-Trackers with a Distilled and Reinforced Model Matteo Dunnhofer N. Martinel C. Micheloni VOT OffRL 32 4 0 08 Jul 2020
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 50 162 0 03 Jul 2020
Vehicle Attribute Recognition by Appearance: Computer Vision Methods for Vehicle Type, Make and Model Classification Xingyang Ni H. Huttunen CVBM 19 20 0 29 Jun 2020
Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures Julien Launay Iacopo Poli Franccois Boniface Florent Krzakala 41 63 0 23 Jun 2020
MaxVA: Fast Adaptation of Step Sizes by Maximizing Observed Variance of Gradients Chenfei Zhu Yu Cheng Zhe Gan Furong Huang Jingjing Liu Tom Goldstein ODL 40 2 0 21 Jun 2020
Match $^2$ : A Matching over Matching Model for Similar Question Identification Zizhen Wang Yixing Fan Jiafeng Guo Liu Yang Ruqing Zhang Yanyan Lan Xueqi Cheng Hui Jiang Xiaozhao Wang 21 15 0 21 Jun 2020
Training (Overparametrized) Neural Networks in Near-Linear Time Jan van den Brand Binghui Peng Zhao Song Omri Weinstein ODL 39 83 0 20 Jun 2020
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation Robert Mansel Gower Othmane Sebbouh Nicolas Loizou 32 74 0 18 Jun 2020
Categorical Normalizing Flows via Continuous Transformations Phillip Lippe E. Gavves BDL 31 43 0 17 Jun 2020
Deep Learning based Segmentation of Fish in Noisy Forward Looking MBES Images J. H. Christensen L. V. Mogensen Ole Ravn 28 17 0 16 Jun 2020
The Limit of the Batch Size Yang You Yuhui Wang Huan Zhang Zhao-jie Zhang J. Demmel Cho-Jui Hsieh 16 15 0 15 Jun 2020
Getting a CLUE: A Method for Explaining Uncertainty Estimates Javier Antorán Umang Bhatt T. Adel Adrian Weller José Miguel Hernández-Lobato UQCV BDL 56 112 0 11 Jun 2020
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines Marius Mosbach Maksym Andriushchenko Dietrich Klakow 31 354 0 08 Jun 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 69 2,652 0 05 Jun 2020
sEMG Gesture Recognition with a Simple Model of Attention David Josephs Carson Drake Andrew Heroy John Santerre 20 48 0 05 Jun 2020
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning Z. Yao A. Gholami Sheng Shen Mustafa Mustafa Kurt Keutzer Michael W. Mahoney ODL 44 278 0 01 Jun 2020
Universal Lesion Detection by Learning from Multiple Heterogeneously Labeled Datasets K. Yan Jinzheng Cai Adam P. Harrison D. Jin Jing Xiao Le Lu 23 17 0 28 May 2020
Detecting Scatteredly-Distributed, Small, andCritically Important Objects in 3D OncologyImaging via Decision Stratification Zhuotun Zhu K. Yan D. Jin Jinzheng Cai T. Ho ... Chun-Hung Chao X. Ye Jing Xiao Alan Yuille Le Lu 30 10 0 27 May 2020
Few-shot Compositional Font Generation with Dual Memory Junbum Cha Sanghyuk Chun Gayoung Lee Bado Lee Seonghyeon Kim Hwalsuk Lee 32 78 0 21 May 2020
Quasi-Periodic Parallel WaveGAN Vocoder: A Non-autoregressive Pitch-dependent Dilated Convolution Model for Parametric Speech Generation Yi-Chiao Wu Tomoki Hayashi T. Okamoto Hisashi Kawai Tomoki Toda 39 4 0 18 May 2020
DiscreTalk: Text-to-Speech as a Machine Translation Problem Tomoki Hayashi Shinji Watanabe 32 32 0 12 May 2020
BlackBox: Generalizable Reconstruction of Extremal Values from Incomplete Spatio-Temporal Data T. Ivek Domagoj Vlah 48 4 0 30 Apr 2020
Adversarial Training for Large Neural Language Models Xiaodong Liu Hao Cheng Pengcheng He Weizhu Chen Yu Wang Hoifung Poon Jianfeng Gao AAML 57 183 0 20 Apr 2020
Organ at Risk Segmentation for Head and Neck Cancer using Stratified Learning and Neural Architecture Search Dazhou Guo D. Jin Zhuotun Zhu T. Ho Adam P. Harrison Chun-Hung Chao Jing Xiao Alan Yuille Chien-Yu Lin Le Lu 32 59 0 17 Apr 2020
A Cross-Stitch Architecture for Joint Registration and Segmentation in Adaptive Radiotherapy Laurens Beljaards Mohamed S. Elmahdy F. Verbeek Marius Staring 3DV 36 15 0 17 Apr 2020
Show Us the Way: Learning to Manage Dialog from Demonstrations Gabriel Gordon-Hall P. Gorinski Gerasimos Lampouras Ignacio Iacobacci OffRL 27 11 0 17 Apr 2020
An Adaptive Intelligence Algorithm for Undersampled Knee MRI Reconstruction Nicola Pezzotti Sahar Yousefi Mohamed S. Elmahdy J. V. Gemert C. Schulke ... Sergey Kastryulin B. Lelieveldt M. Osch E. Weerdt Marius Staring 27 97 0 15 Apr 2020
iTAML: An Incremental Task-Agnostic Meta-learning Approach Jathushan Rajasegaran Salman Khan Munawar Hayat Fahad Shahbaz Khan M. Shah CLL OOD 30 154 0 25 Mar 2020
End-to-End Deep Diagnosis of X-ray Images Kudaibergen Urinbayev Yerassyl Orazbek Yernur Nurambek A. Mirzakhmetov H. A. Varol MedIm 28 12 0 19 Mar 2020
Getting to 99% Accuracy in Interactive Segmentation Marco Forte Brian L. Price Scott D. Cohen N. Xu Franccois Pitié 53 37 0 17 Mar 2020
Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation Huiyu Wang Yukun Zhu Bradley Green Hartwig Adam Alan Yuille Liang-Chieh Chen 3DPC 35 663 0 17 Mar 2020
$F$ , $B$ , Alpha Matting Marco Forte François Pitié 30 84 0 17 Mar 2020
3D dynamic hand gestures recognition using the Leap Motion sensor and convolutional neural networks Katia Lupinetti A. Ranieri F. Giannini M. Monti SLR 12 28 0 03 Mar 2020
Iterative Averaging in the Quest for Best Test Error Diego Granziol Xingchen Wan Samuel Albanie Stephen J. Roberts 32 3 0 02 Mar 2020
Uncertainty-Gated Stochastic Sequential Model for EHR Mortality Prediction E. Jun A. Mulyadi Jaehun Choi Heung-Il Suk 30 20 0 02 Mar 2020
Using a thousand optimization tasks to learn hyperparameter search strategies Luke Metz Niru Maheswaranathan Ruoxi Sun C. Freeman Ben Poole Jascha Narain Sohl-Dickstein 25 46 0 27 Feb 2020
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence Nicolas Loizou Sharan Vaswani I. Laradji Simon Lacoste-Julien 29 182 0 24 Feb 2020
Meta-learning Extractors for Music Source Separation David Samuel Aditya Ganeshan Jason Naradowsky 32 61 0 17 Feb 2020
Electricity Theft Detection with self-attention Paulo Finardi Israel Campiotti Gustavo Plensack R. Souza Rodrigo Nogueira G. Pinheiro R. Lotufo 22 29 0 14 Feb 2020
Liver Segmentation in Abdominal CT Images via Auto-Context Neural Network and Self-Supervised Contour Attention Minyoung Chung Jingyu Lee Jeongjin Lee Y. Shin MedIm 21 29 0 14 Feb 2020
LaProp: Separating Momentum and Adaptivity in Adam Liu Ziyin Zhikang T.Wang Masahito Ueda ODL 18 18 0 12 Feb 2020
On Layer Normalization in the Transformer Architecture Ruibin Xiong Yunchang Yang Di He Kai Zheng Shuxin Zheng Chen Xing Huishuai Zhang Yanyan Lan Liwei Wang Tie-Yan Liu AI4CE 61 954 0 12 Feb 2020
Short Term Blood Glucose Prediction based on Continuous Glucose Monitoring Data Ali Mohebbi Alexander R. Johansen Nicklas Hansen Peter Ebert Christensen J. Tarp M. L. Jensen Henrik Bengtsson Morten Mørup 11 23 0 06 Feb 2020
SAUNet: Shape Attentive U-Net for Interpretable Medical Image Segmentation Jesse Sun Fatemeh Darbeha M. Zaidi Bo Wang AAML 19 110 0 21 Jan 2020
Data-Driven Permanent Magnet Temperature Estimation in Synchronous Motors with Supervised Machine Learning Wilhelm Kirchgässner Oliver Wallscheid J. Böcker 25 69 0 17 Jan 2020