On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019

Xiaodong Liu

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 373 papers shown

Title
LogAvgExp Provides a Principled and Performant Global Pooling Operator S. Lowe Thomas Trappenberg Sageev Oore FAtt 26 2 0 02 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 35 14 0 01 Nov 2021
Whole Brain Segmentation with Full Volume Neural Network Yeshu Li Jianwei Cui Yilun Sheng Xiao Liang Jingdong Wang E. Chang Yan Xu 56 11 0 29 Oct 2021
Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization Tao Sun Huaming Ling Zuoqiang Shi Dongsheng Li Bao Wang ODL 32 13 0 18 Oct 2021
Hierarchical Curriculum Learning for AMR Parsing Peiyi Wang Liang Chen Tianyu Liu Damai Dai Yunbo Cao Baobao Chang Zhifang Sui 45 15 0 15 Oct 2021
Dynamic Inference with Neural Interpreters Nasim Rahaman Muhammad Waleed Gondal S. Joshi Peter V. Gehler Yoshua Bengio Francesco Locatello Bernhard Schölkopf 65 31 0 12 Oct 2021
Vision Transformer based COVID-19 Detection using Chest X-rays Koushik Sivarama Krishnan Karthik Sivarama Krishnan ViT MedIm 41 55 0 09 Oct 2021
Taming Sparsely Activated Transformer with Stochastic Experts Simiao Zuo Xiaodong Liu Jian Jiao Young Jin Kim Hany Hassan Ruofei Zhang T. Zhao Jianfeng Gao MoE 44 110 0 08 Oct 2021
Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect Yuqing Wang Minshuo Chen T. Zhao Molei Tao AI4CE 64 40 0 07 Oct 2021
A Hybrid Spatial-temporal Deep Learning Architecture for Lane Detection Yongqi Dong S. Patil B. Arem Haneen Farah 36 38 0 05 Oct 2021
Multilingual AMR Parsing with Noisy Knowledge Distillation Deng Cai Xin Li Jackie Chun-Sing Ho Lidong Bing W. Lam 29 18 0 30 Sep 2021
AdaInject: Injection Based Adaptive Gradient Descent Optimizers for Convolutional Neural Networks S. Dubey S. H. Shabbeer Basha S. Singh B. B. Chaudhuri ODL 53 9 0 26 Sep 2021
Commonsense Knowledge in Word Associations and ConceptNet Chunhua Liu Trevor Cohn Lea Frermann 39 7 0 20 Sep 2021
Towards Joint Intent Detection and Slot Filling via Higher-order Attention Dongsheng Chen Zhiqi Huang Xian Wu Shen Ge Yuexian Zou 39 20 0 18 Sep 2021
TrouSPI-Net: Spatio-temporal attention on parallel atrous convolutions and U-GRUs for skeletal pedestrian crossing prediction Joseph Gesnouin Steve Pechberti B. Stanciulescu Fabien Moutarde 53 22 0 02 Sep 2021
Iterative Filter Adaptive Network for Single Image Defocus Deblurring Junyong Lee Hyeongseok Son Jaesung Rim Sunghyun Cho Seungyong Lee 35 121 0 31 Aug 2021
HAN: Higher-order Attention Network for Spoken Language Understanding Dongsheng Chen Zhiqi Huang Yuexian Zou 29 1 0 26 Aug 2021
MimicBot: Combining Imitation and Reinforcement Learning to win in Bot Bowl Nicola Pezzotti 35 1 0 21 Aug 2021
Logit Attenuating Weight Normalization Aman Gupta R. Ramanath Jun Shi Anika Ramachandran Sirou Zhou Mingzhou Zhou S. Keerthi 50 1 0 12 Aug 2021
Transformer-based deep imitation learning for dual-arm robot manipulation Heecheol Kim Yoshiyuki Ohmura Yasuo Kuniyoshi 31 48 0 01 Aug 2021
Self-Paced Contrastive Learning for Semi-supervised Medical Image Segmentation with Meta-labels Jizong Peng Ping Wang Chrisitian Desrosiers M. Pedersoli SSL 31 64 0 29 Jul 2021
3D fluorescence microscopy data synthesis for segmentation and benchmarking Dennis Eschweiler Malte Rethwisch Mareike Jarchow Simon Koppers Johannes Stegmaier 3DV MedIm 56 15 0 21 Jul 2021
A New Adaptive Gradient Method with Gradient Decomposition Zhou Shao Tong Lin ODL 26 0 0 18 Jul 2021
TGIF: Tree-Graph Integrated-Format Parser for Enhanced UD with Two-Stage Generic- to Individual-Language Finetuning Tianze Shi Lillian Lee 35 7 0 14 Jul 2021
KOALA: A Kalman Optimization Algorithm with Loss Adaptivity A. Davtyan Sepehr Sameni L. Cerkezi Givi Meishvili Adam Bielski Paolo Favaro ODL 64 2 0 07 Jul 2021
Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons Zuowei Shen Haizhao Yang Shijun Zhang 70 36 0 06 Jul 2021
Morphological Classification of Galaxies in S-PLUS using an Ensemble of Convolutional Networks N. M. Cardoso G. B. O. Schwarz L. O. Dias C. R. Bom L. Sodré C. Mendes de Oliveira 27 0 0 05 Jul 2021
Ranger21: a synergistic deep learning optimizer Less Wright Nestor Demeure ODL AI4CE 47 87 0 25 Jun 2021
Probabilistic Attention for Interactive Segmentation Prasad Gabbur Manjot Bilkhu J. Movellan 39 13 0 23 Jun 2021
Multi-head or Single-head? An Empirical Comparison for Transformer Training Liyuan Liu Jialu Liu Jiawei Han 28 32 0 17 Jun 2021
Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation Haoxiang Wang Han Zhao Yue Liu 44 88 0 16 Jun 2021
BoolNet: Minimizing The Energy Consumption of Binary Neural Networks Nianhui Guo Joseph Bethge Haojin Yang Kai Zhong Xuefei Ning Christoph Meinel Yu Wang MQ 29 11 0 13 Jun 2021
Machine Translation into Low-resource Language Varieties Sachin Kumar Antonios Anastasopoulos S. Wintner Yulia Tsvetkov 24 29 0 12 Jun 2021
Generative Feature-driven Image Replay for Continual Learning Kevin Thandiackal Tiziano Portenier Andrea Giovannini M. Gabrani O. Goksel CLL VLM DiffM 23 9 0 09 Jun 2021
Cherry-Picking Gradients: Learning Low-Rank Embeddings of Visual Data via Differentiable Cross-Approximation Mikhail (Misha) Usvyatsov Anastasia Makarova R. Ballester-Ripoll M. Rakhuba Andreas Krause Konrad Schindler 36 5 0 29 May 2021
Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks Dong-Young Lim Sotirios Sabanis 56 12 0 28 May 2021
AngularGrad: A New Optimization Technique for Angular Convergence of Convolutional Neural Networks S. K. Roy Mercedes Eugenia Paoletti J. Haut S. Dubey Purushottam Kar A. Plaza B. B. Chaudhuri ODL 36 18 0 21 May 2021
Body Meshes as Points Jianfeng Zhang Dongdong Yu Jun Hao Liew Xuecheng Nie Jiashi Feng 3DH 27 64 0 06 May 2021
Audio Retrieval with Natural Language Queries Andreea-Maria Oncescu A. Sophia Koepke João F. Henriques Zeynep Akata Samuel Albanie 26 77 0 05 May 2021
Non-Autoregressive vs Autoregressive Neural Networks for System Identification Daniel Weber C. Gühmann 32 7 0 05 May 2021
PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Literature Parsing Task B: Table Recognition to HTML Jiaquan Ye Xianbiao Qi Yelin He Yihao Chen Dengyi Gu Peng Gao Rong Xiao LMTD 39 49 0 05 May 2021
Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer Mohamed S. Elmahdy Laurens Beljaards Sahar Yousefi Hessam Sokooti F. Verbeek U. A. van der Heide Marius Staring 34 21 0 05 May 2021
Acoustic Scene Classification Using Multichannel Observation with Partially Missing Channels Keisuke Imoto 17 8 0 05 May 2021
Learning from Event Cameras with Sparse Spiking Convolutional Neural Networks Loic Cordone Benoit Miramond Sonia Ferrante 38 36 0 26 Apr 2021
E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion Tianyi Wei Dongdong Chen Wenbo Zhou Jing Liao Weiming Zhang Lu Yuan Gang Hua Nenghai Yu 44 60 0 15 Apr 2021
RIANN -- A Robust Neural Network Outperforms Attitude Estimation Filters Daniel Weber C. Gühmann Thomas Seel 28 35 0 15 Apr 2021
TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph Jiaxin Shi S. Cao Lei Hou Juan-Zi Li Hanwang Zhang GNN 34 105 0 15 Apr 2021
SVDistNet: Self-Supervised Near-Field Distance Estimation on Surround View Fisheye Cameras Varun Ravi Kumar Marvin Klingner S. Yogamani Markus Bach Stefan Milz Tim Fingscheidt Patrick Mäder MDE 55 37 0 09 Apr 2021
Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems Derek Chen Howard Chen Yi Yang A. Lin Zhou Yu 30 66 0 01 Apr 2021
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization Zeke Xie Li-xin Yuan Zhanxing Zhu Masashi Sugiyama 43 29 0 31 Mar 2021