On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019

Xiaodong Liu

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 373 papers shown

Title
PSO-Convolutional Neural Networks with Heterogeneous Learning Rate N. H. Phong A. Santos B. Ribeiro 37 8 0 20 May 2022
Neural Network Architecture Beyond Width and Depth Zuowei Shen Haizhao Yang Shijun Zhang 3DV MDE 52 13 0 19 May 2022
CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classification Marcos V. Conde Kerem Turgutlu CLIP VLM 44 97 0 29 Apr 2022
Narcissus: A Practical Clean-Label Backdoor Attack with Limited Information Yi Zeng Minzhou Pan H. Just Lingjuan Lyu M. Qiu R. Jia AAML 44 171 0 11 Apr 2022
How Information on Acoustic Scenes and Sound Events Mutually Benefits Event Detection and Scene Classification Tasks Keisuke Imoto Yuka Komatsu Shunsuke Tsubaki Tatsuya Komatsu 41 5 0 05 Apr 2022
The Group Loss++: A deeper look into group loss for deep metric learning Ismail Elezi Jenny Seidenschwarz Laurin Wagner Sebastiano Vascon Alessandro Torcinovich Marcello Pelillo Laura Leal-Taixe 37 12 0 04 Apr 2022
SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image Dejia Xu Yi Ding Peihao Wang Zhiwen Fan Humphrey Shi Zhangyang Wang 46 188 0 02 Apr 2022
Learning to Deblur using Light Field Generated and Real Defocus Images Lingyan Ruan Bin Chen Jizhou Li Miuling Lam 34 68 0 01 Apr 2022
Weakly Supervised Patch Label Inference Networks for Efficient Pavement Distress Detection and Recognition in the Wild Sheng Huang Wenhao Tang Guixin Huang Luwen Huangfu Dan Yang 25 8 0 31 Mar 2022
Reference-based Video Super-Resolution Using Multi-Camera Video Triplets Junyong Lee Myeonghee Lee Sunghyun Cho Seungyong Lee SupR 35 27 0 28 Mar 2022
A DNN Optimizer that Improves over AdaBelief by Suppression of the Adaptive Stepsize Range Guoqiang Zhang Kenta Niwa W. Kleijn ODL 23 2 0 24 Mar 2022
An Adaptive Gradient Method with Energy and Momentum Hailiang Liu Xuping Tian ODL 26 9 0 23 Mar 2022
Practical tradeoffs between memory, compute, and performance in learned optimizers Luke Metz C. Freeman James Harrison Niru Maheswaranathan Jascha Narain Sohl-Dickstein 46 32 0 22 Mar 2022
ESS: Learning Event-based Semantic Segmentation from Still Images Zhaoning Sun Nico Messikommer Daniel Gehrig Davide Scaramuzza 40 78 0 18 Mar 2022
Goal-conditioned dual-action imitation learning for dexterous dual-arm robot manipulation Heecheol Kim Yoshiyuki Ohmura Yasuo Kuniyoshi 37 27 0 18 Mar 2022
Style Transformer for Image Inversion and Editing Xueqi Hu Qiusheng Huang Zhengyi Shi Siyuan Li Changxin Gao Li Sun Qingli Li 46 55 0 15 Mar 2022
GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting Yan Di Ruida Zhang Zhiqiang Lou Fabian Manhardt Xiangyang Ji Nassir Navab F. Tombari 43 119 0 15 Mar 2022
RecursiveMix: Mixed Learning with History Lingfeng Yang Xiang Li Borui Zhao Renjie Song Jian Yang VLM 36 18 0 14 Mar 2022
Near-optimal Deep Reinforcement Learning Policies from Data for Zone Temperature Control L. D. Natale B. Svetozarevic Philipp Heer Colin N. Jones OffRL AI4CE 40 6 0 10 Mar 2022
Rethinking data-driven point spread function modeling with a differentiable optical model T. Liaudat Jean-Luc Starck M. Kilbinger P. Frugier 11 12 0 09 Mar 2022
SkinningNet: Two-Stream Graph Convolutional Neural Network for Skinning Prediction of Synthetic Characters Albert Mosella-Montoro Javier Ruiz-Hidalgo 3DH 48 12 0 09 Mar 2022
DeepNet: Scaling Transformers to 1,000 Layers Hongyu Wang Shuming Ma Li Dong Shaohan Huang Dongdong Zhang Furu Wei MoE AI4CE 50 157 0 01 Mar 2022
Training Robots without Robots: Deep Imitation Learning for Master-to-Robot Policy Transfer Heecheol Kim Yoshiyuki Ohmura Akihiko Nagakubo Yasuo Kuniyoshi 26 23 0 19 Feb 2022
Motion Puzzle: Arbitrary Motion Style Transfer by Body Part Deok-Kyeong Jang S. Park Sung-Hee Lee 3DH 42 59 0 10 Feb 2022
Particle Transformer for Jet Tagging H. Qu Congqiao Li Sitian Qian ViT MedIm 29 98 0 08 Feb 2022
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models Chen Liang Haoming Jiang Simiao Zuo Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen T. Zhao 30 14 0 06 Feb 2022
Global Optimization Networks Sen Zhao Erez Louidor Ilan Oleksandr Mangylov Maya R. Gupta 57 5 0 02 Feb 2022
On the Power-Law Hessian Spectrums in Deep Learning Zeke Xie Qian-Yuan Tang Yunfeng Cai Mingming Sun P. Li ODL 44 9 0 31 Jan 2022
A Stochastic Bundle Method for Interpolating Networks Alasdair Paren Leonard Berrada Rudra P. K. Poudel M. P. Kumar 31 4 0 29 Jan 2022
Data-Efficient Information Extraction from Form-Like Documents Beliz Gunel Navneet Potti Sandeep Tata James Bradley Wendt Marc Najork Jing Xie 40 2 0 07 Jan 2022
Sign Language Video Retrieval with Free-Form Textual Queries A. Duarte Samuel Albanie Xavier Giró-i-Nieto Gül Varol SLR 58 29 0 07 Jan 2022
Including STDP to eligibility propagation in multi-layer recurrent spiking neural networks Werner van der Veen 44 1 0 05 Jan 2022
Class-Incremental Continual Learning into the eXtended DER-verse Matteo Boschini Lorenzo Bonicelli Pietro Buzzega Angelo Porrello Simone Calderara CLL BDL 37 133 0 03 Jan 2022
PointCaps: Raw Point Cloud Processing using Capsule Networks with Euclidean Distance Routing Dishanika Denipitiyage Vinoj Jayasundara Ranga Rodrigo Chamira U. S. Edussooriya 3DPC 40 6 0 21 Dec 2021
Improving Unsupervised Stain-To-Stain Translation using Self-Supervision and Meta-Learning Nassim Bouteldja B. Klinkhammer Tarek Schlaich P. Boor Dorit Merhof MedIm 37 20 0 16 Dec 2021
Self-Supervised Bot Play for Conversational Recommendation with Justifications Shuyang Li Bodhisattwa Prasad Majumder Julian McAuley 38 7 0 09 Dec 2021
More layers! End-to-end regression and uncertainty on tabular data with deep learning Ivan Bondarenko OOD LMTD UQCV 30 4 0 07 Dec 2021
A Novel Convergence Analysis for Algorithms of the Adam Family Zhishuai Guo Yi Tian Xu W. Yin Rong Jin Tianbao Yang 42 48 0 07 Dec 2021
JointLK: Joint Reasoning with Language Models and Knowledge Graphs for Commonsense Question Answering Yueqing Sun Qi Shi Le Qi Yu Zhang RALM LRM 41 70 0 06 Dec 2021
HyperInverter: Improving StyleGAN Inversion via Hypernetwork Tan M. Dinh Anh Tran Rang Nguyen Binh-Son Hua 38 116 0 01 Dec 2021
Environmental Sound Extraction Using Onomatopoeic Words Yuki Okamoto Shota Horiguchi Masaaki Yamamoto Keisuke Imoto Yohei Kawaguchi 34 9 0 01 Dec 2021
DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation Lukas Hoyer Dengxin Dai Luc Van Gool AI4CE 49 454 0 29 Nov 2021
Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion Nobuhiko Wakai Satoshi Sato Yasunori Ishii Takayoshi Yamashita 26 8 0 25 Nov 2021
Rethinking the modeling of the instrumental response of telescopes with a differentiable optical model T. Liaudat Jean-Luc Starck M. Kilbinger P. Frugier 14 9 0 24 Nov 2021
Hidden-Fold Networks: Random Recurrent Residuals Using Sparse Supermasks Ángel López García-Arias Masanori Hashimoto Masato Motomura Jaehoon Yu 41 5 0 24 Nov 2021
Hierarchical Knowledge Distillation for Dialogue Sequence Labeling Shota Orihashi Yoshihiro Yamazaki Naoki Makishima Mana Ihori Akihiko Takashima Tomohiro Tanaka Ryo Masumura 29 0 0 22 Nov 2021
Capitalization and Punctuation Restoration: a Survey V. Pais D. Tufis 31 19 0 21 Nov 2021
Diversified Multi-prototype Representation for Semi-supervised Segmentation Jizong Peng Christian Desrosiers M. Pedersoli 34 1 0 16 Nov 2021
Deep Network Approximation in Terms of Intrinsic Parameters Zuowei Shen Haizhao Yang Shijun Zhang 26 9 0 15 Nov 2021
Conformal prediction for text infilling and part-of-speech prediction N. Dey Jing Ding Jack G. Ferrell Carolina Kapper Maxwell Lovig Emiliano Planchon Jonathan P. Williams UQLM 29 19 0 04 Nov 2021