On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019

Xiaodong Liu

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 373 papers shown

Title
MFIM: Megapixel Facial Identity Manipulation Sanghyeon Na PICV CVBM 43 4 0 03 Aug 2023
Multiplicative update rules for accelerating deep learning training and increasing robustness Manos Kirtas Nikolaos Passalis Anastasios Tefas AAML OOD 41 2 0 14 Jul 2023
Align With Purpose: Optimize Desired Properties in CTC Models with a General Plug-and-Play Framework Eliya Segev Maya Alroy Ronen Katsir Noam Wies Ayana Shenhav ... D. Zar Oren Tadmor Jacob Bitterman Amnon Shashua Tal Rosenwein 47 2 0 04 Jul 2023
Relation-aware graph structure embedding with co-contrastive learning for drug-drug interaction prediction Mengying Jiang Guizhong Liu Biao Zhao Yuanchao Su Weiqiang Jin CML 62 7 0 04 Jul 2023
Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers Yineng Chen Z. Li Lefei Zhang Bo Du Hai Zhao 43 4 0 02 Jul 2023
Nonconvex Stochastic Bregman Proximal Gradient Method with Application to Deep Learning Kuan-Fu Ding Jingyang Li Kim-Chuan Toh 46 8 0 26 Jun 2023
Addressing Cold Start Problem for End-to-end Automatic Speech Scoring Jungbae Park Seungtaek Choi 37 4 0 25 Jun 2023
Single-Stage 3D Geometry-Preserving Depth Estimation Model Training on Dataset Mixtures with Uncalibrated Stereo Data Nikolay Patakin Mikhail Romanov Anna Vorontsova M. Artemyev Anton Konushin MDE 52 6 0 05 Jun 2023
Combining Explicit and Implicit Regularization for Efficient Learning in Deep Networks Dan Zhao 54 5 0 01 Jun 2023
Intelligent gradient amplification for deep neural networks S. Basodi K. Pusuluri Xueli Xiao Yi Pan ODL 26 1 0 29 May 2023
Stochastic Pitch Prediction Improves the Diversity and Naturalness of Speech in Glow-TTS Sewade Ogun Vincent Colotte Emmanuel Vincent DiffM 45 4 0 28 May 2023
SING: A Plug-and-Play DNN Learning Technique Adrien Courtois Damien Scieur Jean-Michel Morel Pablo Arias Thomas Eboli 45 0 0 25 May 2023
Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods Junchi Yang Xiang Li Ilyas Fatkhullin Niao He 53 15 0 21 May 2023
Bridging Discrete and Backpropagation: Straight-Through and Beyond Liyuan Liu Chengyu Dong Xiaodong Liu Bin Yu Jianfeng Gao BDL 31 20 0 17 Apr 2023
CrowdCLIP: Unsupervised Crowd Counting via Vision-Language Model Dingkang Liang Jiahao Xie Zhikang Zou Xiaoqing Ye Wei Xu Xiang Bai SSL CLIP VLM 51 54 0 09 Apr 2023
AgileGAN3D: Few-Shot 3D Portrait Stylization by Augmented Transfer Learning Guoxian Song Hongyi Xu Jing Liu Tiancheng Zhi Yichun Shi Jianfeng Zhang Zihang Jiang Jiashi Feng S. Sang Linjie Luo 3DH 39 6 0 24 Mar 2023
TriPlaneNet: An Encoder for EG3D Inversion A. Bhattarai Matthias Nießner Artem Sevastopolsky 56 34 0 23 Mar 2023
Unsupervised Domain Adaptation for Training Event-Based Networks Using Contrastive Learning and Uncorrelated Conditioning Dayuan Jian Mohammad Rostami 36 14 0 22 Mar 2023
SPOTR: Spatio-temporal Pose Transformers for Human Motion Prediction A. A. Nargund Misha Sra ViT 41 2 0 11 Mar 2023
EfficientTempNet: Temporal Super-Resolution of Radar Rainfall B. Demiray M. Sit Ibrahim Demir 31 4 0 09 Mar 2023
Diffusing Gaussian Mixtures for Generating Categorical Data Florence Regol Mark Coates DiffM 44 5 0 08 Mar 2023
FoundationTTS: Text-to-Speech for ASR Customization with Generative Language Model Rui Xue Yanqing Liu Lei He Xuejiao Tan Linquan Liu Ed Lin Sheng Zhao 43 7 0 06 Mar 2023
Fixed-point quantization aware training for on-device keyword-spotting Sashank Macha Om Oza Alex Escott Francesco Calivá Robert M. Armitano S. Cheekatmalla S. Parthasarathi Yuzong Liu MQ 26 4 0 04 Mar 2023
Consistency Models Yang Song Prafulla Dhariwal Mark Chen Ilya Sutskever VLM DiffM 74 884 0 02 Mar 2023
One-Shot Face Video Re-enactment using Hybrid Latent Spaces of StyleGAN2 Trevine Oorloff Yaser Yacoob CVBM 39 3 0 15 Feb 2023
The Role of Semantic Parsing in Understanding Procedural Text Hossein Rajaby Faghihi Parisa Kordjamshidi C. Teng J. Allen 27 5 0 14 Feb 2023
Symbolic Discovery of Optimization Algorithms Xiangning Chen Chen Liang Da Huang Esteban Real Kaiyuan Wang ... Xuanyi Dong Thang Luong Cho-Jui Hsieh Yifeng Lu Quoc V. Le 86 356 0 13 Feb 2023
Multi-scale Feature Alignment for Continual Learning of Unlabeled Domains Kevin Thandiackal Luigi Piccinelli Pushpak Pati O. Goksel CLL OOD MedIm 45 7 0 02 Feb 2023
A Survey of Deep Learning: From Activations to Transformers Johannes Schneider Michalis Vlachos ViT MedIm AI4TS AI4CE 55 10 0 01 Feb 2023
Weight Prediction Boosts the Convergence of AdamW Lei Guan 42 16 0 01 Feb 2023
What Decreases Editing Capability? Domain-Specific Hybrid Refinement for Improved GAN Inversion Pu Cao Lu Yang Dongxu Liu Zhiwei Liu Shan Li Q. Song 32 6 0 28 Jan 2023
FewShotTextGCN: K-hop neighborhood regularization for few-shot learning on graphs Niels van der Heijden Ekaterina Shutova H. Yannakoudakis 43 0 0 25 Jan 2023
Read the Signs: Towards Invariance to Gradient Descent's Hyperparameter Initialization Davood Wadi M. Fredette S. Sénécal ODL AI4CE 16 0 0 24 Jan 2023
Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation Razvan-George Pasca Alexey Gavryushin Muhammad Hamza Yen-Ling Kuo Kaichun Mo Luc Van Gool Otmar Hilliges Xi Wang 59 14 0 22 Jan 2023
Convolution-enhanced Evolving Attention Networks Yujing Wang Yaming Yang Zhuowan Li Jiangang Bai Mingliang Zhang Xiangtai Li Jiahao Yu Ce Zhang Gao Huang Yu Tong ViT 57 6 0 16 Dec 2022
Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamics Manuela Brenner Florian Hess G. Koppe Daniel Durstewitz 42 10 0 15 Dec 2022
Cross-Domain Transfer via Semantic Skill Imitation Karl Pertsch Ruta Desai Vikash Kumar Franziska Meier Joseph J. Lim Dhruv Batra Akshara Rai LM&Ro 26 19 0 14 Dec 2022
Improving Depression estimation from facial videos with face alignment, training optimization and scheduling Manuel Lage Cañellas Constantino Álvarez Casado L. Nguyen Miguel Bordallo López CVBM 26 3 0 13 Dec 2022
Punctuation Restoration for Singaporean Spoken Languages: English, Malay, and Mandarin Abhinav Rao Ho Thi-Nga Chng Eng Siong 28 3 0 10 Dec 2022
Parameter Efficient Transfer Learning for Various Speech Processing Tasks Shinta Otake Rei Kawakami Nakamasa Inoue 29 16 0 06 Dec 2022
Semantic Role Labeling Meets Definition Modeling: Using Natural Language to Describe Predicate-Argument Structures Simone Conia Edoardo Barba Alessandro Sciré Roberto Navigli 37 7 0 02 Dec 2022
The Vanishing Decision Boundary Complexity and the Strong First Component Hengshuai Yao UQCV 41 0 0 25 Nov 2022
Join the High Accuracy Club on ImageNet with A Binary Neural Network Ticket Nianhui Guo Joseph Bethge Christoph Meinel Haojin Yang MQ 41 20 0 23 Nov 2022
$β$ -Multivariational Autoencoder for Entangled Representation Learning in Video Frames F. Nouri R. Bergevin 26 0 0 22 Nov 2022
Uncertainty-aware Vision-based Metric Cross-view Geolocalization F. Fervers Sebastian Bullinger C. Bodensteiner Michael Arens Rainer Stiefelhagen 40 40 0 22 Nov 2022
GAN Inversion for Image Editing via Unsupervised Domain Adaptation Siyu Xing Chen Gong Hewei Guo Xiaoyi Zhang Xinwen Hou Yu Liu 55 6 0 22 Nov 2022
Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint Hongyu Liu Yibing Song Qifeng Chen DiffM 40 21 0 21 Nov 2022
Novel transfer learning schemes based on Siamese networks and synthetic data Dominik Stallmann Philip Kenneweg Barbara Hammer 26 6 0 21 Nov 2022
VeLO: Training Versatile Learned Optimizers by Scaling Up Luke Metz James Harrison C. Freeman Amil Merchant Lucas Beyer ... Naman Agrawal Ben Poole Igor Mordatch Adam Roberts Jascha Narain Sohl-Dickstein 42 60 0 17 Nov 2022
Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization Yiyang Chen Zhedong Zheng Wei Ji Leigang Qu Tat-Seng Chua 61 38 0 14 Nov 2022