v1v2 (latest)

Lookahead Optimizer: k steps forward, 1 step back

19 July 2019

Jimmy Ba

Papers citing "Lookahead Optimizer: k steps forward, 1 step back"

50 / 357 papers shown

Title
Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training Lu Yin Vlado Menkovski Meng Fang Tianjin Huang Yulong Pei Mykola Pechenizkiy Decebal Constantin Mocanu Shiwei Liu 117 8 0 30 May 2022
AttentionCode: Ultra-Reliable Feedback Codes for Short-Packet Communications Yulin Shao Emre Ozfatura A. Perotti B. Popović Deniz Gunduz 66 23 0 30 May 2022
Object-wise Masked Autoencoders for Fast Pre-training Jiantao Wu Shentong Mo ViT OCL 75 15 0 28 May 2022
Trainable Weight Averaging: Accelerating Training and Improving Generalization Tao Li Zhehao Huang Yingwen Wu Zhengbao He Qinghua Tao Xiaolin Huang Chih-Jen Lin MoMe 108 0 0 26 May 2022
Semi-Parametric Inducing Point Networks and Neural Processes R. Rastogi Yair Schiff Alon Hacohen Zhaozhi Li I-Hsiang Lee Yuntian Deng M. Sabuncu Volodymyr Kuleshov 3DPC 86 7 0 24 May 2022
Diverse Weight Averaging for Out-of-Distribution Generalization Alexandre Ramé Matthieu Kirchmeyer Thibaud Rahier A. Rakotomamonjy Patrick Gallinari Matthieu Cord OOD 258 138 0 19 May 2022
Local Attention Graph-based Transformer for Multi-target Genetic Alteration Prediction Daniel Reisenbüchler S. J. Wagner Melanie Boxberg T. Peng MedIm 62 22 0 13 May 2022
Multimodal Indoor Localisation for Measuring Mobility in Parkinson's Disease using Transformers Ferdian Jovan Ryan McConville Catherine Morgan E. Tonkin Alan Whone I. Craddock 23 1 0 12 May 2022
LAWS: Look Around and Warm-Start Natural Gradient Descent for Quantum Neural Networks Zeyi Tao Jindi Wu Qi Xia Qun Li 60 9 0 05 May 2022
CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classification Marcos V. Conde Kerem Turgutlu CLIP VLM 99 99 0 29 Apr 2022
Differentiable Zooming for Multiple Instance Learning on Whole-Slide Images Kevin Thandiackal Boqi Chen Pushpak Pati Guillaume Jaume Drew F. K. Williamson M. Gabrani O. Goksel 82 46 0 26 Apr 2022
Investigating Neural Architectures by Synthetic Dataset Design Adrien Courtois Jean-Michel Morel Pablo Arias 72 4 0 23 Apr 2022
VISTA: Vision Transformer enhanced by U-Net and Image Colorfulness Frame Filtration for Automatic Retail Checkout Md. Istiak Hossain Shihab Nazia Tasnim H. Zunair L. Rupty Nabeel Mohammed 86 8 0 23 Apr 2022
Weakly Supervised Patch Label Inference Networks for Efficient Pavement Distress Detection and Recognition in the Wild Sheng Huang Wenhao Tang Guixin Huang Luwen Huangfu Dan Yang 121 9 0 31 Mar 2022
Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimation Gu Wang Fabian Manhardt Xingyu Liu Xiangyang Ji F. Tombari SSL 81 59 0 19 Mar 2022
Surrogate Gap Minimization Improves Sharpness-Aware Training Juntang Zhuang Boqing Gong Liangzhe Yuan Huayu Chen Hartwig Adam Nicha Dvornek S. Tatikonda James Duncan Ting Liu 105 158 0 15 Mar 2022
Style Transformer for Image Inversion and Editing Xueqi Hu Qiusheng Huang Zhengyi Shi Siyuan Li Changxin Gao Li Sun Qingli Li 87 56 0 15 Mar 2022
GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting Yan Di Ruida Zhang Zhiqiang Lou Fabian Manhardt Xiangyang Ji Nassir Navab F. Tombari 80 122 0 15 Mar 2022
Optimizer Amalgamation Tianshu Huang Tianlong Chen Sijia Liu Shiyu Chang Lisa Amini Zhangyang Wang MoMe 69 4 0 12 Mar 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time Mitchell Wortsman Gabriel Ilharco S. Gadre Rebecca Roelofs Raphael Gontijo-Lopes ... Hongseok Namkoong Ali Farhadi Y. Carmon Simon Kornblith Ludwig Schmidt MoMe 201 1,013 1 10 Mar 2022
Acceleration of Federated Learning with Alleviated Forgetting in Local Training Chencheng Xu Zhiwei Hong Minlie Huang Tao Jiang FedML 88 47 0 05 Mar 2022
A Multi-Scale Time-Frequency Spectrogram Discriminator for GAN-based Non-Autoregressive TTS Haohan Guo Hui Lu Xixin Wu Helen Meng 356 7 0 02 Mar 2022
DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training Joya Chen Kai Xu Yuhui Wang Yifei Cheng Angela Yao 52 8 0 28 Feb 2022
Echofilter: A Deep Learning Segmentation Model Improves the Automation, Standardization, and Timeliness for Post-Processing Echosounder Data in Tidal Energy Streams S. Lowe L. McGarry Jessica Douglas Jason Newport Sageev Oore Chris Whidden Daniel J. Hasselman 39 3 0 19 Feb 2022
Particle Transformer for Jet Tagging H. Qu Congqiao Li Sitian Qian ViT MedIm 85 106 0 08 Feb 2022
On the Power-Law Hessian Spectrums in Deep Learning Zeke Xie Qian-Yuan Tang Yunfeng Cai Mingming Sun P. Li ODL 99 10 0 31 Jan 2022
Bilevel Online Deep Learning in Non-stationary Environment Ya-nan Han Jian Liu Bing-biao Xiao Xin-Tan Wang Xiong-lin Luo 123 3 0 25 Jan 2022
Learning to Predict Gradients for Semi-Supervised Continual Learning Yan Luo Yongkang Wong Mohan S. Kankanhalli Qi Zhao SSL CLL 68 8 0 23 Jan 2022
Learning to Minimize the Remainder in Supervised Learning Yan Luo Yongkang Wong Mohan S. Kankanhalli Qi Zhao 97 1 0 23 Jan 2022
Last-Iterate Convergence of Saddle-Point Optimizers via High-Resolution Differential Equations Tatjana Chavdarova Michael I. Jordan Manolis Zampetakis 146 16 0 27 Dec 2021
Audio Retrieval with Natural Language Queries: A Benchmark Study A. Sophia Koepke Andreea-Maria Oncescu João F. Henriques Zeynep Akata Samuel Albanie 78 102 0 17 Dec 2021
CLIP-Lite: Information Efficient Visual Representation Learning with Language Supervision A. Shrivastava Ramprasaath R. Selvaraju Nikhil Naik Vicente Ordonez VLM CLIP 64 6 0 14 Dec 2021
More layers! End-to-end regression and uncertainty on tabular data with deep learning Ivan Bondarenko OOD LMTD UQCV 55 4 0 07 Dec 2021
Leveraging Human Selective Attention for Medical Image Analysis with Limited Training Data Yifei Huang Xiaoxiao Li Lijin Yang Lin Gu Yingying Zhu Hirofumi Seo Qiuming Meng Tatsuya Harada Yoichi Sato MedIm 82 9 0 02 Dec 2021
HyperInverter: Improving StyleGAN Inversion via Hypernetwork Tan M. Dinh Anh Tran Rang Nguyen Binh-Son Hua 83 119 0 01 Dec 2021
Neuron with Steady Response Leads to Better Generalization Qiang Fu Lun Du Haitao Mao Xu Chen Wei Fang Shi Han Dongmei Zhang 40 5 0 30 Nov 2021
AutoDrop: Training Deep Learning Models with Automatic Learning Rate Drop Yunfei Teng Jing Wang A. Choromańska 80 2 0 30 Nov 2021
An Expectation-Maximization Perspective on Federated Learning Christos Louizos M. Reisser Joseph B. Soriaga Max Welling FedML 83 12 0 19 Nov 2021
EditGAN: High-Precision Semantic Image Editing Huan Ling Karsten Kreis Daiqing Li Seung Wook Kim Antonio Torralba Sanja Fidler DiffM GAN 70 203 0 04 Nov 2021
Multilayer Lookahead: a Nested Version of Lookahead Denys Pushkin Luis Barba 97 1 0 27 Oct 2021
Applications and Techniques for Fast Machine Learning in Science A. Deiana Nhan Tran Joshua C. Agar Michaela Blott G. D. Guglielmo ... Ashish Sharma S. Summers Pietro Vischia J. Vlimant Olivia Weng 94 72 0 25 Oct 2021
The Efficiency Misnomer Daoyuan Chen Liuyi Yao Dawei Gao Ashish Vaswani Yaliang Li 109 103 0 25 Oct 2021
OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data Christoph Reich Tim Prangemeier Ozdemir Cetin Heinz Koeppl 68 12 0 20 Oct 2021
M2GAN: A Multi-Stage Self-Attention Network for Image Rain Removal on Autonomous Vehicles Duc Manh Nguyen Sang-Woong Lee GAN 39 1 0 12 Oct 2021
UnfairGAN: An Enhanced Generative Adversarial Network for Raindrop Removal from A Single Image Duc Manh Nguyen Sang-Woong Lee GAN 53 14 0 11 Oct 2021
Momentum Centering and Asynchronous Update for Adaptive Gradient Methods Juntang Zhuang Yifan Ding Tommy M. Tang Nicha Dvornek S. Tatikonda James S. Duncan ODL 55 4 0 11 Oct 2021
Rethinking Person Re-Identification via Semantic-Based Pretraining Suncheng Xiang Jingsheng Gao Zi-Yu Zhang Mengyuan Guan Binghai Yan Ting Liu Xiaobo Li Yuzhuo Fu VLM 111 11 0 11 Oct 2021
Efficient Sharpness-aware Minimization for Improved Training of Neural Networks Jiawei Du Hanshu Yan Jiashi Feng Qiufeng Wang Liangli Zhen Rick Siow Mong Goh Vincent Y. F. Tan AAML 193 135 0 07 Oct 2021
Stochastic Anderson Mixing for Nonconvex Stochastic Optimization Fu Wei Chenglong Bao Yang Liu 76 20 0 04 Oct 2021
DualNet: Continual Learning, Fast and Slow Quang Pham Chenghao Liu Guosheng Lin CLL 147 45 0 01 Oct 2021