Momentum-based Weight Interpolation of Strong Zero-Shot Models for Continual Learning

6 November 2022

Papers citing "Momentum-based Weight Interpolation of Strong Zero-Shot Models for Continual Learning"

24 / 24 papers shown

Title
BECAME: BayEsian Continual Learning with Adaptive Model MErging Mei Li Yuxiang Lu Qinyan Dai Suizhi Huang Yue Ding Hongtao Lu CLL MoMe 103 0 0 03 Apr 2025
Continual Learning with Foundation Models: An Empirical Study of Latent Replay O. Ostapenko Timothée Lesort P. Rodríguez Md Rifat Arefin Arthur Douillard Irina Rish Laurent Charlin 65 53 0 30 Apr 2022
An Empirical Investigation of the Role of Pre-training in Lifelong Learning Sanket Vaibhav Mehta Darshan Patil Sarath Chandar Emma Strubell CLL 87 144 0 16 Dec 2021
Learning to Prompt for Continual Learning Zifeng Wang Zizhao Zhang Chen-Yu Lee Han Zhang Ruoxi Sun Xiaoqi Ren Guolong Su Vincent Perot Jennifer Dy Tomas Pfister CLL VPVLM KELM VLM 84 769 0 16 Dec 2021
Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima Guangyuan Shi Jiaxin Chen Wenlong Zhang Li-Ming Zhan Xiao-Ming Wu CLL 119 152 0 30 Oct 2021
DualNet: Continual Learning, Fast and Slow Quang Pham Chenghao Liu Guosheng Lin CLL 116 45 0 01 Oct 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 117 721 0 04 Sep 2021
Co $^2$ L: Contrastive Continual Learning Hyuntak Cha Jaeho Lee Jinwoo Shin CLL SSL VLM 102 301 0 28 Jun 2021
Rainbow Memory: Continual Learning with a Memory of Diverse Samples Jihwan Bang Heesu Kim Y. Yoo Jung-Woo Ha Jonghyun Choi CLL 74 331 0 31 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 818 29,167 0 26 Feb 2021
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 184 1,344 0 03 Oct 2020
Understanding the Role of Training Regimes in Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar Razvan Pascanu H. Ghasemzadeh CLL 67 225 0 12 Jun 2020
Dropout as an Implicit Gating Mechanism For Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar H. Ghasemzadeh CLL 61 43 0 24 Apr 2020
Dark Experience for General Continual Learning: a Strong, Simple Baseline Pietro Buzzega Matteo Boschini Angelo Porrello Davide Abati Simone Calderara BDL CLL 72 909 0 15 Apr 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 384 42,299 0 03 Dec 2019
Lookahead Optimizer: k steps forward, 1 step back Michael Ruogu Zhang James Lucas Geoffrey E. Hinton Jimmy Ba ODL 118 728 0 19 Jul 2019
Efficient Lifelong Learning with A-GEM Arslan Chaudhry MarcÁurelio Ranzato Marcus Rohrbach Mohamed Elhoseiny CLL 188 1,449 0 02 Dec 2018
Progress & Compress: A scalable framework for continual learning Jonathan Richard Schwarz Jelena Luketina Wojciech M. Czarnecki A. Grabska-Barwinska Yee Whye Teh Razvan Pascanu R. Hadsell CLL 108 886 0 16 May 2018
Averaging Weights Leads to Wider Optima and Better Generalization Pavel Izmailov Dmitrii Podoprikhin T. Garipov Dmitry Vetrov A. Wilson FedML MoMe 112 1,658 0 14 Mar 2018
Gradient Episodic Memory for Continual Learning David Lopez-Paz MarcÁurelio Ranzato VLM CLL 105 2,705 0 26 Jun 2017
Overcoming Catastrophic Forgetting by Incremental Moment Matching Sang-Woo Lee Jin-Hwa Kim Jaehyun Jun Jung-Woo Ha Byoung-Tak Zhang CLL 68 674 0 24 Mar 2017
Overcoming catastrophic forgetting in neural networks J. Kirkpatrick Razvan Pascanu Neil C. Rabinowitz J. Veness Guillaume Desjardins ... A. Grabska-Barwinska Demis Hassabis Claudia Clopath D. Kumaran R. Hadsell CLL 325 7,478 0 02 Dec 2016
Learning without Forgetting Zhizhong Li Derek Hoiem CLL OOD SSL 282 4,391 0 29 Jun 2016
Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes Ohad Shamir Tong Zhang 146 574 0 08 Dec 2012