A Good Start Matters: Enhancing Continual Learning with Data-Driven Weight Initialization

9 March 2025

Papers citing "A Good Start Matters: Enhancing Continual Learning with Data-Driven Weight Initialization"

33 / 33 papers shown

Title
Continual Learning: Applications and the Road Forward Eli Verwimp Rahaf Aljundi Shai Ben-David Matthias Bethge Andrea Cossu ... Joost van de Weijer Bing Liu Vincenzo Lomonaco Tinne Tuytelaars Gido M. van de Ven CLL 100 47 0 20 Nov 2023
RanPAC: Random Projections and Pre-trained Models for Continual Learning Mark D Mcdonnell Dong Gong Amin Parvaneh Ehsan Abbasnejad Anton Van Den Hengel VLM CLL 86 111 0 05 Jul 2023
Understanding plasticity in neural networks Clare Lyle Zeyu Zheng Evgenii Nikishin Bernardo Avila-Pires Razvan Pascanu Will Dabney AI4CE 109 104 0 02 Mar 2023
Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model A. Luccioni S. Viguier Anne-Laure Ligozat 99 286 0 03 Nov 2022
Are All Losses Created Equal: A Neural Collapse Perspective Jinxin Zhou Chong You Xiao Li Kangning Liu Sheng Liu Qing Qu Zhihui Zhu 72 65 0 04 Oct 2022
Continual evaluation for lifelong learning: Identifying the stability gap Matthias De Lange Gido M. van de Ven Tinne Tuytelaars CLL 159 42 0 26 May 2022
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning Zifeng Wang Zizhao Zhang Sayna Ebrahimi Ruoxi Sun Han Zhang ... Xiaoqi Ren Guolong Su Vincent Perot Jennifer Dy Tomas Pfister CLL VLM VPVLM 117 501 0 10 Apr 2022
On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Features Jinxin Zhou Xiao Li Tian Ding Chong You Qing Qu Zhihui Zhu 74 102 0 02 Mar 2022
Architecture Matters in Continual Learning Seyed Iman Mirzadeh Arslan Chaudhry Dong Yin Timothy Nguyen Razvan Pascanu Dilan Görür Mehrdad Farajtabar OOD KELM 159 61 0 01 Feb 2022
Learning to Prompt for Continual Learning Zifeng Wang Zizhao Zhang Chen-Yu Lee Han Zhang Ruoxi Sun Xiaoqi Ren Guolong Su Vincent Perot Jennifer Dy Tomas Pfister CLL VPVLM KELM VLM 101 794 0 16 Dec 2021
Sustainable AI: Environmental Implications, Challenges and Opportunities Carole-Jean Wu Ramya Raghavendra Udit Gupta Bilge Acun Newsha Ardalani ... Maximilian Balandat Joe Spisak R. Jain Michael G. Rabbat K. Hazelwood 126 410 0 30 Oct 2021
Towards Continual Knowledge Learning of Language Models Joel Jang Seonghyeon Ye Sohee Yang Joongbo Shin Janghoon Han Gyeonghun Kim Stanley Jungkyu Choi Minjoon Seo CLL KELM 295 161 0 07 Oct 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 490 10,496 0 17 Jun 2021
A Geometric Analysis of Neural Collapse with Unconstrained Features Zhihui Zhu Tianyu Ding Jinxin Zhou Xiao Li Chong You Jeremias Sulam Qing Qu 72 205 0 06 May 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 337 680 0 21 Apr 2021
Prevalence of Neural Collapse during the terminal phase of deep learning training Vardan Papyan Xuemei Han D. Donoho 205 582 0 18 Aug 2020
RODEO: Replay for Online Object Detection Manoj Acharya Tyler L. Hayes Christopher Kanan ObjD CLL 57 46 0 14 Aug 2020
Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Tasks Like Hui M. Belkin UQCV AAML VLM 48 172 0 12 Jun 2020
Learning under Concept Drift: A Review Jie Lu Anjin Liu Fan Dong Feng Gu João Gama Guangquan Zhang AI4TS 67 1,290 0 13 Apr 2020
ReZero is All You Need: Fast Convergence at Large Depth Thomas C. Bachlechner Bodhisattwa Prasad Majumder H. H. Mao G. Cottrell Julian McAuley AI4CE 83 282 0 10 Mar 2020
Green AI Roy Schwartz Jesse Dodge Noah A. Smith Oren Etzioni 116 1,148 0 22 Jul 2019
Large Scale Incremental Learning Yue Wu Yinpeng Chen Lijuan Wang Yuancheng Ye Zicheng Liu Yandong Guo Y. Fu CLL 102 1,258 0 30 May 2019
Large-Scale Long-Tailed Recognition in an Open World Ziwei Liu Zhongqi Miao Xiaohang Zhan Jiayun Wang Boqing Gong Stella X. Yu 151 1,163 0 10 Apr 2019
On Tiny Episodic Memories in Continual Learning Arslan Chaudhry Marcus Rohrbach Mohamed Elhoseiny Thalaiyasingam Ajanthan P. Dokania Philip Torr MarcÁurelio Ranzato CLL 91 400 0 27 Feb 2019
Scalable and Order-robust Continual Learning with Additive Parameter Decomposition Jaehong Yoon Saehoon Kim Eunho Yang Sung Ju Hwang CLL 73 177 0 25 Feb 2019
Learning without Memorizing Prithviraj Dhar Rajat Vikram Singh Kuan-Chuan Peng Ziyan Wu Rama Chellappa CLL 88 486 0 20 Nov 2018
Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence Arslan Chaudhry P. Dokania Thalaiyasingam Ajanthan Philip Torr CLL 105 1,145 0 30 Jan 2018
Memory Aware Synapses: Learning what (not) to forget Rahaf Aljundi F. Babiloni Mohamed Elhoseiny Marcus Rohrbach Tinne Tuytelaars KELM CLL 87 1,646 0 27 Nov 2017
Overcoming catastrophic forgetting in neural networks J. Kirkpatrick Razvan Pascanu Neil C. Rabinowitz J. Veness Guillaume Desjardins ... A. Grabska-Barwinska Demis Hassabis Claudia Clopath D. Kumaran R. Hadsell CLL 374 7,561 0 02 Dec 2016
iCaRL: Incremental Classifier and Representation Learning Sylvestre-Alvise Rebuffi Alexander Kolesnikov G. Sperl Christoph H. Lampert CLL OOD 160 3,781 0 23 Nov 2016
Identity Matters in Deep Learning Moritz Hardt Tengyu Ma OOD 92 399 0 14 Nov 2016
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 338 18,651 0 06 Feb 2015
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.7K 39,595 0 01 Sep 2014