Supermasks in Superposition

26 June 2020

Papers citing "Supermasks in Superposition"

50 / 64 papers shown

Title
Achieving Upper Bound Accuracy of Joint Training in Continual Learning Saleh Momeni Bing Liu CLL 84 1 0 17 Feb 2025
The impact of allocation strategies in subset learning on the expressive power of neural networks Ofir Schlisselberg Ran Darshan 93 0 0 10 Feb 2025
Modular Training of Neural Networks aids Interpretability Satvik Golechha Maheep Chaudhary Joan Velja Alessandro Abate Nandi Schoots 79 0 0 04 Feb 2025
TinySubNets: An efficient and low capacity continual learning strategy Marcin Pietroñ Kamil Faber Dominik Zurek Roberto Corizzo 80 0 0 14 Dec 2024
More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routing Sagi Shaier Francisco Pereira K. Wense Lawrence E Hunter Matt Jones MoE 46 0 0 10 Oct 2024
Self-Masking Networks for Unsupervised Adaptation Alfonso Taboada Warmerdam Mathilde Caron Yuki M. Asano 43 1 0 11 Sep 2024
Buffer-based Gradient Projection for Continual Federated Learning Shenghong Dai Jy-yong Sohn Yicong Chen S. Alam Ravikumar Balakrishnan Suman Banerjee N. Himayat Kangwook Lee FedML 75 2 0 03 Sep 2024
Mitigating the Stability-Plasticity Dilemma in Adaptive Train Scheduling with Curriculum-Driven Continual DQN Expansion Achref Jaziri Etienne Kunzel Visvanathan Ramesh CLL 50 0 0 19 Aug 2024
Learning System Dynamics without Forgetting Xikun Zhang Dongjin Song Yushan Jiang Yixin Chen Dacheng Tao AI4CE 37 2 0 30 Jun 2024
Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System Chen Chen Ruizhe Li Yuchen Hu Yuanyuan Chen Chengwei Qin Qiang Zhang CLL 41 1 0 16 May 2024
Understanding Multimodal Deep Neural Networks: A Concept Selection View Chenming Shang Hengyuan Zhang Hao Wen Yujiu Yang 43 5 0 13 Apr 2024
Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning Huiyi Wang Haodong Lu Lina Yao Dong Gong KELM CLL 42 8 0 27 Mar 2024
MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superposition Nicolas Menet Michael Hersche G. Karunaratne Luca Benini Abu Sebastian Abbas Rahimi 28 13 0 05 Dec 2023
Efficient Rehearsal Free Zero Forgetting Continual Learning using Adaptive Weight Modulation Yonatan Sverdlov Shimon Ullman 23 0 0 26 Nov 2023
Continual Learning of Diffusion Models with Generative Distillation Sergi Masip Pau Rodriguez Tinne Tuytelaars Gido M. van de Ven VLM DiffM 36 7 0 23 Nov 2023
Density Distribution-based Learning Framework for Addressing Online Continual Learning Challenges Shilin Zhang Jiahui Wang CLL 32 0 0 22 Nov 2023
Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs Yu-xin Zhang Lirui Zhao Mingbao Lin Yunyun Sun Yiwu Yao Xingjia Han Jared Tanner Shiwei Liu Rongrong Ji SyDa 37 40 0 13 Oct 2023
Class-Incremental Learning based on Label Generation Yijia Shao Yiduo Guo Dongyan Zhao Bin Liu CLL 24 14 0 22 Jun 2023
Lightweight Learner for Shared Knowledge Lifelong Learning Yunhao Ge Yuecheng Li Di Wu Ao Xu Adam M. Jones ... Gozde Sahin Shuo Ni Kiran Lekkala S. Sontakke Laurent Itti CLL 19 10 0 24 May 2023
Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models Guillermo Ortiz-Jiménez Alessandro Favero P. Frossard MoMe 51 110 0 22 May 2023
Provable Pathways: Learning Multiple Tasks over Multiple Paths Yingcong Li Samet Oymak MoE 26 4 0 08 Mar 2023
Quantum Ridgelet Transform: Winning Lottery Ticket of Neural Networks with Quantum Computation H. Yamasaki Sathyawageeswar Subramanian Satoshi Hayakawa Sho Sonoda MLT 30 4 0 27 Jan 2023
A Distinct Unsupervised Reference Model From The Environment Helps Continual Learning Seyyed Amirhossein Ameli Kalkhoran Mohammadamin Banayeeanzade Mahdi Samiei M. Baghshah BDL CLL 23 0 0 11 Jan 2023
Generative appearance replay for continual unsupervised domain adaptation Boqi Chen Kevin Thandiackal Pushpak Pati O. Goksel CLL MedIm 32 11 0 03 Jan 2023
PIVOT: Prompting for Video Continual Learning Andrés Villa Juan Carlos León Alcázar Motasem Alfarra Kumail Alhamoud J. Hurtado Fabian Caba Heilbron Alvaro Soto Guohao Li VLM CLL 40 45 0 09 Dec 2022
You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets Tianjin Huang Tianlong Chen Meng Fang Vlado Menkovski Jiaxu Zhao ... Yulong Pei D. Mocanu Zhangyang Wang Mykola Pechenizkiy Shiwei Liu GNN 44 14 0 28 Nov 2022
Cooperative data-driven modeling Aleksandr Dekhovich O. T. Turan Jiaxiang Yi Miguel A. Bessa CLL KELM AI4CE 19 5 0 23 Nov 2022
Continual Learning of Natural Language Processing Tasks: A Survey Zixuan Ke Bin Liu KELM CLL VLM 26 68 0 23 Nov 2022
Building a Subspace of Policies for Scalable Continual Learning Jean-Baptiste Gaya T. Doan Lucas Caccia Laure Soulier Ludovic Denoyer Roberta Raileanu CLL 29 29 0 18 Nov 2022
A Theoretical Study on Solving Continual Learning Gyuhak Kim Changnan Xiao Tatsuya Konishi Zixuan Ke Bin Liu CLL 21 70 0 04 Nov 2022
Is Multi-Task Learning an Upper Bound for Continual Learning? Zihao Wu Huy Tran Hamed Pirsiavash Soheil Kolouri SSL CLL 26 4 0 26 Oct 2022
Efficient Meta-Learning for Continual Learning with Taylor Expansion Approximation Xiaohan Zou Tong Lin CLL 21 2 0 03 Oct 2022
A Multi-Head Model for Continual Learning via Out-of-Distribution Replay Gyuhak Kim Zixuan Ke Bin Liu VLM CLL OODD 17 29 0 20 Aug 2022
Continual Prune-and-Select: Class-incremental learning with specialized subnetworks Aleksandr Dekhovich David Tax M. Sluiter Miguel A. Bessa CLL 21 20 0 09 Aug 2022
Incremental Task Learning with Incremental Rank Updates Rakib Hyder Ken Shao Boyu Hou P. Markopoulos Ashley Prater-Bennette M. Salman Asif CLL 16 13 0 19 Jul 2022
Reactive Exploration to Cope with Non-Stationarity in Lifelong Reinforcement Learning C. Steinparz Thomas Schmied Fabian Paischer Marius-Constantin Dinu Vihang Patil Angela Bitto-Nemling Hamid Eghbalzadeh Sepp Hochreiter CLL 24 11 0 12 Jul 2022
Explain to Not Forget: Defending Against Catastrophic Forgetting with XAI Sami Ede Serop Baghdadlian Leander Weber A. Nguyen Dario Zanca Wojciech Samek Sebastian Lapuschkin CLL 27 6 0 04 May 2022
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning Zifeng Wang Zizhao Zhang Sayna Ebrahimi Ruoxi Sun Han Zhang ... Xiaoqi Ren Guolong Su Vincent Perot Jennifer Dy Tomas Pfister CLL VLM VPVLM 36 455 0 10 Apr 2022
Class-Incremental Learning with Strong Pre-trained Models Tz-Ying Wu Gurumurthy Swaminathan Zhizhong Li Avinash Ravichandran Nuno Vasconcelos Rahul Bhotika Stefano Soatto CLL 24 67 0 07 Apr 2022
Continual Learning for Multivariate Time Series Tasks with Variable Input Dimensions Vibhor Gupta Jyoti Narwariya Pankaj Malhotra L. Vig Gautam M. Shroff AI4TS 24 20 0 14 Mar 2022
Memory Efficient Continual Learning with Transformers B. Ermiş Giovanni Zappella Martin Wistuba Aditya Rawal Cédric Archambeau CLL 21 42 0 09 Mar 2022
Low-Loss Subspace Compression for Clean Gains against Multi-Agent Backdoor Attacks Siddhartha Datta N. Shadbolt AAML 24 6 0 07 Mar 2022
Continual Learning with Invertible Generative Models Jary Pomponi Simone Scardapane A. Uncini BDL 48 4 0 11 Feb 2022
Architecture Matters in Continual Learning Seyed Iman Mirzadeh Arslan Chaudhry Dong Yin Timothy Nguyen Razvan Pascanu Dilan Görür Mehrdad Farajtabar OOD KELM 116 58 0 01 Feb 2022
Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments A. Iyer Karan Grewal Akash Velu Lucas O. Souza Jérémy Forest Subutai Ahmad AI4CE 27 41 0 31 Dec 2021
Generative Kernel Continual learning Mohammad Mahdi Derakhshani Xiantong Zhen Ling Shao Cees G. M. Snoek BDL VLM 33 0 0 26 Dec 2021
Neural Network Module Decomposition and Recomposition Hiroaki Kingetsu Kenichi Kobayashi Taiji Suzuki 27 10 0 25 Dec 2021
Forward Compatible Training for Large-Scale Embedding Retrieval Systems Vivek Ramanujan Pavan Kumar Anasosalu Vasu Ali Farhadi Oncel Tuzel Hadi Pouransari VLM 24 16 0 06 Dec 2021
Hidden-Fold Networks: Random Recurrent Residuals Using Sparse Supermasks Ángel López García-Arias Masanori Hashimoto Masato Motomura Jaehoon Yu 31 5 0 24 Nov 2021
Avoiding Forgetting and Allowing Forward Transfer in Continual Learning via Sparse Networks Ghada Sokar D. Mocanu Mykola Pechenizkiy CLL 32 8 0 11 Oct 2021