Neural networks with late-phase weights

Neural networks with late-phase weights

25 July 2020

Seijin Kobayashi

Alexander Meulemans

Christian Henning

João Sacramento

Papers citing "Neural networks with late-phase weights"

16 / 16 papers shown

Title
State Soup: In-Context Skill Learning, Retrieval and Mixing Maciej Pióro Maciej Wołczyk Razvan Pascanu J. Oswald João Sacramento 32 1 0 12 Jun 2024
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models Nastaran Saadati Minh Pham Nasla Saleem Joshua R. Waite Aditya Balu Zhanhong Jiang Chinmay Hegde Soumik Sarkar MoMe 47 1 0 11 Apr 2024
Adapt then Unlearn: Exploring Parameter Space Semantics for Unlearning in Generative Adversarial Networks Piyush Tiwary Atri Guha Subhodip Panda Prathosh A.P. MU GAN 55 7 0 25 Sep 2023
Understanding the effect of sparsity on neural networks robustness Lukas Timpl R. Entezari Hanie Sedghi Behnam Neyshabur O. Saukh 41 12 0 22 Jun 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time Mitchell Wortsman Gabriel Ilharco S. Gadre Rebecca Roelofs Raphael Gontijo-Lopes ... Hongseok Namkoong Ali Farhadi Y. Carmon Simon Kornblith Ludwig Schmidt MoMe 54 922 1 10 Mar 2022
Low-Loss Subspace Compression for Clean Gains against Multi-Agent Backdoor Attacks Siddhartha Datta N. Shadbolt AAML 32 6 0 07 Mar 2022
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity Shiwei Liu Tianlong Chen Zahra Atashgahi Xiaohan Chen Ghada Sokar Elena Mocanu Mykola Pechenizkiy Zhangyang Wang Decebal Constantin Mocanu OOD 31 49 0 28 Jun 2021
Repulsive Deep Ensembles are Bayesian Francesco DÁngelo Vincent Fortuin UQCV BDL 59 94 0 22 Jun 2021
Reinforcement Learning, Bit by Bit Xiuyuan Lu Benjamin Van Roy Vikranth Dwaracherla M. Ibrahimi Ian Osband Zheng Wen 30 70 0 06 Mar 2021
Posterior Meta-Replay for Continual Learning Christian Henning Maria R. Cervera Francesco DÁngelo J. Oswald Regina Traber Benjamin Ehret Seijin Kobayashi Benjamin Grewe João Sacramento CLL BDL 51 55 0 01 Mar 2021
Learning Neural Network Subspaces Mitchell Wortsman Maxwell Horton Carlos Guestrin Ali Farhadi Mohammad Rastegari UQCV 27 85 0 20 Feb 2021
Large scale distributed neural network training through online distillation Rohan Anil Gabriel Pereyra Alexandre Passos Róbert Ormándi George E. Dahl Geoffrey E. Hinton FedML 278 404 0 09 Apr 2018
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 389 11,700 0 09 Mar 2017
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 276 5,675 0 05 Dec 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,892 0 15 Sep 2016
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning Y. Gal Zoubin Ghahramani UQCV BDL 285 9,145 0 06 Jun 2015