Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization

15 February 2019

Papers citing "Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization"

26 / 76 papers shown

Title
Can Subnetwork Structure be the Key to Out-of-Distribution Generalization? Dinghuai Zhang Kartik Ahuja Yilun Xu Yisen Wang Aaron Courville OOD 20 95 0 05 Jun 2021
Initialization and Regularization of Factorized Neural Layers M. Khodak Neil A. Tenenholtz Lester W. Mackey Nicolò Fusi 65 56 0 03 May 2021
Effective Sparsification of Neural Networks with Global Sparsity Constraint Xiao Zhou Weizhong Zhang Hang Xu Tong Zhang 21 61 0 03 May 2021
Lottery Jackpots Exist in Pre-trained Models Yuxin Zhang Mingbao Lin Yan Wang Rongrong Ji Rongrong Ji 30 15 0 18 Apr 2021
Recent Advances on Neural Network Pruning at Initialization Huan Wang Can Qin Yue Bai Yulun Zhang Yun Fu CVBM 33 64 0 11 Mar 2021
Sparse Training Theory for Scalable and Efficient Agents Decebal Constantin Mocanu Elena Mocanu T. Pinto Selima Curci Phuong H. Nguyen M. Gibescu D. Ernst Z. Vale 45 17 0 02 Mar 2021
An Information-Theoretic Justification for Model Pruning Berivan Isik Tsachy Weissman Albert No 95 35 0 16 Feb 2021
Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks Itay Hubara Brian Chmiel Moshe Island Ron Banner S. Naor Daniel Soudry 53 111 0 16 Feb 2021
Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch Aojun Zhou Yukun Ma Junnan Zhu Jianbo Liu Zhijie Zhang Kun Yuan Wenxiu Sun Hongsheng Li 50 239 0 08 Feb 2021
Time-Correlated Sparsification for Communication-Efficient Federated Learning Emre Ozfatura Kerem Ozfatura Deniz Gunduz FedML 38 47 0 21 Jan 2021
AttentionLite: Towards Efficient Self-Attention Models for Vision Souvik Kundu Sairam Sundaresan 18 22 0 21 Dec 2020
Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders Zahra Atashgahi Ghada Sokar T. Lee Elena Mocanu Decebal Constantin Mocanu Raymond N. J. Veldhuis Mykola Pechenizkiy 19 37 0 01 Dec 2020
Rethinking Weight Decay For Efficient Neural Network Pruning Hugo Tessier Vincent Gripon Mathieu Léonardon M. Arzel T. Hannagan David Bertrand 26 25 0 20 Nov 2020
FPRaker: A Processing Element For Accelerating Neural Network Training Omar Mohamed Awad Mostafa Mahmoud Isak Edo Vivancos Ali Hadi Zadeh Ciaran Bannon Anand Jayarajan Gennady Pekhimenko Andreas Moshovos 22 15 0 15 Oct 2020
Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win Utku Evci Yani Andrew Ioannou Cem Keskin Yann N. Dauphin 27 87 0 07 Oct 2020
Procrustes: a Dataflow and Accelerator for Sparse Deep Neural Network Training Dingqing Yang Amin Ghasemazar X. Ren Maximilian Golub G. Lemieux Mieszko Lis 8 48 0 23 Sep 2020
Training highly effective connectivities within neural networks with randomly initialized, fixed weights Cristian Ivan Razvan V. Florian 19 4 0 30 Jun 2020
Progressive Skeletonization: Trimming more fat from a network at initialization Pau de Jorge Amartya Sanyal Harkirat Singh Behl Philip Torr Grégory Rogez P. Dokania 31 95 0 16 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 45 98 0 05 Jun 2020
Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layers Junjie Liu Zhe Xu Runbin Shi R. Cheung Hayden Kwok-Hay So 14 119 0 14 May 2020
Sparse Weight Activation Training Md Aamir Raihan Tor M. Aamodt 34 73 0 07 Jan 2020
Sparse Networks from Scratch: Faster Training without Losing Performance Tim Dettmers Luke Zettlemoyer 20 334 0 10 Jul 2019
On improving deep learning generalization with adaptive sparse connectivity Shiwei Liu Decebal Constantin Mocanu Mykola Pechenizkiy ODL 17 7 0 27 Jun 2019
Intrinsically Sparse Long Short-Term Memory Networks Shiwei Liu Decebal Constantin Mocanu Mykola Pechenizkiy 24 9 0 26 Jan 2019
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 183 1,185 0 30 Nov 2014