v1v2v3 (latest)

Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization

15 February 2019

Papers citing "Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization"

50 / 56 papers shown

Title
Sparse-to-Sparse Training of Diffusion Models Inês Cardoso Oliveira Decebal Constantin Mocanu Luis A. Leiva DiffM 155 0 0 30 Apr 2025
E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation Boqian Wu Q. Xiao Shiwei Liu Lu Yin Mykola Pechenizkiy Decebal Constantin Mocanu M. V. Keulen Elena Mocanu MedIm 147 6 0 20 Feb 2025
Pushing the Limits of Sparsity: A Bag of Tricks for Extreme Pruning Andy Li A. Durrant Milan Markovic Lu Yin Georgios Leontidis Tianlong Chen Lu Yin Georgios Leontidis 154 0 0 20 Nov 2024
More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routing Sagi Shaier Francisco Pereira Katharina von der Wense Lawrence E Hunter Matt Jones MoE 106 0 0 10 Oct 2024
Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness Boqian Wu Q. Xiao Shunxin Wang N. Strisciuglio Mykola Pechenizkiy M. V. Keulen Decebal Constantin Mocanu Elena Mocanu OOD 3DH 201 3 0 03 Oct 2024
Always-Sparse Training by Growing Connections with Guided Stochastic Exploration Mike Heddes Narayan Srinivasa T. Givargis Alexandru Nicolau 258 0 0 12 Jan 2024
Rethinking the Value of Network Pruning Zhuang Liu Mingjie Sun Tinghui Zhou Gao Huang Trevor Darrell 42 1,477 0 11 Oct 2018
Grow and Prune Compact, Fast, and Accurate LSTMs Xiaoliang Dai Hongxu Yin N. Jha VLM SyDa 52 91 0 30 May 2018
The loss landscape of overparameterized neural networks Y. Cooper 74 79 0 26 Apr 2018
Measuring the Intrinsic Dimension of Objective Landscapes Chunyuan Li Heerad Farkhoor Rosanne Liu J. Yosinski 89 416 0 24 Apr 2018
A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers Tianyun Zhang Shaokai Ye Kaiqi Zhang Jian Tang Wujie Wen M. Fardad Yanzhi Wang 69 438 0 10 Apr 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 277 3,489 0 09 Mar 2018
Sensitivity and Generalization in Neural Networks: an Empirical Study Roman Novak Yasaman Bahri Daniel A. Abolafia Jeffrey Pennington Jascha Narain Sohl-Dickstein AAML 97 442 0 23 Feb 2018
Training wide residual networks for deployment using a single bit for each weight Mark D Mcdonnell MQ 87 71 0 23 Feb 2018
Recovering from Random Pruning: On the Plasticity of Deep Convolutional Neural Networks Deepak Mittal S. Bhardwaj Mitesh M. Khapra Balaraman Ravindran VLM 78 66 0 31 Jan 2018
Theory of Deep Learning IIb: Optimization Properties of SGD Chiyuan Zhang Q. Liao Alexander Rakhlin Brando Miranda Noah Golowich T. Poggio ODL 61 71 0 07 Jan 2018
Theory of Deep Learning III: explaining the non-overfitting puzzle T. Poggio Kenji Kawaguchi Q. Liao Brando Miranda Lorenzo Rosasco Xavier Boix Jack Hidary H. Mhaskar ODL 81 128 0 30 Dec 2017
Deep Rewiring: Training very sparse deep networks G. Bellec David Kappel Wolfgang Maass Robert Legenstein BDL 189 279 0 14 Nov 2017
NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigm Xiaoliang Dai Hongxu Yin N. Jha DD 90 239 0 06 Nov 2017
SGD Learns Over-parameterized Networks that Provably Generalize on Linearly Separable Data Alon Brutzkus Amir Globerson Eran Malach Shai Shalev-Shwartz MLT 156 279 0 27 Oct 2017
To prune, or not to prune: exploring the efficacy of pruning for model compression Michael Zhu Suyog Gupta 202 1,282 0 05 Oct 2017
Learning Efficient Convolutional Networks through Network Slimming Zhuang Liu Jianguo Li Zhiqiang Shen Gao Huang Shoumeng Yan Changshui Zhang 140 2,427 0 22 Aug 2017
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression Jian-Hao Luo Jianxin Wu Weiyao Lin 58 1,761 0 20 Jul 2017
Channel Pruning for Accelerating Very Deep Neural Networks Yihui He Xiangyu Zhang Jian Sun 211 2,531 0 19 Jul 2017
Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science Decebal Constantin Mocanu Elena Mocanu Peter Stone Phuong H. Nguyen M. Gibescu A. Liotta 184 637 0 15 Jul 2017
Data-Driven Sparse Structure Selection for Deep Neural Networks Zehao Huang Naiyan Wang 122 563 0 05 Jul 2017
Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes Lei Wu Zhanxing Zhu E. Weinan ODL 67 221 0 30 Jun 2017
NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Maps Alessandro Aimar Hesham Mostafa Enrico Calabrese A. Rios-Navarro Ricardo Tapiador-Morales ... Moritz B. Milde Federico Corradi A. Linares-Barranco Shih-Chii Liu T. Delbruck 142 245 0 05 Jun 2017
Exploring Sparsity in Recurrent Neural Networks Sharan Narang Erich Elsen G. Diamos Shubho Sengupta 76 312 0 17 Apr 2017
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 1.2K 20,900 0 17 Apr 2017
Sharp Minima Can Generalize For Deep Nets Laurent Dinh Razvan Pascanu Samy Bengio Yoshua Bengio ODL 145 774 0 15 Mar 2017
An empirical analysis of the optimization of deep network loss surfaces Daniel Jiwoong Im Michael Tao K. Branson ODL 87 63 0 13 Dec 2016
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 356 4,636 0 10 Nov 2016
Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations Itay Hubara Matthieu Courbariaux Daniel Soudry Ran El-Yaniv Yoshua Bengio MQ 168 1,870 0 22 Sep 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 442 2,946 0 15 Sep 2016
Pruning Filters for Efficient ConvNets Hao Li Asim Kadav Igor Durdanovic H. Samet H. Graf 3DPC 195 3,707 0 31 Aug 2016
Learning Structured Sparsity in Deep Neural Networks W. Wen Chunpeng Wu Yandan Wang Yiran Chen Hai Helen Li 199 2,341 0 12 Aug 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 362 8,005 0 23 May 2016
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks Tim Salimans Diederik P. Kingma ODL 203 1,946 0 25 Feb 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.3K 194,641 0 10 Dec 2015
Unitary Evolution Recurrent Neural Networks Martín Arjovsky Amar Shah Yoshua Bengio ODL 84 771 0 20 Nov 2015
ACDC: A Structured Efficient Linear Layer Marcin Moczulski Misha Denil J. Appleyard Nando de Freitas 95 98 0 18 Nov 2015
Structured Transforms for Small-Footprint Deep Learning Vikas Sindhwani Tara N. Sainath Sanjiv Kumar 70 240 0 06 Oct 2015
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han Huizi Mao W. Dally 3DGS 263 8,864 0 01 Oct 2015
Learning both Weights and Connections for Efficient Neural Networks Song Han Jeff Pool J. Tran W. Dally CVBM 316 6,715 0 08 Jun 2015
Fast ConvNets Using Group-wise Brain Damage V. Lebedev Victor Lempitsky AAML 201 449 0 08 Jun 2015
Compressing Neural Networks with the Hashing Trick Wenlin Chen James T. Wilson Stephen Tyree Kilian Q. Weinberger Yixin Chen 168 1,191 0 19 Apr 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 367 19,745 0 09 Mar 2015
Deep Fried Convnets Zichao Yang Marcin Moczulski Misha Denil Nando de Freitas Alex Smola Le Song Ziyu Wang 129 267 0 22 Dec 2014
Qualitatively characterizing neural network optimization problems Ian Goodfellow Oriol Vinyals Andrew M. Saxe ODL 118 524 0 19 Dec 2014