PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfiguration

26 January 2019

Sujay Sanghavi

Papers citing "PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfiguration"

34 / 34 papers shown

Title
EoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation Shih-yang Liu Huck Yang Nai Chit Fung Charbel Sakr Hongxu Yin ... Jan Kautz Yu-Chun Wang Pavlo Molchanov Min-Hung Chen Min-Hung Chen MQ 67 0 0 28 Oct 2024
DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference Chaojian Li Wuyang Chen Yuchen Gu Tianlong Chen Yonggan Fu Zhangyang Wang Yingyan Lin 107 0 0 16 Jul 2021
Mini-batch Serialization: CNN Training with Inter-layer Data Reuse Sangkug Lym Armand Behroozi W. Wen Ge Li Yongkee Kwon M. Erez 33 25 0 30 Sep 2018
Large Scale Language Modeling: Converging on 40GB of Text in Four Hours Raul Puri Robert M. Kirby Nikolai Yakovenko Bryan Catanzaro 57 29 0 03 Aug 2018
Norm matters: efficient and accurate normalization schemes in deep networks Elad Hoffer Ron Banner Itay Golan Daniel Soudry OffRL 59 179 0 05 Mar 2018
AMC: AutoML for Model Compression and Acceleration on Mobile Devices Yihui He Ji Lin Zhijian Liu Hanrui Wang Li Li Song Han 86 1,348 0 10 Feb 2018
Compression-aware Training of Deep Networks J. Álvarez Mathieu Salzmann 44 171 0 07 Nov 2017
Don't Decay the Learning Rate, Increase the Batch Size Samuel L. Smith Pieter-Jan Kindermans Chris Ying Quoc V. Le ODL 97 994 0 01 Nov 2017
Mixed Precision Training Paulius Micikevicius Sharan Narang Jonah Alben G. Diamos Erich Elsen ... Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh Hao Wu 149 1,792 0 10 Oct 2017
Learning Intrinsic Sparse Structures within Long Short-Term Memory W. Wen Yuxiong He Samyam Rajbhandari Minjia Zhang Wenhan Wang Fang Liu Bin Hu Yiran Chen H. Li MQ 62 141 0 15 Sep 2017
Squeeze-and-Excitation Networks Jie Hu Li Shen Samuel Albanie Gang Sun Enhua Wu 399 26,365 0 05 Sep 2017
Channel Pruning for Accelerating Very Deep Neural Networks Yihui He Xiangyu Zhang Jian Sun 196 2,521 0 19 Jul 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 120 3,675 0 08 Jun 2017
TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning W. Wen Cong Xu Feng Yan Chunpeng Wu Yandan Wang Yiran Chen Hai Helen Li 140 987 0 22 May 2017
Mask R-CNN Kaiming He Georgia Gkioxari Piotr Dollár Ross B. Girshick ObjD 344 27,129 0 20 Mar 2017
Feature Pyramid Networks for Object Detection Nayeon Lee Piotr Dollár Ross B. Girshick Kaiming He Bharath Hariharan Serge J. Belongie ObjD 446 22,040 0 09 Dec 2016
ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA Song Han Junlong Kang Huizi Mao Yiming Hu Xin Li ... Hong Luo Song Yao Yu Wang Huazhong Yang W. Dally 70 628 0 01 Dec 2016
Pruning Convolutional Neural Networks for Resource Efficient Inference Pavlo Molchanov Stephen Tyree Tero Karras Timo Aila Jan Kautz CVBM VLM 55 417 0 19 Nov 2016
Learning the Number of Neurons in Deep Networks J. Álvarez Mathieu Salzmann 171 413 0 19 Nov 2016
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Zhuowen Tu Kaiming He 495 10,305 0 16 Nov 2016
An overview of gradient descent optimization algorithms Sebastian Ruder ODL 198 6,183 0 15 Sep 2016
Pruning Filters for Efficient ConvNets Hao Li Asim Kadav Igor Durdanovic H. Samet H. Graf 3DPC 188 3,693 0 31 Aug 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 729 36,708 0 25 Aug 2016
Learning Structured Sparsity in Deep Neural Networks W. Wen Chunpeng Wu Yandan Wang Yiran Chen Hai Helen Li 168 2,337 0 12 Aug 2016
Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures Hengyuan Hu Rui Peng Yu-Wing Tai Chi-Keung Tang 64 891 0 12 Jul 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 324 7,971 0 23 May 2016
Identity Mappings in Deep Residual Networks Kaiming He Xinming Zhang Shaoqing Ren Jian Sun 344 10,172 0 16 Mar 2016
EIE: Efficient Inference Engine on Compressed Deep Neural Network Song Han Xingyu Liu Huizi Mao Jing Pu A. Pedram M. Horowitz W. Dally 118 2,455 0 04 Feb 2016
Structured Pruning of Deep Convolutional Neural Networks S. Anwar Kyuyeon Hwang Wonyong Sung 103 747 0 29 Dec 2015
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.1K 193,426 0 10 Dec 2015
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han Huizi Mao W. Dally 3DGS 245 8,821 0 01 Oct 2015
Learning both Weights and Connections for Efficient Neural Networks Song Han Jeff Pool J. Tran W. Dally CVBM 302 6,660 0 08 Jun 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 437 43,234 0 11 Feb 2015
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.5K 100,213 0 04 Sep 2014