Global Sparse Momentum SGD for Pruning Very Deep Neural Networks

27 September 2019

Jungong Han

Papers citing "Global Sparse Momentum SGD for Pruning Very Deep Neural Networks"

35 / 35 papers shown

Title
GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance Jinuk Kim Marwa El Halabi W. Park Clemens JS Schaefer Deokjae Lee Yeonhong Park Jae W. Lee Hyun Oh Song MQ 36 0 0 11 May 2025
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel Sebastian Pokutta VLM 55 10 0 23 Dec 2023
ARBiBench: Benchmarking Adversarial Robustness of Binarized Neural Networks Peng Zhao Jiehua Zhang Bowen Peng Longguang Wang Yingmei Wei Yu Liu Li Liu AAML 37 0 0 21 Dec 2023
Spatial Re-parameterization for N:M Sparsity Yuxin Zhang Mingbao Lin Mingliang Xu Yonghong Tian Rongrong Ji 46 2 0 09 Jun 2023
Magnitude Attention-based Dynamic Pruning Jihye Back Namhyuk Ahn Jang-Hyun Kim 38 2 0 08 Jun 2023
Surrogate Lagrangian Relaxation: A Path To Retrain-free Deep Neural Network Pruning Shangli Zhou Mikhail A. Bragin Lynn Pepin Deniz Gurevin Fei Miao Caiwen Ding 24 3 0 08 Apr 2023
Towards a Smaller Student: Capacity Dynamic Distillation for Efficient Image Retrieval Yi Xie Huaidong Zhang Xuemiao Xu Jianqing Zhu Shengfeng He VLM 21 13 0 16 Mar 2023
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers Chen Liang Haoming Jiang Zheng Li Xianfeng Tang Bin Yin Tuo Zhao VLM 29 24 0 19 Feb 2023
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance Qingru Zhang Simiao Zuo Chen Liang Alexander Bukharin Pengcheng He Weizhu Chen T. Zhao 27 78 0 25 Jun 2022
FCN-Pose: A Pruned and Quantized CNN for Robot Pose Estimation for Constrained Devices M. Dantas I. R. R. Silva A. T. O. Filho Gibson B. N. Barbosa Daniel Bezerra D. Sadok J. Kelner M. Marquezini Ricardo F. D. Silva 17 1 0 26 May 2022
Compression-aware Training of Neural Networks using Frank-Wolfe Max Zimmer Christoph Spiegel Sebastian Pokutta 31 9 0 24 May 2022
Pruning Networks with Cross-Layer Ranking & k-Reciprocal Nearest Filters Mingbao Lin Liujuan Cao Yuxin Zhang Ling Shao Chia-Wen Lin Rongrong Ji 35 51 0 15 Feb 2022
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models Chen Liang Haoming Jiang Simiao Zuo Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen T. Zhao 22 14 0 06 Feb 2022
RepMLPNet: Hierarchical Vision MLP with Re-parameterized Locality Xiaohan Ding Honghao Chen Xinming Zhang Jungong Han Guiguang Ding 19 71 0 21 Dec 2021
Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation Zechun Liu Kwang-Ting Cheng Dong Huang Eric P. Xing Zhiqiang Shen MQ 25 103 0 29 Nov 2021
Joint Channel and Weight Pruning for Model Acceleration on Moblie Devices Tianli Zhao Xi Sheryl Zhang Wentao Zhu Jiaxing Wang Sen Yang Ji Liu Jian Cheng 56 2 0 15 Oct 2021
Global Vision Transformer Pruning with Hessian-Aware Saliency Huanrui Yang Hongxu Yin Maying Shen Pavlo Molchanov Hai Helen Li Jan Kautz ViT 30 39 0 10 Oct 2021
Weight Evolution: Improving Deep Neural Networks Training through Evolving Inferior Weight Values Zhenquan Lin K. Guo Xiaofen Xing Xiangmin Xu ODL 24 1 0 09 Oct 2021
$On the Compression of Neural Networks Using $\ell_0$-Norm Regularization and Weight Pruning$ On the Compression of Neural Networks Using $\ell_0$ -Norm Regularization and Weight Pruning F. Oliveira E. Batista R. Seara 20 9 0 10 Sep 2021
Architecture Aware Latency Constrained Sparse Neural Networks Tianli Zhao Qinghao Hu Xiangyu He Weixiang Xu Jiaxing Wang Cong Leng Jian Cheng 39 0 0 01 Sep 2021
Carrying out CNN Channel Pruning in a White Box Yuxin Zhang Mingbao Lin Chia-Wen Lin Jie Chen Feiyue Huang Yongjian Wu Yonghong Tian Rongrong Ji VLM 39 58 0 24 Apr 2021
ReCU: Reviving the Dead Weights in Binary Neural Networks Zihan Xu Mingbao Lin Jianzhuang Liu Jie Chen Ling Shao Yue Gao Yonghong Tian Rongrong Ji MQ 24 81 0 23 Mar 2021
Neural Pruning via Growing Regularization Huan Wang Can Qin Yulun Zhang Y. Fu 37 144 0 16 Dec 2020
FreezeNet: Full Performance by Reduced Storage Costs Paul Wimmer Jens Mehnert Alexandru Paul Condurache 31 13 0 28 Nov 2020
AutoPruning for Deep Neural Network with Dynamic Channel Masking Baopu Li Yanwen Fan Zhihong Pan Gang Zhang 36 1 0 22 Oct 2020
Rotated Binary Neural Network Mingbao Lin Rongrong Ji Zi-Han Xu Baochang Zhang Yan Wang Yongjian Wu Feiyue Huang Chia-Wen Lin MQ 19 129 0 28 Sep 2020
Training Sparse Neural Networks using Compressed Sensing Jonathan W. Siegel Jianhong Chen Pengchuan Zhang Jinchao Xu 26 5 0 21 Aug 2020
T-Basis: a Compact Representation for Neural Networks Anton Obukhov M. Rakhuba Stamatios Georgoulis Menelaos Kanakis Dengxin Dai Luc Van Gool 41 27 0 13 Jul 2020
Revisiting Loss Modelling for Unstructured Pruning César Laurent Camille Ballas Thomas George Nicolas Ballas Pascal Vincent 30 14 0 22 Jun 2020
Real-time Universal Style Transfer on High-resolution Images via Zero-channel Pruning Jie An Tao Li Haozhi Huang Li Shen Xuan Wang Yongyi Tang Jinwen Ma Wei Liu Jiebo Luo 3DH 3DPC 31 18 0 16 Jun 2020
Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors C. Coelho Aki Kuusela Shane Li Zhuang Hao T. Aarrestad Vladimir Loncar J. Ngadiuba M. Pierini Adrian Alan Pol S. Summers MQ 34 178 0 15 Jun 2020
HourNAS: Extremely Fast Neural Architecture Search Through an Hourglass Lens Zhaohui Yang Yunhe Wang Xinghao Chen Jianyuan Guo Wei Zhang Chao Xu Chunjing Xu Dacheng Tao Chang Xu 39 17 0 29 May 2020
Movement Pruning: Adaptive Sparsity by Fine-Tuning Victor Sanh Thomas Wolf Alexander M. Rush 32 470 0 15 May 2020
SASL: Saliency-Adaptive Sparsity Learning for Neural Network Acceleration Jun Shi Jianfeng Xu K. Tasaka Zhibo Chen 6 25 0 12 Mar 2020
Structured Pruning for Efficient ConvNets via Incremental Regularization Huan Wang Qiming Zhang Yuehai Wang Haoji Hu 3DPC 40 45 0 20 Nov 2018