v1v2v3 (latest)

Global Sparse Momentum SGD for Pruning Very Deep Neural Networks

27 September 2019

Jungong Han

Papers citing "Global Sparse Momentum SGD for Pruning Very Deep Neural Networks"

50 / 86 papers shown

Title
Dynamic Acoustic Model Architecture Optimization in Training for ASR Jingjing Xu Zijian Yang Albert Zeyer Eugen Beck Ralf Schlueter Hermann Ney 23 0 0 16 Jun 2025
GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance Jinuk Kim Marwa El Halabi W. Park Clemens JS Schaefer Deokjae Lee Yeonhong Park Jae W. Lee Hyun Oh Song MQ 148 1 0 11 May 2025
IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining Yixiao Li Xianzhi Du Ajay Jaiswal Tao Lei T. Zhao Chong-Jun Wang Jianyu Wang 77 1 0 07 Mar 2025
Hyperbolic Binary Neural Network Jun Chen Jingyang Xiang Tianxin Huang Xiangrui Zhao Yong Liu MQ 49 0 0 08 Jan 2025
Magnitude Pruning of Large Pretrained Transformer Models with a Mixture Gaussian Prior Mingxuan Zhang Y. Sun F. Liang 118 0 0 01 Nov 2024
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel Sebastian Pokutta VLM 179 10 0 23 Dec 2023
ARBiBench: Benchmarking Adversarial Robustness of Binarized Neural Networks Peng Zhao Jiehua Zhang Bowen Peng Longguang Wang Yingmei Wei Yu Liu Li Liu AAML 86 0 0 21 Dec 2023
CRISP: Hybrid Structured Sparsity for Class-aware Model Pruning Shivam Aggarwal Kuluhan Binici Tulika Mitra VLM 62 4 0 24 Nov 2023
SUBP: Soft Uniform Block Pruning for 1xN Sparse CNNs Multithreading Acceleration Jingyang Xiang Siqi Li Jun Chen Shipeng Bai Yukai Ma Guang Dai Yong-Jin Liu 66 1 0 10 Oct 2023
Dynamic Shuffle: An Efficient Channel Mixture Method Kaijun Gong Zhuowen Yin Yushu Li K. Guo Xiangmin Xu 77 0 0 04 Oct 2023
LAPP: Layer Adaptive Progressive Pruning for Compressing CNNs from Scratch P. Zhai K. Guo Fan Liu Xiaofen Xing Xiangmin Xu 58 3 0 25 Sep 2023
Understanding Self-attention Mechanism via Dynamical System Perspective Zhongzhan Huang Mingfu Liang Jinghui Qin Shan Zhong Liang Lin 75 15 0 19 Aug 2023
Differentiable Transportation Pruning Yun-qiang Li Jan van Gemert Torsten Hoefler Bert Moons E. Eleftheriou Bram-Ernst Verhoef OT 89 8 0 17 Jul 2023
Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging Max Zimmer Christoph Spiegel Sebastian Pokutta MoMe 129 14 0 29 Jun 2023
Spatial Re-parameterization for N:M Sparsity Yuxin Zhang Mingbao Lin Mingliang Xu Mengzhao Chen Yong Li 228 2 0 09 Jun 2023
Magnitude Attention-based Dynamic Pruning Jihye Back Namhyuk Ahn Jang-Hyun Kim 82 2 0 08 Jun 2023
Surrogate Lagrangian Relaxation: A Path To Retrain-free Deep Neural Network Pruning Shangli Zhou Mikhail A. Bragin Lynn Pepin Deniz Gurevin Fei Miao Caiwen Ding 66 3 0 08 Apr 2023
Towards a Smaller Student: Capacity Dynamic Distillation for Efficient Image Retrieval Yi Xie Huaidong Zhang Xuemiao Xu Jianqing Zhu Shengfeng He VLM 58 14 0 16 Mar 2023
Structured Pruning for Deep Convolutional Neural Networks: A survey Yang He Lingao Xiao 3DPC 120 145 0 01 Mar 2023
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers Chen Liang Haoming Jiang Zheng Li Xianfeng Tang Bin Yin Tuo Zhao VLM 132 25 0 19 Feb 2023
Language-Aware Multilingual Machine Translation with Self-Supervised Learning Haoran Xu Jean Maillard Vedanuj Goswami LRM 67 4 0 10 Feb 2023
Learning to Linearize Deep Neural Networks for Secure and Efficient Private Inference Souvik Kundu Shun Lu Yuke Zhang Jacqueline Liu Peter A. Beerel 40 29 0 23 Jan 2023
PATS: Sensitivity-aware Noisy Learning for Pretrained Language Models Yupeng Zhang Hongzhi Zhang Sirui Wang Wei Wu Zhoujun Li AAML 94 1 0 22 Oct 2022
Lottery Aware Sparsity Hunting: Enabling Federated Learning on Resource-Limited Edge Sara Babakniya Souvik Kundu Saurav Prakash Yue Niu Salman Avestimehr FedML 96 12 0 27 Aug 2022
WeightMom: Learning Sparse Networks using Iterative Momentum-based pruning Elvis Johnson Xiao-Ming Tang Sriramacharyulu Samudrala 42 1 0 11 Aug 2022
RepBNN: towards a precise Binary Neural Network with Enhanced Feature Map via Repeating Xulong Shi Zhi Qi Jiaxuan Cai Keqi Fu Yaru Zhao Zan Li Xuanyu Liu Hao Liu MQ 67 3 0 19 Jul 2022
Sparse Periodic Systolic Dataflow for Lowering Latency and Power Dissipation of Convolutional Neural Network Accelerators J. Heo A. Fayyazi Amirhossein Esmaili Massoud Pedram 38 3 0 30 Jun 2022
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance Qingru Zhang Simiao Zuo Chen Liang Alexander Bukharin Pengcheng He Weizhu Chen T. Zhao 81 81 0 25 Jun 2022
Learning Best Combination for Efficient N:M Sparsity Yuxin Zhang Mingbao Lin Zhihang Lin Yiting Luo Ke Li Yong Li Yongjian Wu Rongrong Ji 94 51 0 14 Jun 2022
A Directed-Evolution Method for Sparsification and Compression of Neural Networks with Application to Object Identification and Segmentation and considerations of optimal quantization using small number of bits L. Franca-Neto 20 0 0 12 Jun 2022
Neural Network Compression via Effective Filter Analysis and Hierarchical Pruning Ziqi Zhou Li Lian Yilong Yin Ze Wang 31 1 0 07 Jun 2022
FCN-Pose: A Pruned and Quantized CNN for Robot Pose Estimation for Constrained Devices M. Dantas I. R. R. Silva A. T. O. Filho Gibson B. N. Barbosa Daniel Bezerra D. Sadok J. Kelner M. Marquezini Ricardo F. D. Silva 49 1 0 26 May 2022
Compression-aware Training of Neural Networks using Frank-Wolfe Max Zimmer Christoph Spiegel Sebastian Pokutta 95 11 0 24 May 2022
The Importance of Being Parameters: An Intra-Distillation Method for Serious Gains Haoran Xu Philipp Koehn Kenton W. Murray MoMe 40 5 0 23 May 2022
Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Survey Paul Wimmer Jens Mehnert Alexandru Paul Condurache DD 98 21 0 17 May 2022
Domino Saliency Metrics: Improving Existing Channel Saliency Metrics with Structural Information Kaveena Persand Andrew Anderson David Gregg 29 0 0 04 May 2022
Searching for Network Width with Bilaterally Coupled Network Xiu Su Shan You Jiyang Xie Fei Wang Chao Qian Changshui Zhang Chang Xu 75 7 0 25 Mar 2022
Pruning Networks with Cross-Layer Ranking & k-Reciprocal Nearest Filters Mingbao Lin Liujuan Cao Yuxin Zhang Ling Shao Chia-Wen Lin Rongrong Ji 80 53 0 15 Feb 2022
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models Chen Liang Haoming Jiang Simiao Zuo Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen T. Zhao 72 14 0 06 Feb 2022
OptG: Optimizing Gradient-driven Criteria in Network Sparsity Yuxin Zhang Mingbao Lin Mengzhao Chen Yong Li Rongrong Ji 109 5 0 30 Jan 2022
RepMLPNet: Hierarchical Vision MLP with Re-parameterized Locality Xiaohan Ding Honghao Chen Xinming Zhang Jungong Han Guiguang Ding 84 74 0 21 Dec 2021
SNF: Filter Pruning via Searching the Proper Number of Filters Pengkun Liu Yaru Yue Yanjun Guo Xingxiang Tao Xiaoguang Zhou 3DPC 51 0 0 14 Dec 2021
Inf-CP: A Reliable Channel Pruning based on Channel Influence Bilan Lai Haoran Xiang S. Furao 33 0 0 05 Dec 2021
Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation Zechun Liu Kwang-Ting Cheng Dong Huang Eric P. Xing Zhiqiang Shen MQ 89 111 0 29 Nov 2021
How I Learned to Stop Worrying and Love Retraining Max Zimmer Christoph Spiegel Sebastian Pokutta CLL 94 9 0 01 Nov 2021
Joint Channel and Weight Pruning for Model Acceleration on Moblie Devices Tianli Zhao Xi Sheryl Zhang Wentao Zhu Jiaxing Wang Sen Yang Ji Liu Jian Cheng 88 2 0 15 Oct 2021
Global Vision Transformer Pruning with Hessian-Aware Saliency Huanrui Yang Hongxu Yin Maying Shen Pavlo Molchanov Hai Helen Li Jan Kautz ViT 86 45 0 10 Oct 2021
Weight Evolution: Improving Deep Neural Networks Training through Evolving Inferior Weight Values Zhenquan Lin K. Guo Xiaofen Xing Xiangmin Xu ODL 51 1 0 09 Oct 2021
Multi-Scale Aligned Distillation for Low-Resolution Detection Lu Qi Jason Kuen Jiuxiang Gu Zhe Lin Yi Wang Yukang Chen Yanwei Li Jiaya Jia 96 53 0 14 Sep 2021
$On the Compression of Neural Networks Using $\ell_0$-Norm Regularization and Weight Pruning$ On the Compression of Neural Networks Using $\ell_0$ -Norm Regularization and Weight Pruning F. Oliveira E. Batista R. Seara 59 10 0 10 Sep 2021