v1v2 (latest)

Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning

12 January 2023

Huan Wang

Can Qin

Yue Bai

Yun Fu

ArXiv (abs)PDF HTML Github (40★)

Papers citing "Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning"

50 / 60 papers shown

Title
Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries Chris Kolb T. Weber Bernd Bischl David Rügamer 328 1 0 04 Feb 2025
DepGraph: Towards Any Structural Pruning Gongfan Fang Xinyin Ma Mingli Song Michael Bi Mi Xinchao Wang GNN 155 270 0 30 Jan 2023
Attentive Fine-Grained Structured Sparsity for Image Restoration Junghun Oh Heewon Kim Seungjun Nah Chee Hong Jonghyun Choi Kyoung Mu Lee 98 20 0 26 Apr 2022
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 474 15,734 0 20 Dec 2021
Towards Compact CNNs via Collaborative Compression Yuchao Li Shaohui Lin Jianzhuang Liu QiXiang Ye Mengdi Wang Chia-Wen Lin Fan Yang Jincheng Ma Qi Tian Rongrong Ji 3DV 58 88 0 24 May 2021
Network Pruning That Matters: A Case Study on Retraining Variants Duong H. Le Binh-Son Hua 82 41 0 07 May 2021
Recent Advances on Neural Network Pruning at Initialization Huan Wang Can Qin Yue Bai Yulun Zhang Yun Fu CVBM 75 67 0 11 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 967 29,810 0 26 Feb 2021
Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch Aojun Zhou Yukun Ma Junnan Zhu Jianbo Liu Zhijie Zhang Kun Yuan Wenxiu Sun Hongsheng Li 207 248 0 08 Feb 2021
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks Torsten Hoefler Dan Alistarh Tal Ben-Nun Nikoli Dryden Alexandra Peste MQ 314 724 0 31 Jan 2021
Neural Pruning via Growing Regularization Huan Wang Can Qin Yulun Zhang Y. Fu 88 146 0 16 Dec 2020
Pruning Neural Networks at Initialization: Why are We Missing the Mark? Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin 65 240 0 18 Sep 2020
ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting Xiaohan Ding Tianxiang Hao Jianchao Tan Ji Liu Jungong Han Yuchen Guo Guiguang Ding 67 166 0 07 Jul 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 856 42,332 0 28 May 2020
WoodFisher: Efficient Second-Order Approximation for Neural Network Compression Sidak Pal Singh Dan Alistarh 43 28 0 29 Apr 2020
What is the State of Neural Network Pruning? Davis W. Blalock Jose Javier Gonzalez Ortiz Jonathan Frankle John Guttag 275 1,053 0 06 Mar 2020
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 278 388 0 05 Mar 2020
HRank: Filter Pruning using High-Rank Feature Map Mingbao Lin Rongrong Ji Yan Wang Yichen Zhang Baochang Zhang Yonghong Tian Ling Shao 77 728 0 24 Feb 2020
Linear Mode Connectivity and the Lottery Ticket Hypothesis Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin MoMe 159 628 0 11 Dec 2019
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 532 42,559 0 03 Dec 2019
What's Hidden in a Randomly Weighted Neural Network? Vivek Ramanujan Mitchell Wortsman Aniruddha Kembhavi Ali Farhadi Mohammad Rastegari 66 360 0 29 Nov 2019
Rigging the Lottery: Making All Tickets Winners Utku Evci Trevor Gale Jacob Menick Pablo Samuel Castro Erich Elsen 197 602 0 25 Nov 2019
Provable Filter Pruning for Efficient Neural Networks Lucas Liebenwein Cenk Baykal Harry Lang Dan Feldman Daniela Rus VLM 3DPC 74 141 0 18 Nov 2019
Importance Estimation for Neural Network Pruning Pavlo Molchanov Arun Mallya Stephen Tyree I. Frosio Jan Kautz 3DPC 81 885 0 25 Jun 2019
A Signal Propagation Perspective for Pruning Neural Networks at Initialization Namhoon Lee Thalaiyasingam Ajanthan Stephen Gould Philip Torr AAML 64 155 0 14 Jun 2019
EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis Chaoqi Wang Roger C. Grosse Sanja Fidler Guodong Zhang 65 124 0 15 May 2019
Towards Optimal Structured CNN Pruning via Generative Adversarial Learning Shaohui Lin Rongrong Ji Chenqian Yan Baochang Zhang Liujuan Cao QiXiang Ye Feiyue Huang David Doermann CVBM 53 510 0 22 Mar 2019
The State of Sparsity in Deep Neural Networks Trevor Gale Erich Elsen Sara Hooker 161 761 0 25 Feb 2019
Structured Pruning for Efficient ConvNets via Incremental Regularization Huan Wang Qiming Zhang Yuehai Wang Haoji Hu 3DPC 85 45 0 20 Nov 2018
Discrimination-aware Channel Pruning for Deep Neural Networks Zhuangwei Zhuang Mingkui Tan Bohan Zhuang Jing Liu Yong Guo Qingyao Wu Junzhou Huang Jin-Hui Zhu 121 601 0 28 Oct 2018
Rethinking the Value of Network Pruning Zhuang Liu Mingjie Sun Tinghui Zhou Gao Huang Trevor Darrell 36 1,474 0 11 Oct 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,175 0 11 Oct 2018
SNIP: Single-shot Network Pruning based on Connection Sensitivity Namhoon Lee Thalaiyasingam Ajanthan Philip Torr VLM 266 1,206 0 04 Oct 2018
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks Yang He Guoliang Kang Xuanyi Dong Yanwei Fu Yi Yang AAML VLM 69 965 0 21 Aug 2018
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 304 354 0 14 Jun 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 245 3,485 0 09 Mar 2018
Faster gaze prediction with dense networks and Fisher pruning Lucas Theis I. Korshunova Alykhan Tejani Ferenc Huszár 54 207 0 17 Jan 2018
To prune, or not to prune: exploring the efficacy of pruning for model compression Michael Zhu Suyog Gupta 197 1,281 0 05 Oct 2017
Squeeze-and-Excitation Networks Jie Hu Li Shen Samuel Albanie Gang Sun Enhua Wu 427 26,557 0 05 Sep 2017
Learning Efficient Convolutional Networks through Network Slimming Zhuang Liu Jianguo Li Zhiqiang Shen Gao Huang Shoumeng Yan Changshui Zhang 125 2,424 0 22 Aug 2017
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression Jian-Hao Luo Jianxin Wu Weiyao Lin 58 1,761 0 20 Jul 2017
Channel Pruning for Accelerating Very Deep Neural Networks Yihui He Xiangyu Zhang Jian Sun 204 2,529 0 19 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 730 132,363 0 12 Jun 2017
Efficient Processing of Deep Neural Networks: A Tutorial and Survey Vivienne Sze Yu-hsin Chen Tien-Ju Yang J. Emer AAML 3DV 120 3,026 0 27 Mar 2017
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 475 5,378 0 05 Nov 2016
Pruning Filters for Efficient ConvNets Hao Li Asim Kadav Igor Durdanovic H. Samet H. Graf 3DPC 195 3,705 0 31 Aug 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 775 36,881 0 25 Aug 2016
SGDR: Stochastic Gradient Descent with Warm Restarts I. Loshchilov Frank Hutter ODL 347 8,169 0 13 Aug 2016
Learning Structured Sparsity in Deep Neural Networks W. Wen Chunpeng Wu Yandan Wang Yiran Chen Hai Helen Li 187 2,340 0 12 Aug 2016
TensorFlow: A system for large-scale machine learning Martín Abadi P. Barham Jianmin Chen Zhiwen Chen Andy Davis ... Vijay Vasudevan Pete Warden Martin Wicke Yuan Yu Xiaoqiang Zhang GNN AI4CE 433 18,361 0 27 May 2016