Are Straight-Through gradients and Soft-Thresholding all you need for
Sparse Training?

Are Straight-Through gradients and Soft-Thresholding all you need for Sparse Training?

2 December 2022

A. Vanderschueren

Christophe De Vleeschouwer

Papers citing "Are Straight-Through gradients and Soft-Thresholding all you need for Sparse Training?"

13 / 13 papers shown

Title
SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization T. Yasuda Kyriakos Axiotis Gang Fu M. Bateni Vahab Mirrokni 105 0 0 27 Feb 2024
Neural Pruning via Growing Regularization Huan Wang Can Qin Yulun Zhang Y. Fu 57 144 0 16 Dec 2020
The Two Regimes of Deep Network Training Guillaume Leclerc Aleksander Madry 49 45 0 24 Feb 2020
Soft Threshold Weight Reparameterization for Learnable Sparsity Aditya Kusupati Vivek Ramanujan Raghav Somani Mitchell Wortsman Prateek Jain Sham Kakade Ali Farhadi 94 243 0 08 Feb 2020
Rigging the Lottery: Making All Tickets Winners Utku Evci Trevor Gale Jacob Menick Pablo Samuel Castro Erich Elsen 136 592 0 25 Nov 2019
Discovering Neural Wirings Mitchell Wortsman Ali Farhadi Mohammad Rastegari AI4CE 85 121 0 03 Jun 2019
SNIP: Single-shot Network Pruning based on Connection Sensitivity Namhoon Lee Thalaiyasingam Ajanthan Philip Torr VLM 209 1,190 0 04 Oct 2018
To prune, or not to prune: exploring the efficacy of pruning for model compression Michael Zhu Suyog Gupta 132 1,262 0 05 Oct 2017
Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science Decebal Constantin Mocanu Elena Mocanu Peter Stone Phuong H. Nguyen M. Gibescu A. Liotta 107 625 0 15 Jul 2017
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 1.0K 20,747 0 17 Apr 2017
Pruning Filters for Efficient ConvNets Hao Li Asim Kadav Igor Durdanovic H. Samet H. Graf 3DPC 162 3,687 0 31 Aug 2016
Learning both Weights and Connections for Efficient Neural Networks Song Han Jeff Pool J. Tran W. Dally CVBM 226 6,628 0 08 Jun 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 168 18,534 0 06 Feb 2015