Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks

16 February 2021

Papers citing "Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks"

28 / 78 papers shown

Title
Vision Models Can Be Efficiently Specialized via Few-Shot Task-Aware Compression Denis Kuznedelev Soroush Tabesh Kimia Noorbakhsh Elias Frantar Sara Beery Eldar Kurtic Dan Alistarh MQ VLM 26 2 0 25 Mar 2023
Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency Vithursan Thangarasa Shreyas Saxena Abhay Gupta Sean Lie 31 3 0 21 Mar 2023
Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution Jiamian Wang Huan Wang Yulun Zhang Yun Fu Zhiqiang Tao SupR 35 2 0 16 Mar 2023
Gradient-Free Structured Pruning with Unlabeled Data Azade Nova H. Dai Dale Schuurmans SyDa 40 20 0 07 Mar 2023
Bi-directional Masks for Efficient N:M Sparse Training Yuxin Zhang Yiting Luo Mingbao Lin Mingliang Xu Jingjing Xie Rongrong Ji Rongrong Ji 52 15 0 13 Feb 2023
SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks Mahdi Nikdan Tommaso Pegolotti Eugenia Iofinova Eldar Kurtic Dan Alistarh 26 11 0 09 Feb 2023
ZipLM: Inference-Aware Structured Pruning of Language Models Eldar Kurtic Elias Frantar Dan Alistarh MQ 25 24 0 07 Feb 2023
STEP: Learning N:M Structured Sparsity Masks from Scratch with Precondition Yucheng Lu Shivani Agrawal Suvinay Subramanian Oleg Rybakov Chris De Sa Amir Yazdanbakhsh 21 16 0 02 Feb 2023
Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction Bowen Lei Dongkuan Xu Ruqi Zhang Shuren He Bani Mallick 35 6 0 09 Jan 2023
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot Elias Frantar Dan Alistarh VLM 33 638 0 02 Jan 2023
SNIPER Training: Single-Shot Sparse Training for Text-to-Speech Perry Lam Huayun Zhang Nancy F. Chen Berrak Sisman Dorien Herremans VLM 30 0 0 14 Nov 2022
Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Training Mingliang Xu Gongrui Nan Yuxin Zhang Rongrong Ji Rongrong Ji MQ 18 3 0 12 Nov 2022
CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Models Denis Kuznedelev Eldar Kurtic Elias Frantar Dan Alistarh VLM ViT 19 11 0 14 Oct 2022
Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning Elias Frantar Sidak Pal Singh Dan Alistarh MQ 28 216 0 24 Aug 2022
CrAM: A Compression-Aware Minimizer Alexandra Peste Adrian Vladu Eldar Kurtic Christoph H. Lampert Dan Alistarh 37 8 0 28 Jul 2022
DRESS: Dynamic REal-time Sparse Subnets Zhongnan Qu Syed Shakib Sarwar Xin Dong Yuecheng Li Huseyin Ekin Sumbul B. D. Salvo 3DH 18 1 0 01 Jul 2022
Learning Best Combination for Efficient N:M Sparsity Yuxin Zhang Mingbao Lin Zhihang Lin Yiting Luo Ke Li Rongrong Ji Yongjian Wu Rongrong Ji 29 49 0 14 Jun 2022
Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Survey Paul Wimmer Jens Mehnert A. P. Condurache DD 34 20 0 17 May 2022
Attentive Fine-Grained Structured Sparsity for Image Restoration Junghun Oh Heewon Kim Seungjun Nah Chee Hong Jonghyun Choi Kyoung Mu Lee 21 18 0 26 Apr 2022
Bimodal Distributed Binarized Neural Networks T. Rozen Moshe Kimhi Brian Chmiel A. Mendelson Chaim Baskin MQ 50 4 0 05 Apr 2022
Minimum Variance Unbiased N:M Sparsity for the Neural Gradients Brian Chmiel Itay Hubara Ron Banner Daniel Soudry 21 10 0 21 Mar 2022
Accelerating DNN Training with Structured Data Gradient Pruning Bradley McDanel Helia Dinh J. Magallanes 17 7 0 01 Feb 2022
SPDY: Accurate Pruning with Speedup Guarantees Elias Frantar Dan Alistarh 41 33 0 31 Jan 2022
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration Shiwei Liu Tianlong Chen Xiaohan Chen Zahra Atashgahi Lu Yin Huanyu Kou Li Shen Mykola Pechenizkiy Zhangyang Wang Decebal Constantin Mocanu 40 111 0 19 Jun 2021
1xN Pattern for Pruning Convolutional Neural Networks Mingbao Lin Yu-xin Zhang Yuchao Li Bohong Chen Rongrong Ji Mengdi Wang Shen Li Yonghong Tian Rongrong Ji 3DPC 33 40 0 31 May 2021
Search Spaces for Neural Model Training Darko Stosic Dusan Stosic 25 4 0 27 May 2021
Accelerating Sparse Deep Neural Networks Asit K. Mishra J. Latorre Jeff Pool Darko Stosic Dusan Stosic Ganesh Venkatesh Chong Yu Paulius Micikevicius 16 221 0 16 Apr 2021
Faster Neural Network Training with Approximate Tensor Operations Menachem Adelman Kfir Y. Levy Ido Hakimi M. Silberstein 29 26 0 21 May 2018