The State of Sparsity in Deep Neural Networks

25 February 2019

Papers citing "The State of Sparsity in Deep Neural Networks"

50 / 155 papers shown

Title
What Matters In The Structured Pruning of Generative Language Models? Michael Santacroce Zixin Wen Yelong Shen Yuan-Fang Li 21 33 0 07 Feb 2023
Fast, Differentiable and Sparse Top-k: a Convex Analysis Perspective Michael E. Sander J. Puigcerver Josip Djolonga Gabriel Peyré Mathieu Blondel 21 18 0 02 Feb 2023
An Empirical Study on the Transferability of Transformer Modules in Parameter-Efficient Fine-Tuning Mohammad AkbarTajari S. Rajaee Mohammad Taher Pilehvar 13 2 0 01 Feb 2023
Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning Huan Wang Can Qin Yue Bai Yun Fu 37 20 0 12 Jan 2023
NIRVANA: Neural Implicit Representations of Videos with Adaptive Networks and Autoregressive Patch-wise Modeling Shishira R. Maiya Sharath Girish Max Ehrlich Hanyu Wang Kwot Sin Lee Patrick Poirson Pengxiang Wu Chen Wang Abhinav Shrivastava VGen 47 40 0 30 Dec 2022
COLT: Cyclic Overlapping Lottery Tickets for Faster Pruning of Convolutional Neural Networks Md. Ismail Hossain Mohammed Rakib M. M. L. Elahi Nabeel Mohammed Shafin Rahman 21 1 0 24 Dec 2022
Pruning On-the-Fly: A Recoverable Pruning Method without Fine-tuning Danyang Liu Xue Liu 28 0 0 24 Dec 2022
Constraints on the design of neuromorphic circuits set by the properties of neural population codes S. Panzeri Ella Janotte Alejandro Pequeño-Zurro Jacopo Bonato Chiara Bartolozzi 26 2 0 08 Dec 2022
The Effect of Data Dimensionality on Neural Network Prunability Zachary Ankner Alex Renda Gintare Karolina Dziugaite Jonathan Frankle Tian Jin 34 5 0 01 Dec 2022
MegaBlocks: Efficient Sparse Training with Mixture-of-Experts Trevor Gale Deepak Narayanan C. Young Matei A. Zaharia MoE 25 103 0 29 Nov 2022
You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets Tianjin Huang Tianlong Chen Meng Fang Vlado Menkovski Jiaxu Zhao ... Yulong Pei Decebal Constantin Mocanu Zhangyang Wang Mykola Pechenizkiy Shiwei Liu GNN 47 14 0 28 Nov 2022
Intriguing Properties of Compression on Multilingual Models Kelechi Ogueji Orevaoghene Ahia Gbemileke Onilude Sebastian Gehrmann Sara Hooker Julia Kreutzer 21 12 0 04 Nov 2022
LOFT: Finding Lottery Tickets through Filter-wise Training Qihan Wang Chen Dun Fangshuo Liao C. Jermaine Anastasios Kyrillidis 27 3 0 28 Oct 2022
Compressing And Debiasing Vision-Language Pre-Trained Models for Visual Question Answering Q. Si Yuanxin Liu Zheng Lin Peng Fu Weiping Wang VLM 42 1 0 26 Oct 2022
GMP*: Well-Tuned Gradual Magnitude Pruning Can Outperform Most BERT-Pruning Methods Eldar Kurtic Dan Alistarh AI4MH 38 14 0 12 Oct 2022
Advancing Model Pruning via Bi-level Optimization Yihua Zhang Yuguang Yao Parikshit Ram Pu Zhao Tianlong Chen Min-Fong Hong Yanzhi Wang Sijia Liu 49 68 0 08 Oct 2022
EPIC TTS Models: Empirical Pruning Investigations Characterizing Text-To-Speech Models Perry Lam Huayun Zhang Nancy F. Chen Berrak Sisman 19 2 0 22 Sep 2022
One-shot Network Pruning at Initialization with Discriminative Image Patches Yinan Yang Yu Wang Yi Ji Heng Qi Jien Kato VLM 31 4 0 13 Sep 2022
Seeking Interpretability and Explainability in Binary Activated Neural Networks Benjamin Leblanc Pascal Germain FAtt 40 1 0 07 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 33 109 0 31 Aug 2022
Complexity-Driven CNN Compression for Resource-constrained Edge AI Muhammad Zawish Steven Davy L. Abraham 41 16 0 26 Aug 2022
Doge Tickets: Uncovering Domain-general Language Models by Playing Lottery Tickets Yi Yang Chen Zhang Benyou Wang Dawei Song LRM 24 6 0 20 Jul 2022
SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevance Edouard Yvinec Arnaud Dapogny Matthieu Cord Kévin Bailly 42 9 0 08 Jul 2022
Cut Inner Layers: A Structured Pruning Strategy for Efficient U-Net GANs Bo-Kyeong Kim Shinkook Choi Hancheol Park 21 4 0 29 Jun 2022
Understanding the effect of sparsity on neural networks robustness Lukas Timpl R. Entezari Hanie Sedghi Behnam Neyshabur O. Saukh 41 12 0 22 Jun 2022
Sparse Double Descent: Where Network Pruning Aggravates Overfitting Zhengqi He Zeke Xie Quanzhi Zhu Zengchang Qin 79 27 0 17 Jun 2022
Spartan: Differentiable Sparsity via Regularized Transportation Kai Sheng Tai Taipeng Tian Ser-Nam Lim 31 11 0 27 May 2022
Compression-aware Training of Neural Networks using Frank-Wolfe Max Zimmer Christoph Spiegel Sebastian Pokutta 29 9 0 24 May 2022
What Do Compressed Multilingual Machine Translation Models Forget? Alireza Mohammadshahi Vassilina Nikoulina Alexandre Berard Caroline Brun James Henderson Laurent Besacier AI4CE 44 9 0 22 May 2022
Sharp asymptotics on the compression of two-layer neural networks Mohammad Hossein Amani Simone Bombari Marco Mondelli Rattana Pukdee Stefano Rini MLT 27 0 0 17 May 2022
Federated Progressive Sparsification (Purge, Merge, Tune)+ Dimitris Stripelis Umang Gupta Greg Ver Steeg J. Ambite FedML 23 9 0 26 Apr 2022
Accelerating Attention through Gradient-Based Learned Runtime Pruning Zheng Li Soroush Ghodrati Amir Yazdanbakhsh H. Esmaeilzadeh Mingu Kang 27 17 0 07 Apr 2022
Monarch: Expressive Structured Matrices for Efficient and Accurate Training Tri Dao Beidi Chen N. Sohoni Arjun D Desai Michael Poli Jessica Grogan Alexander Liu Aniruddh Rao Atri Rudra Christopher Ré 26 87 0 01 Apr 2022
PADA: Pruning Assisted Domain Adaptation for Self-Supervised Speech Representations L. D. Prasad Sreyan Ghosh S. Umesh 25 12 0 31 Mar 2022
LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT Rui Wang Qibing Bai Junyi Ao Long Zhou Zhixiang Xiong Zhihua Wei Yu Zhang Tom Ko Haizhou Li 34 62 0 29 Mar 2022
Playing Lottery Tickets in Style Transfer Models Meihao Kong Jing Huo Wenbin Li Jing Wu Yu-Kun Lai Yang Gao 27 1 0 25 Mar 2022
Sparse Federated Learning with Hierarchical Personalized Models Xiaofeng Liu Qing Wang Yunfeng Shao Yinchuan Li FedML 47 11 0 25 Mar 2022
Improve Convolutional Neural Network Pruning by Maximizing Filter Variety Nathan Hubens M. Mancas B. Gosselin Marius Preda T. Zaharia 21 2 0 11 Mar 2022
Don't Be So Dense: Sparse-to-Sparse GAN Training Without Sacrificing Performance Shiwei Liu Yuesong Tian Tianlong Chen Li Shen 36 8 0 05 Mar 2022
Rare Gems: Finding Lottery Tickets at Initialization Kartik K. Sreenivasan Jy-yong Sohn Liu Yang Matthew Grinde Alliot Nagle Hongyi Wang Eric P. Xing Kangwook Lee Dimitris Papailiopoulos 30 42 0 24 Feb 2022
Pruning Networks with Cross-Layer Ranking & k-Reciprocal Nearest Filters Mingbao Lin Liujuan Cao Yuxin Zhang Ling Shao Chia-Wen Lin Rongrong Ji 35 51 0 15 Feb 2022
Evolving Neural Networks with Optimal Balance between Information Flow and Connections Cost A. Khalili A. Bouchachia 17 0 0 12 Feb 2022
Selective Network Linearization for Efficient Private Inference Minsu Cho Ameya Joshi S. Garg Brandon Reagen C. Hegde 19 43 0 04 Feb 2022
Training Structured Neural Networks Through Manifold Identification and Variance Reduction Zih-Syuan Huang Ching-pei Lee AAML 48 9 0 05 Dec 2021
How Well Do Sparse Imagenet Models Transfer? Eugenia Iofinova Alexandra Peste Mark Kurtz Dan Alistarh 27 38 0 26 Nov 2021
Toward Compact Parameter Representations for Architecture-Agnostic Neural Network Compression Yuezhou Sun Wenlong Zhao Lijun Zhang Xiao Liu Hui Guan Matei A. Zaharia 26 0 0 19 Nov 2021
The Efficiency Misnomer Daoyuan Chen Liuyi Yao Dawei Gao Ashish Vaswani Yaliang Li 36 99 0 25 Oct 2021
When to Prune? A Policy towards Early Structural Pruning Maying Shen Pavlo Molchanov Hongxu Yin J. Álvarez VLM 28 53 0 22 Oct 2021
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning Soufiane Hayou Bo He Gintare Karolina Dziugaite 37 2 0 22 Oct 2021
Training Deep Neural Networks with Joint Quantization and Pruning of Weights and Activations Xinyu Zhang Ian Colbert Ken Kreutz-Delgado Srinjoy Das MQ 32 11 0 15 Oct 2021