Learning Sparse Neural Networks through $L_0$ Regularization

4 December 2017

Papers citing "Learning Sparse Neural Networks through $L_0$ Regularization"

50 / 221 papers shown

Title
Maximizing information from chemical engineering data sets: Applications to machine learning Alexander Thebelt Johannes Wiebe Jan Kronqvist Calvin Tsay Ruth Misener AI4CE 50 68 0 25 Jan 2022
Pruning-aware Sparse Regularization for Network Pruning Nanfei Jiang Xu Zhao Chaoyang Zhao Yongqi An Ming Tang Jinqiao Wang 3DPC 29 12 0 18 Jan 2022
Transferability in Deep Learning: A Survey Junguang Jiang Yang Shu Jianmin Wang Mingsheng Long OOD 34 101 0 15 Jan 2022
Recursive Least Squares for Training and Pruning Convolutional Neural Networks Tianzong Yu Chunyuan Zhang Yuan Wang Meng-tao Ma Qingwei Song 37 1 0 13 Jan 2022
Automatic Mixed-Precision Quantization Search of BERT Changsheng Zhao Ting Hua Yilin Shen Qian Lou Hongxia Jin MQ 25 19 0 30 Dec 2021
Compact Multi-level Sparse Neural Networks with Input Independent Dynamic Rerouting Minghai Qin Tianyun Zhang Fei Sun Yen-kuang Chen M. Fardad Yanzhi Wang Yuan Xie 49 0 0 21 Dec 2021
Load-balanced Gather-scatter Patterns for Sparse Deep Neural Networks Fei Sun Minghai Qin Tianyun Zhang Xiaolong Ma Haoran Li Junwen Luo Zihao Zhao Yen-kuang Chen Yuan Xie 27 1 0 20 Dec 2021
From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression Runxin Xu Fuli Luo Chengyu Wang Baobao Chang Jun Huang Songfang Huang Fei Huang VLM 27 25 0 14 Dec 2021
Sparse Interventions in Language Models with Differentiable Masking Nicola De Cao Leon Schmid Dieuwke Hupkes Ivan Titov 40 27 0 13 Dec 2021
Training Neural Networks with Fixed Sparse Masks Yi-Lin Sung Varun Nair Colin Raffel FedML 32 197 0 18 Nov 2021
DICE: Leveraging Sparsification for Out-of-Distribution Detection Yiyou Sun Yixuan Li OODD 38 152 0 18 Nov 2021
Efficient Neural Network Training via Forward and Backward Propagation Sparsification Xiao Zhou Weizhong Zhang Zonghao Chen Shizhe Diao Tong Zhang 40 46 0 10 Nov 2021
Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysis Ying Zeng Sijie Mai Haifeng Hu 29 19 0 10 Nov 2021
Support Recovery with Stochastic Gates: Theory and Application for Linear Models Soham Jana Henry Li Yutaro Yamada Ofir Lindenbaum 28 5 0 29 Oct 2021
Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domains Christian Gumbsch Martin Volker Butz Georg Martius AI4CE 28 21 0 29 Oct 2021
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning Soufiane Hayou Bo He Gintare Karolina Dziugaite 37 2 0 22 Oct 2021
End-to-End Supermask Pruning: Learning to Prune Image Captioning Models J. Tan C. Chan Joon Huang Chuah VLM 54 16 0 07 Oct 2021
Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modeling Kyuhong Shim Iksoo Choi Wonyong Sung Jungwook Choi 40 15 0 07 Oct 2021
A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning Iris A. M. Huijben W. Kool Max B. Paulus Ruud J. G. van Sloun 30 94 0 04 Oct 2021
Powerpropagation: A sparsity inducing weight reparameterisation Jonathan Richard Schwarz Siddhant M. Jayakumar Razvan Pascanu P. Latham Yee Whye Teh 98 54 0 01 Oct 2021
The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders Han He Jinho Choi 56 87 0 14 Sep 2021
$On the Compression of Neural Networks Using $\ell_0$-Norm Regularization and Weight Pruning$ On the Compression of Neural Networks Using $\ell_0$ -Norm Regularization and Weight Pruning F. Oliveira E. Batista R. Seara 20 9 0 10 Sep 2021
Block Pruning For Faster Transformers François Lagunas Ella Charlaix Victor Sanh Alexander M. Rush VLM 33 219 0 10 Sep 2021
Survey of Low-Resource Machine Translation Barry Haddow Rachel Bawden Antonio Valerio Miceli Barone Jindvrich Helcl Alexandra Birch AIMat 49 150 0 01 Sep 2021
Layer-wise Model Pruning based on Mutual Information Chun Fan Jiwei Li Xiang Ao Fei Wu Yuxian Meng Xiaofei Sun 48 19 0 28 Aug 2021
An Information Theory-inspired Strategy for Automatic Network Pruning Xiawu Zheng Yuexiao Ma Teng Xi Gang Zhang Errui Ding Yuchao Li Jie Chen Yonghong Tian Rongrong Ji 54 13 0 19 Aug 2021
Differentiable Subset Pruning of Transformer Heads Jiaoda Li Ryan Cotterell Mrinmaya Sachan 45 54 0 10 Aug 2021
Learning a Neural Diff for Speech Models J. Macoskey Grant P. Strimel Ariya Rastrow 18 2 0 03 Aug 2021
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration Shiwei Liu Tianlong Chen Xiaohan Chen Zahra Atashgahi Lu Yin Huanyu Kou Li Shen Mykola Pechenizkiy Zhangyang Wang Decebal Constantin Mocanu 45 112 0 19 Jun 2021
A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness James Diffenderfer Brian Bartoldson Shreya Chaganti Jize Zhang B. Kailkhura OOD 31 69 0 16 Jun 2021
Can Subnetwork Structure be the Key to Out-of-Distribution Generalization? Dinghuai Zhang Kartik Ahuja Yilun Xu Yisen Wang Aaron Courville OOD 30 95 0 05 Jun 2021
Effective Sparsification of Neural Networks with Global Sparsity Constraint Xiao Zhou Weizhong Zhang Hang Xu Tong Zhang 21 61 0 03 May 2021
Do All MobileNets Quantize Poorly? Gaining Insights into the Effect of Quantization on Depthwise Separable Convolutional Networks Through the Eyes of Multi-scale Distributional Dynamics S. Yun Alexander Wong MQ 27 25 0 24 Apr 2021
Partitioning sparse deep neural networks for scalable training and inference G. Demirci Hakan Ferhatosmanoglu 25 11 0 23 Apr 2021
Lottery Jackpots Exist in Pre-trained Models Yuxin Zhang Mingbao Lin Yan Wang Rongrong Ji Rongrong Ji 35 15 0 18 Apr 2021
Understanding Event-Generation Networks via Uncertainties Marco Bellagente Manuel Haussmann Michel Luchmann Tilman Plehn BDL 41 55 0 09 Apr 2021
Local Interpretations for Explainable Natural Language Processing: A Survey Siwen Luo Hamish Ivison S. Han Josiah Poon MILM 48 48 0 20 Mar 2021
Sample Complexity of Offline Reinforcement Learning with Deep ReLU Networks Thanh Nguyen-Tang Sunil R. Gupta Hung The Tran Svetha Venkatesh OffRL 70 7 0 11 Mar 2021
Quantization-Guided Training for Compact TinyML Models Sedigh Ghamari Koray Ozcan Thu Dinh A. Melnikov Juan Carvajal Jan Ernst S. Chai MQ 21 16 0 10 Mar 2021
hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices F. Fahim B. Hawks C. Herwig J. Hirschauer S. Jindariani ... J. Ngadiuba Miaoyuan Liu Duc Hoang E. Kreinar Zhenbin Wu 37 129 0 09 Mar 2021
Sparse Training Theory for Scalable and Efficient Agents Decebal Constantin Mocanu Elena Mocanu T. Pinto Selima Curci Phuong H. Nguyen M. Gibescu D. Ernst Z. Vale 45 17 0 02 Mar 2021
An Information-Theoretic Justification for Model Pruning Berivan Isik Tsachy Weissman Albert No 95 35 0 16 Feb 2021
Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks Itay Hubara Brian Chmiel Moshe Island Ron Banner S. Naor Daniel Soudry 59 111 0 16 Feb 2021
End-to-end learnable EEG channel selection for deep neural networks with Gumbel-softmax Thomas Strypsteen Alexander Bertrand 33 50 0 11 Feb 2021
SeReNe: Sensitivity based Regularization of Neurons for Structured Sparsity in Neural Networks Enzo Tartaglione Andrea Bragagnolo Francesco Odierna Attilio Fiandrotti Marco Grangetto 43 18 0 07 Feb 2021
TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models Chunxing Yin Bilge Acun Xing Liu Carole-Jean Wu 50 103 0 25 Jan 2021
Neural Pruning via Growing Regularization Huan Wang Can Qin Yulun Zhang Y. Fu 37 144 0 16 Dec 2020
Parameter-Efficient Transfer Learning with Diff Pruning Demi Guo Alexander M. Rush Yoon Kim 15 386 0 14 Dec 2020
Interpreting Neural Networks as Gradual Argumentation Frameworks (Including Proof Appendix) Nico Potyka AI4CE 41 49 0 10 Dec 2020
DiffPrune: Neural Network Pruning with Deterministic Approximate Binary Gates and $L_0$ Regularization Yaniv Shulman 46 3 0 07 Dec 2020