To prune, or not to prune: exploring the efficacy of pruning for model compression

5 October 2017

Papers citing "To prune, or not to prune: exploring the efficacy of pruning for model compression"

50 / 265 papers shown

Title
Maestro: Uncovering Low-Rank Structures via Trainable Decomposition Samuel Horváth Stefanos Laskaridis Shashank Rajput Hongyi Wang BDL 37 4 0 28 Aug 2023
Neural Networks at a Fraction with Pruned Quaternions Sahel Mohammad Iqbal Subhankar Mishra 33 4 0 13 Aug 2023
A Simple and Effective Pruning Approach for Large Language Models Mingjie Sun Zhuang Liu Anna Bair J. Zico Kolter 90 361 0 20 Jun 2023
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation Yixiao Li Yifan Yu Qingru Zhang Chen Liang Pengcheng He Weizhu Chen Tuo Zhao 44 69 0 20 Jun 2023
Spatial Re-parameterization for N:M Sparsity Yuxin Zhang Mingbao Lin Mingliang Xu Yonghong Tian Rongrong Ji 46 2 0 09 Jun 2023
Magnitude Attention-based Dynamic Pruning Jihye Back Namhyuk Ahn Jang-Hyun Kim 43 2 0 08 Jun 2023
Federated Graph Learning for Low Probability of Detection in Wireless Ad-Hoc Networks S. Krishnan Jihong Park S. Sagar Gregory Sherman Benjamin Campbell Jinho Choi 21 4 0 01 Jun 2023
Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers Zahra Atashgahi Mykola Pechenizkiy Raymond N. J. Veldhuis Decebal Constantin Mocanu AI4TS AI4CE 34 1 0 28 May 2023
Constrained Probabilistic Mask Learning for Task-specific Undersampled MRI Reconstruction Tobias Weber Michael Ingrisch Bernd Bischl David Rügamer 32 2 0 25 May 2023
Combining Multi-Objective Bayesian Optimization with Reinforcement Learning for TinyML M. Deutel G. Kontes Christopher Mutschler Jürgen Teich 57 0 0 23 May 2023
TinyissimoYOLO: A Quantized, Low-Memory Footprint, TinyML Object Detection Network for Low Power Microcontrollers Julian Moosmann Marco Giordano Christian Vogt Michele Magno MQ ObjD 23 20 0 22 May 2023
Self-Distillation with Meta Learning for Knowledge Graph Completion Yunshui Li Junhao Liu Chengming Li Min Yang 29 5 0 20 May 2023
Accelerator-Aware Training for Transducer-Based Speech Recognition Suhaila M. Shakiah R. Swaminathan Hieu Duy Nguyen Raviteja Chinta Tariq Afzal Nathan Susanj Athanasios Mouchtaris Grant P. Strimel Ariya Rastrow 24 1 0 12 May 2023
Cuttlefish: Low-Rank Model Training without All the Tuning Hongyi Wang Saurabh Agarwal Pongsakorn U-chupala Yoshiki Tanaka Eric P. Xing Dimitris Papailiopoulos OffRL 63 22 0 04 May 2023
Application of Transformers for Nonlinear Channel Compensation in Optical Systems Behnam Behinaein Hamgini H. Najafi Ali Bakhshali Zhuhong Zhang 31 1 0 25 Apr 2023
Identifying Appropriate Intellectual Property Protection Mechanisms for Machine Learning Models: A Systematization of Watermarking, Fingerprinting, Model Access, and Attacks Isabell Lederer Rudolf Mayer Andreas Rauber 29 19 0 22 Apr 2023
STen: Productive and Efficient Sparsity in PyTorch Andrei Ivanov Nikoli Dryden Tal Ben-Nun Saleh Ashkboos Torsten Hoefler 39 4 0 15 Apr 2023
DIPNet: Efficiency Distillation and Iterative Pruning for Image Super-Resolution Lei Yu Xinpeng Li Youwei Li Ting Jiang Qi Wu Haoqiang Fan Shuaicheng Liu SupR 39 25 0 14 Apr 2023
Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference Tao Lei Junwen Bai Siddhartha Brahma Joshua Ainslie Kenton Lee ... Vincent Zhao Yuexin Wu Bo-wen Li Yu Zhang Ming-Wei Chang BDL AI4CE 30 55 0 11 Apr 2023
Dual-Attention Neural Transducers for Efficient Wake Word Spotting in Speech Recognition Saumya Yashmohini Sahai Jing Liu Thejaswi Muniyappa Kanthashree Mysore Sathyendra Anastasios Alexandridis ... Ross McGowan Ariya Rastrow Feng-Ju Chang Athanasios Mouchtaris Siegfried Kunzmann 44 5 0 03 Apr 2023
Factorizers for Distributed Sparse Block Codes Michael Hersche Aleksandar Terzić G. Karunaratne Jovin Langenegger Angeline Pouget G. Cherubini Luca Benini Abu Sebastian Abbas Rahimi 39 4 0 24 Mar 2023
Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency Vithursan Thangarasa Shreyas Saxena Abhay Gupta Sean Lie 41 3 0 21 Mar 2023
Induced Feature Selection by Structured Pruning Nathan Hubens V. Delvigne M. Mancas B. Gosselin Marius Preda T. Zaharia 22 0 0 20 Mar 2023
Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! Shiwei Liu Tianlong Chen Zhenyu Zhang Xuxi Chen Tianjin Huang Ajay Jaiswal Zhangyang Wang 37 29 0 03 Mar 2023
Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection Zhen Cheng Fei Zhu Xu-Yao Zhang Cheng-Lin Liu MoMe OODD 45 11 0 02 Mar 2023
Fast as CHITA: Neural Network Pruning with Combinatorial Optimization Riade Benbaki Wenyu Chen X. Meng Hussein Hazimeh Natalia Ponomareva Zhe Zhao Rahul Mazumder 21 26 0 28 Feb 2023
A Unified Framework for Soft Threshold Pruning Yanqing Chen Zhengyu Ma Wei Fang Xiawu Zheng Zhaofei Yu Yonghong Tian 88 19 0 25 Feb 2023
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers Chen Liang Haoming Jiang Zheng Li Xianfeng Tang Bin Yin Tuo Zhao VLM 32 24 0 19 Feb 2023
High-frequency Matters: An Overwriting Attack and defense for Image-processing Neural Network Watermarking Huajie Chen Tianqing Zhu Chi Liu Shui Yu Wanlei Zhou AAML 29 3 0 17 Feb 2023
SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks Mahdi Nikdan Tommaso Pegolotti Eugenia Iofinova Eldar Kurtic Dan Alistarh 26 11 0 09 Feb 2023
What Matters In The Structured Pruning of Generative Language Models? Michael Santacroce Zixin Wen Yelong Shen Yuan-Fang Li 28 33 0 07 Feb 2023
Certified Invertibility in Neural Networks via Mixed-Integer Programming Tianqi Cui Tom S. Bertalan George J. Pappas M. Morari Ioannis G. Kevrekidis Mahyar Fazlyab AAML 27 2 0 27 Jan 2023
GOHSP: A Unified Framework of Graph and Optimization-based Heterogeneous Structured Pruning for Vision Transformer Miao Yin Burak Uzkent Yilin Shen Hongxia Jin Bo Yuan ViT 32 13 0 13 Jan 2023
Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning Huan Wang Can Qin Yue Bai Yun Fu 37 20 0 12 Jan 2023
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot Elias Frantar Dan Alistarh VLM 38 643 0 02 Jan 2023
Comparative Study of Parameter Selection for Enhanced Edge Inference for a Multi-Output Regression model for Head Pose Estimation A. Lindamulage N. Kodagoda Shyam Reyal Pradeepa Samarasinghe P. Yogarajah CVBM 21 0 0 28 Dec 2022
COLT: Cyclic Overlapping Lottery Tickets for Faster Pruning of Convolutional Neural Networks Md. Ismail Hossain Mohammed Rakib M. M. L. Elahi Nabeel Mohammed Shafin Rahman 21 1 0 24 Dec 2022
Pruning On-the-Fly: A Recoverable Pruning Method without Fine-tuning Danyang Liu Xue Liu 28 0 0 24 Dec 2022
CarFi: Rider Localization Using Wi-Fi CSI Sirajum Munir Hongkai Chen Shiwei Fang Mahathir Monjur Shan Lin S. Nirjon 22 3 0 21 Dec 2022
Constructing Organism Networks from Collaborative Self-Replicators Steffen Illium Maximilian Zorn Cristian Lenta Michael Kolle Claudia Linnhoff-Popien Thomas Gabor 21 0 0 20 Dec 2022
Gradient-based Intra-attention Pruning on Pre-trained Language Models Ziqing Yang Yiming Cui Xin Yao Shijin Wang VLM 42 8 0 15 Dec 2022
PD-Quant: Post-Training Quantization based on Prediction Difference Metric Jiawei Liu Lin Niu Zhihang Yuan Dawei Yang Xinggang Wang Wenyu Liu MQ 98 70 0 14 Dec 2022
AP: Selective Activation for De-sparsifying Pruned Neural Networks Shiyu Liu Rohan Ghosh Dylan Tan Mehul Motani AAML 26 0 0 09 Dec 2022
A Rubric for Human-like Agents and NeuroAI Ida Momennejad 60 14 0 08 Dec 2022
Efficient Stein Variational Inference for Reliable Distribution-lossless Network Pruning Yingchun Wang Song Guo Jingcai Guo Weizhan Zhang Yi Tian Xu Jiewei Zhang Yi Liu 23 17 0 07 Dec 2022
You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets Tianjin Huang Tianlong Chen Meng Fang Vlado Menkovski Jiaxu Zhao ... Yulong Pei Decebal Constantin Mocanu Zhangyang Wang Mykola Pechenizkiy Shiwei Liu GNN 52 14 0 28 Nov 2022
Continual Learning of Neural Machine Translation within Low Forgetting Risk Regions Shuhao Gu Bojie Hu Yang Feng CLL 44 13 0 03 Nov 2022
LOFT: Finding Lottery Tickets through Filter-wise Training Qihan Wang Chen Dun Fangshuo Liao C. Jermaine Anastasios Kyrillidis 33 3 0 28 Oct 2022
Gradient-based Weight Density Balancing for Robust Dynamic Sparse Training Mathias Parger Alexander Ertl Paul Eibensteiner J. H. Mueller Martin Winter M. Steinberger 34 0 0 25 Oct 2022
On the optimization and pruning for Bayesian deep learning X. Ke Yanan Fan BDL UQCV 40 1 0 24 Oct 2022