SNIP: Single-shot Network Pruning based on Connection Sensitivity

4 October 2018

Namhoon Lee

Thalaiyasingam Ajanthan

Philip Torr

VLM

ArXiv PDF HTML

Papers citing "SNIP: Single-shot Network Pruning based on Connection Sensitivity"

50 / 708 papers shown

Title
FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping Ajay Jaiswal Bodun Hu Lu Yin Yeonju Ro Shiwei Liu Tianlong Chen Aditya Akella 58 12 0 05 Apr 2024
Rethinking Pruning for Vision-Language Models: Strategies for Effective Sparsity and Performance Restoration Shwai He Ang Li Tianlong Chen VLM 62 1 0 03 Apr 2024
Accelerating Transformer Pre-training with 2:4 Sparsity Yuezhou Hu Kang Zhao Weiyu Huang Jianfei Chen Jun Zhu 65 7 0 02 Apr 2024
DRIVE: Dual Gradient-Based Rapid Iterative Pruning Dhananjay Saikumar Blesson Varghese 30 0 0 01 Apr 2024
TG-NAS: Leveraging Zero-Cost Proxies with Transformer and Graph Convolution Networks for Efficient Neural Architecture Search Ye Qiao Haocheng Xu Sitao Huang 33 0 0 30 Mar 2024
Separate, Dynamic and Differentiable (SMART) Pruner for Block/Output Channel Pruning on Computer Vision Tasks Guanhua Ding Zexi Ye Zhen Zhong Gang Li David Shao 44 0 0 29 Mar 2024
AZ-NAS: Assembling Zero-Cost Proxies for Network Architecture Search Junghyup Lee Bumsub Ham 32 6 0 28 Mar 2024
Dense Vision Transformer Compression with Few Samples Hanxiao Zhang Yifan Zhou Guo-Hua Wang Jianxin Wu ViT VLM 39 1 0 27 Mar 2024
LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernels Tuo Feng Wenguan Wang Fan Ma Yi Yang 3DV 40 6 0 22 Mar 2024
Evaluating Unsupervised Dimensionality Reduction Methods for Pretrained Sentence Embeddings Gaifan Zhang Yi Zhou Danushka Bollegala 44 4 0 20 Mar 2024
LNPT: Label-free Network Pruning and Training Jinying Xiao Ping Li Zhe Tang Jie Nie 38 2 0 19 Mar 2024
SEVEN: Pruning Transformer Model by Reserving Sentinels Jinying Xiao Ping Li Jie Nie Zhe Tang 39 3 0 19 Mar 2024
Boosting Order-Preserving and Transferability for Neural Architecture Search: a Joint Architecture Refined Search and Fine-tuning Approach Beichen Zhang Xiaoxing Wang Xiaohan Qin Junchi Yan 35 4 0 18 Mar 2024
Anytime Neural Architecture Search on Tabular Data Naili Xing Shaofeng Cai Zhaojing Luo Bengchin Ooi Jian Pei 34 1 0 15 Mar 2024
FOCIL: Finetune-and-Freeze for Online Class Incremental Learning by Training Randomly Pruned Sparse Experts Murat Onur Yildirim Elif Ceren Gok Yildirim Decebal Constantin Mocanu Joaquin Vanschoren CLL 44 0 0 13 Mar 2024
Maxwell's Demon at Work: Efficient Pruning by Leveraging Saturation of Neurons Simon Dufort-Labbé P. DÓro Evgenii Nikishin Razvan Pascanu Pierre-Luc Bacon A. Baratin 45 1 0 12 Mar 2024
Robustifying and Boosting Training-Free Neural Architecture Search Zhenfeng He Yao Shu Zhongxiang Dai K. H. Low 43 1 0 12 Mar 2024
LookupFFN: Making Transformers Compute-lite for CPU inference Zhanpeng Zeng Michael Davies Pranav Pulijala Karthikeyan Sankaralingam Vikas Singh 38 5 0 12 Mar 2024
FALCON: FLOP-Aware Combinatorial Optimization for Neural Network Pruning Xiang Meng Wenyu Chen Riade Benbaki Rahul Mazumder 28 3 0 11 Mar 2024
ECToNAS: Evolutionary Cross-Topology Neural Architecture Search Elisabeth J. Schiessler R. Aydin C. Cyron 34 0 0 08 Mar 2024
SWAP-NAS: Sample-Wise Activation Patterns for Ultra-fast NAS Yameng Peng Andy Song Haytham M. Fayek Victor Ciesielski Xiaojun Chang 37 4 0 07 Mar 2024
Encodings for Prediction-based Neural Architecture Search Yash Akhauri Mohamed S. Abdelfattah AI4CE 41 3 0 04 Mar 2024
Structurally Prune Anything: Any Architecture, Any Framework, Any Time Xun Wang John Rachwan Stephan Günnemann Bertrand Charpentier 41 4 0 03 Mar 2024
"Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approach Lingyu Gu Yongqiang Du Yuan Zhang Di Xie Shiliang Pu Robert C. Qiu Zhenyu Liao 41 6 0 01 Mar 2024
Masks, Signs, And Learning Rate Rewinding Advait Gadhikar R. Burkholz 57 8 0 29 Feb 2024
NeuroPrune: A Neuro-inspired Topological Sparse Training Algorithm for Large Language Models Amit Dhurandhar Tejaswini Pedapati Ronny Luss Soham Dan Aurélie C. Lozano Payel Das Georgios Kollias 22 3 0 28 Feb 2024
Purified and Unified Steganographic Network Guobiao Li Sheng Li Zicong Luo Zhenxing Qian Xinpeng Zhang 22 5 0 27 Feb 2024
Deep Neural Network Initialization with Sparsity Inducing Activations Ilan Price Nicholas Daultry Ball Samuel C.H. Lam Adam C. Jones Jared Tanner AI4CE 31 1 0 25 Feb 2024
Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks Using the Marginal Likelihood Rayen Dhahri Alexander Immer Bertrand Charpentier Stephan Günnemann Vincent Fortuin BDL UQCV 35 4 0 25 Feb 2024
Differentially Private Zeroth-Order Methods for Scalable Large Language Model Finetuning Zhicheng Liu Jian Lou Wenxuan Bao Yihan Hu Baochun Li Zhan Qin K. Ren 37 7 0 12 Feb 2024
Bayesian Deep Learning Via Expectation Maximization and Turbo Deep Approximate Message Passing Wei Xu An Liu Yiting Zhang Vincent Lau BDL 25 1 0 12 Feb 2024
Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications Boyi Wei Kaixuan Huang Yangsibo Huang Tinghao Xie Xiangyu Qi Mengzhou Xia Prateek Mittal Mengdi Wang Peter Henderson AAML 63 84 0 07 Feb 2024
Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers Abhimanyu Bambhaniya Amir Yazdanbakhsh Suvinay Subramanian Sheng-Chun Kao Shivani Agrawal Utku Evci Tushar Krishna 54 16 0 07 Feb 2024
A Survey on Transformer Compression Yehui Tang Yunhe Wang Jianyuan Guo Zhijun Tu Kai Han Hailin Hu Dacheng Tao 41 29 0 05 Feb 2024
Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitate Can Jin Tong Che Hongwu Peng Yiyuan Li Dimitris N. Metaxas Marco Pavone 44 43 0 05 Feb 2024
ParZC: Parametric Zero-Cost Proxies for Efficient NAS Peijie Dong Lujun Li Xinglin Pan Zimian Wei Xiang Liu Qiang-qiang Wang Xiaowen Chu 66 3 0 03 Feb 2024
From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers Bharat Runwal Tejaswini Pedapati Pin-Yu Chen MoE 49 5 0 02 Feb 2024
Evaluating UAV Path Planning Algorithms for Realistic Maritime Search and Rescue Missions Martin Messmer Andreas Zell 17 3 0 02 Feb 2024
Two Heads Are Better Than One: Boosting Graph Sparse Training via Semantic and Topological Awareness Guibin Zhang Xinfeng Li Kun Wang Junfeng Fang Yongduo Sui Kai Wang Keli Zhang Dawei Cheng Shirui Pan Tianlong Chen 27 10 0 02 Feb 2024
No Free Prune: Information-Theoretic Barriers to Pruning at Initialization Tanishq Kumar Kevin Luo Mark Sellke 38 3 0 02 Feb 2024
EPSD: Early Pruning with Self-Distillation for Efficient Model Compression Dong Chen Ning Liu Yichen Zhu Zhengping Che Rui Ma Fachao Zhang Xiaofeng Mou Yi Chang Jian Tang 31 4 0 31 Jan 2024
ELRT: Efficient Low-Rank Training for Compact Convolutional Neural Networks Yang Sui Miao Yin Yu Gong Jinqi Xiao Huy Phan Bo Yuan 33 5 0 18 Jan 2024
Model Compression Techniques in Biometrics Applications: A Survey Eduarda Caldeira Pedro C. Neto Marco Huber Naser Damer Ana F. Sequeira 40 11 0 18 Jan 2024
Always-Sparse Training by Growing Connections with Guided Stochastic Exploration Mike Heddes Narayan Srinivasa T. Givargis Alexandru Nicolau 91 0 0 12 Jan 2024
Scissorhands: Scrub Data Influence via Connection Sensitivity in Networks Jing Wu Mehrtash Harandi 36 14 0 11 Jan 2024
Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization K. Balaskas Andreas Karatzas Christos Sad K. Siozios Iraklis Anagnostopoulos Georgios Zervakis Jörg Henkel MQ 41 10 0 23 Dec 2023
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment Lingling Xu Haoran Xie S. J. Qin Xiaohui Tao F. Wang 55 135 0 19 Dec 2023
Split-Ensemble: Efficient OOD-aware Ensemble via Task and Model Splitting Anthony Chen Huanrui Yang Yulu Gan Denis A. Gudovskiy Zhen Dong Haofan Wang Tomoyuki Okuno Yohei Nakata Kurt Keutzer Shanghang Zhang 34 2 0 14 Dec 2023
MaxQ: Multi-Axis Query for N:M Sparsity Network Jingyang Xiang Siqi Li Junhao Chen Zhuangzhi Chen Tianxin Huang Linpeng Peng Yong-Jin Liu 18 0 0 12 Dec 2023
Evaluating the Utility of Model Explanations for Model Development Shawn Im Jacob Andreas Yilun Zhou XAI FAtt ELM 27 1 0 10 Dec 2023